summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/sql/base.py
Commit message (Collapse)AuthorAgeFilesLines
* Improve oracle index reflectionFederico Caselli2023-04-281-7/+13
| | | | | | | | Added reflection support in the Oracle dialect to expression based indexes and the ordering direction of index expressions. Fixes: #9597 Change-Id: I40e163496789774e9930f46823d2208c35eab6f8
* support slice access for .cMike Bayer2023-04-221-5/+17
| | | | | | | | | | | | Added support for slice access with :class:`.ColumnCollection`, e.g. ``table.c[0:5]``, ``subquery.c[:-1]`` etc. Slice access returns a sub :class:`.ColumnCollection` in the same way as passing a tuple of keys. This is a natural continuation of the key-tuple access added for :ticket:`8285`, which it appears to be an oversight that this usage was omitted. Change-Id: I6378642f39501ffbbae4acadf1dc38a43c39d722 References: #8285 References: #9690
* add deterministic imv returning ordering using sentinel columnsMike Bayer2023-04-211-0/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Repaired a major shortcoming which was identified in the :ref:`engine_insertmanyvalues` performance optimization feature first introduced in the 2.0 series. This was a continuation of the change in 2.0.9 which disabled the SQL Server version of the feature due to a reliance in the ORM on apparent row ordering that is not guaranteed to take place. The fix applies new logic to all "insertmanyvalues" operations, which takes effect when a new parameter :paramref:`_dml.Insert.returning.sort_by_parameter_order` on the :meth:`_dml.Insert.returning` or :meth:`_dml.UpdateBase.return_defaults` methods, that through a combination of alternate SQL forms, direct correspondence of client side parameters, and in some cases downgrading to running row-at-a-time, will apply sorting to each batch of returned rows using correspondence to primary key or other unique values in each row which can be correlated to the input data. Performance impact is expected to be minimal as nearly all common primary key scenarios are suitable for parameter-ordered batching to be achieved for all backends other than SQLite, while "row-at-a-time" mode operates with a bare minimum of Python overhead compared to the very heavyweight approaches used in the 1.x series. For SQLite, there is no difference in performance when "row-at-a-time" mode is used. It's anticipated that with an efficient "row-at-a-time" INSERT with RETURNING batching capability, the "insertmanyvalues" feature can be later be more easily generalized to third party backends that include RETURNING support but not necessarily easy ways to guarantee a correspondence with parameter order. Fixes: #9618 References: #9603 Change-Id: I1d79353f5f19638f752936ba1c35e4dc235a8b7c
* Merge "Remove old versionadded and versionchanged" into mainmike bayer2023-04-121-7/+0
|\
| * Remove old versionadded and versionchangedFederico Caselli2023-04-121-7/+0
| | | | | | | | | | | | | | Removed versionadded and versionchanged for version prior to 1.2 since they are no longer useful. Change-Id: I5c53d1188bc5fec3ab4be39ef761650ed8fa6d3e
* | establish column_property and query_expression as readonly from a dc perspectiveMike Bayer2023-04-121-0/+3
|/ | | | | | | | | | | | | | | | | | | | | | | | | | Fixed bug in ORM Declarative Dataclasses where the :func:`_orm.queryable_attribute` and :func:`_orm.column_property` constructs, which are documented as read-only constructs in the context of a Declarative mapping, could not be used with a :class:`_orm.MappedAsDataclass` class without adding ``init=False``, which in the case of :func:`_orm.queryable_attribute` was not possible as no ``init`` parameter was included. These constructs have been modified from a dataclass perspective to be assumed to be "read only", setting ``init=False`` by default and no longer including them in the pep-681 constructor. The dataclass parameters for :func:`_orm.column_property` ``init``, ``default``, ``default_factory``, ``kw_only`` are now deprecated; these fields don't apply to :func:`_orm.column_property` as used in a Declarative dataclasses configuration where the construct would be read-only. Also added read-specific parameter :paramref:`_orm.queryable_attribute.compare` to :func:`_orm.queryable_attribute`; :paramref:`_orm.queryable_attribute.repr` was already present. Added missing :paramref:`_orm.mapped_column.active_history` parameter to :func:`_orm.mapped_column` construct. Fixes: #9628 Change-Id: I2ab44d6b763b20410bd1ebb5ac949a6d223f1ce2
* Remove `typing.Self` workaroundYurii Karabas2023-02-081-24/+16
| | | | | | | | | | | | Remove ``typing.Self`` workaround, now using :pep:`673` for most methods that return ``Self``. Pull request courtesy Yurii Karabas. Fixes: #9254 Closes: #9255 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9255 Pull-request-sha: 2947df8ada79f5c3afe9c838e65993302199c2f7 Change-Id: Ic32015ad52e95a61f3913d43ea436aa9402804df
* Type annotations for sqlalchemy.sql.selectableDzmitar2023-01-131-3/+6
| | | | | | | | | Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Closes: #9028 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9028 Pull-request-sha: e2f8ddeac0b08feaad917285e988acf1e9465a26 Change-Id: I5caad31bfeeed2d224657a55f067ba1d86b8733f
* happy new year 2023Mike Bayer2023-01-031-1/+1
| | | | Change-Id: I625af65b3fb1815b1af17dc2ef47dd697fdc3fb1
* disallow same-named columns, unchecked replacement in TableMike Bayer2022-12-041-2/+9
| | | | | | | | | | | | | | | Fixed issue where table reflection using :paramref:`.Table.extend_existing` would fail to deduplicate a same-named column if the existing :class:`.Table` used a separate key. The :paramref:`.Table.autoload_replace` parameter would allow the column to be skipped but under no circumstances should a :class:`.Table` ever have the same-named column twice. Additionally, changed deprecation warnings to exceptions as were implemented in I1d58c8ebe081079cb669e7ead60886ffc1b1a7f5 . Fixes: #8925 Change-Id: I83d0f8658177a7ffbb06e01dbca91377d1a98d49
* update for mypy 1.0 devFederico Caselli2022-11-291-0/+8
| | | | | | | | | | | | | | | | As I need dmypy to work without facing [1], I am running the latest build of mypy which seems so far to finally not have that issue. update constructs that latest mypy is being more picky about, including better typing for the _NONE_NAME symbol used in constraints (porting those elements from the Enum patch at I15ac3daee770408b5795746f47c1bbd931b7d26d) [1] https://github.com/python/mypy/issues/12744 Change-Id: Ib3f56787fa65ea9bb2e6a0bccc4d99f54c516dad
* Try running pyupgrade on the codeFederico Caselli2022-11-161-5/+3
| | | | | | | | command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>" pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not exists in sqlalchemy fixtures Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
* perf improvements related to corresponding_column (2)Mike Bayer2022-11-151-72/+172
| | | | | | | | | | | | | | | | | commit two of two. this reorganizes ColumnCollection to build a new index up front that's used to optimize the corresponding_column() method. Additional performance enhancements within ORM-enabled SQL statements, specifically targeting callcounts within the construction of ORM statements, using combinations of :func:`_orm.aliased` with :func:`_sql.union` and similar "compound" constructs, in addition to direct performance improvements to the ``corresponding_column()`` internal method that is used heavily by the ORM by constructs like :func:`_orm.aliased` and similar. Fixes: #8796 Change-Id: I4a76788007d5a802b9a4081e6a0f6e4b52497b50
* Improve typings of execution optionsFederico Caselli2022-11-021-7/+37
| | | | | Fixes: #8605 Change-Id: I4aec83b9f321462427c3f4ac941c3b272255c088
* New ORM Query Guide featuring DML supportMike Bayer2022-09-251-1/+1
| | | | | | | | | | | | | | | | | reviewers: these docs publish periodically at: https://docs.sqlalchemy.org/en/gerrit/4042/orm/queryguide/index.html See the "last generated" timestamp near the bottom of the page to ensure the latest version is up Change includes some other adjustments: * small typing fixes for end-user benefit * removal of a bunch of old examples for patterns that nobody uses or aren't really what we promote now * modernization of some examples, including inheritance Change-Id: I9929daab7797be9515f71c888b28af1209e789ff
* implement tuple-slices from .c collectionsMike Bayer2022-08-011-26/+60
| | | | | | | | | | | | Added new syntax to the ``.c`` collection on all :class:`.FromClause` objects allowing tuples of keys to be passed to ``__getitem__()``, along with support for ``select()`` handling of ``.c`` collections directly, allowing the syntax ``select(table.c['a', 'b', 'c'])`` to be possible. The sub-collection returned is itself a :class:`.ColumnCollection` which is also directly consumable by :func:`_sql.select` and similar now. Fixes: #8285 Change-Id: I2236662c477ffc50af079310589e213323c960d1
* rearchitect reflection for batched performanceFederico Caselli2022-06-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Rearchitected the schema reflection API to allow some dialects to make use of high performing batch queries to reflect the schemas of many tables at once using much fewer queries. The new performance features are targeted first at the PostgreSQL and Oracle backends, and may be applied to any dialect that makes use of SELECT queries against system catalog tables to reflect tables (currently this omits the MySQL and SQLite dialects which instead make use of parsing the "CREATE TABLE" statement, however these dialects do not have a pre-existing performance issue with reflection. MS SQL Server is still a TODO). The new API is backwards compatible with the previous system, and should require no changes to third party dialects to retain compatibility; third party dialects can also opt into the new system by implementing batched queries for schema reflection. Along with this change is an updated reflection API that is fully :pep:`484` typed, features many new methods and some changes. Fixes: #4379 Change-Id: I897ec09843543aa7012bcdce758792ed3d415d08
* fix race conditions in lambda statementsMike Bayer2022-06-081-1/+2
| | | | | | | | | | | | | | | | | | Fixed multiple observed race conditions related to :func:`.lambda_stmt`, including an initial "dogpile" issue when a new Python code object is initially analyzed among multiple simultaneous threads which created both a performance issue as well as some internal corruption of state. Additionally repaired observed race condition which could occur when "cloning" an expression construct that is also in the process of being compiled or otherwise accessed in a different thread due to memoized attributes altering the ``__dict__`` while iterated, for Python versions prior to 3.10; in particular the lambda SQL construct is sensitive to this as it holds onto a single statement object persistently. The iteration has been refined to use ``dict.copy()`` with or without an additional iteration instead. Fixes: #8098 Change-Id: I4e0b627bfa187f1780dc68ec81b94db1c78f846a
* revenge of pep 484Mike Bayer2022-05-151-0/+22
| | | | | | trying to get remaining must-haves for ORM Change-Id: I66a3ecbbb8e5ba37c818c8a92737b576ecf012f7
* pep484: attributes and relatedMike Bayer2022-05-031-1/+3
| | | | | | | also implements __slots__ for QueryableAttribute, InstrumentedAttribute, Relationship.Comparator. Change-Id: I47e823160706fc35a616f1179a06c7864089e5b5
* inline mypy config; files ignoring type errors for the momentMike Bayer2022-04-281-0/+1
| | | | | | | | | | | | | | | | | | | to simplify pyproject.toml change the remaining files that aren't going to be typed on this first pass (unless of course someone wants to type some of these) to include # mypy: ignore-errors. for the moment, only a handful of ORM modules are to have more type checking implemented. It's important that ignore-errors is used and not "# type: ignore", as in the latter case, mypy doesn't even read the existing types in the file, which makes it impossible to type any files that refer to those modules at all. to simplify ongoing typing work use inline mypy config for remaining files that are "done" for now, indicating the level of type checking they currently have. Change-Id: I98669c1a305c2f0adba85d10b5425541f3fe9533
* pep484 ORM / SQL result supportMike Bayer2022-04-271-6/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | after some experimentation it seems mypy is more amenable to the generic types being fully integrated rather than having separate spin-off types. so key structures like Result, Row, Select become generic. For DML Insert, Update, Delete, these are spun into type-specific subclasses ReturningInsert, ReturningUpdate, ReturningDelete, which is fine since the "row-ness" of these constructs doesn't happen until returning() is called in any case. a Tuple based model is then integrated so that these objects can carry along information about their return types. Overloads at the .execute() level carry through the Tuple from the invoked object to the result. To suit the issue of AliasedClass generating attributes that are dynamic, experimented with a custom subclass AsAliased, but then just settled on having aliased() lie to the type checker and return `Type[_O]`, essentially. will need some type-related accessors for with_polymorphic() also. Additionally, identified an issue in Update when used "mysql style" against a join(), it basically doesn't work if asked to UPDATE two tables on the same column name. added an error message to the specific condition where it happens with a very non-specific error message that we hit a thing we can't do right now, suggest multi-table update as a possible cause. Change-Id: I5eff7eefe1d6166ee74160b2785c5e6a81fa8b95
* pep-484: ORM public API, constructorsMike Bayer2022-04-201-10/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | for the moment, abandoning using @overload with relationship() and mapped_column(). The overloads are very difficult to get working at all, and the overloads that were there all wouldn't pass on mypy. various techniques of getting them to "work", meaning having right hand side dictate what's legal on the left, have mixed success and wont give consistent results; additionally, it's legal to have Optional / non-optional independent of nullable in any case for columns. relationship cases are less ambiguous but mypy was not going along with things. we have a comprehensive system of allowing left side annotations to drive the right side, in the absense of explicit settings on the right. so type-centric SQLAlchemy will be left-side driven just like dataclasses, and the various flags and switches on the right side will just not be needed very much. in other matters, one surprise, forgot to remove string support from orm.join(A, B, "somename") or do deprecations for it in 1.4. This is a really not-directly-used structure barely mentioned in the docs for many years, the example shows a relationship being used, not a string, so we will just change it to raise the usual error here. Change-Id: Iefbbb8d34548b538023890ab8b7c9a5d9496ec6e
* mypy .950 updatesMike Bayer2022-04-171-5/+11
| | | | | Fixes: #7942 Change-Id: Ice1243e1704e88bb8fa13fb0d1f8e24dcd94bfd4
* pep484: schema APIMike Bayer2022-04-151-35/+59
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | implement strict typing for schema.py this module has lots of public API, lots of old decisions and very hard to follow construction sequences in many cases, and is also where we get a lot of new feature requests, so strict typing should help keep things clean. among improvements here, fixed the pool .info getters and also figured out how to get ColumnCollection and related to be covariant so that we may set them up as returning Column or ColumnClause without any conflicts. DDL was affected, noting that superclasses of DDLElement (_DDLCompiles, added recently) can now be passed into "ddl_if" callables; reorganized ddl into ExecutableDDLElement as a new name for DDLElement and _DDLCompiles renamed to BaseDDLElement. setting up strict also located an API use case that is completely broken, which is connection.execute(some_default) returns a scalar value. This case has been deprecated and new paths have been set up so that connection.scalar() may be used. This likely wasn't possible in previous versions because scalar() would assume a CursorResult. The scalar() change also impacts Session as we have explicit support (since someone had reported it as a regression) for session.execute(Sequence()) to work. They will get the same deprecation message (which omits the word "Connection", just uses ".execute()" and ".scalar()") and they can then use Session.scalar() as well. Getting this to type correctly while still supporting ORM use cases required some refactoring, and I also set up a keyword only delimeter for Session.execute() and related as execution_options / bind_arguments should always be keyword only, applied these changes to AsyncSession as well. Additionally simpify Table __init__ now that we are Python 3 only, we can have positional plus explicit kwargs finally. Simplify Column.__init__ as well again taking advantage of kw only arguments. Fill in most/all __init__ methods in sqltypes.py as the constructor for types is most of the API. should likely do this for dialect-specific types as well. Apply _InfoType for all info attributes as should have been done originally and update descriptor decorators. Change-Id: I3f9f8ff3f1c8858471ff4545ac83d68c88107527
* Don't pass vistor to immutables in cloned traverseMike Bayer2022-04-121-1/+11
| | | | | | | | | | Saw someone using cloned_traverse to move columns around (changing their .table) and not surprisingly having poor results. As cloned traversal is to provide a hook for in-place mutation of elements, it should not be given Immutable objects as these should not be changed once they are structurally composed. Change-Id: I43b22f52f243ef481a75d2cf5ecc73d50f110a81
* pep-484: asyncioMike Bayer2022-04-111-1/+2
| | | | | | | | | | | | | | | | | | | | | in this patch the asyncio/events.py module, which existed only to raise errors when trying to attach event listeners, is removed, as we were already coding an asyncio-specific workaround in upstream Pool / Session to raise this error, just moved the error out to the target and did the same thing for Engine. We also add an async_sessionmaker class. The initial rationale here is because sessionmaker() is hardcoded to Session subclasses, and there's not a way to get the use case of sessionmaker(class_=AsyncSession) to type correctly without changing the sessionmaker() symbol itself to be a function and not a class, which gets too complicated for what this is. Additionally, _SessionClassMethods has only three methods on it, one of which is not usable with asyncio (close_all()), the others not generally used from the session class. Change-Id: I064a5fa5d91cc8d5bbe9597437536e37b4e801fe
* pep-484: session, instancestate, etcMike Bayer2022-04-121-2/+6
| | | | | | | | Also adds some fixes to annotation-based mapping that have come up, as well as starts to add more pep-484 test cases Change-Id: Ia722bbbc7967a11b23b66c8084eb61df9d233fee
* update flake8 noqa skips with proper syntaxFederico Caselli2022-04-111-1/+1
| | | | Change-Id: I42ed77f559e3ee5b8c600d98457ee37803ef0ea6
* cx_Oracle modernizeMike Bayer2022-04-071-2/+2
| | | | | | | | | | | | | | | | | | | Full "RETURNING" support is implemented for the cx_Oracle dialect, meaning multiple RETURNING rows are now recived for DML statements that produce more than one row for RETURNING. cx_Oracle 7 is now the minimum version for cx_Oracle. Getting Oracle to do multirow returning took about 5 minutes. however, getting Oracle's RETURNING system to integrate with ORM-enabled insert, update, delete, is a big deal because that architecture wasn't really working very robustly, including some recent changes in 1.4 for FromStatement were done in a hurry, so this patch also cleans up the FromStatement situation and begins to establish it more concretely as the base for all ReturnsRows / TextClause ORM scenarios. Fixes: #6245 Change-Id: I2b4e6007affa51ce311d2d5baa3917f356ab961f
* pep484 - sql.selectableMike Bayer2022-04-041-5/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the pep484 task becomes more intense as there is mounting pressure to come up with a consistency in how data moves from end-user to instance variable. current thinking is coming into: 1. there are _typing._XYZArgument objects that represent "what the user sent" 2. there's the roles, which represent a kind of "filter" for different kinds of objects. These are mostly important as the argument we pass to coerce(). 3. there's the thing that coerce() returns, which should be what the construct uses as its internal representation of the thing. This is _typing._XYZElement. but there's some controversy over whether or not we should pass actual ClauseElements around by their role or not. I think we shouldn't at the moment, but this makes the "role-ness" of something a little less portable. Like, we have to set DMLTableRole for TableClause, Join, and Alias, but then also we have to repeat those three types in order to set up _DMLTableElement. Other change introduced here, there was a deannotate=True for the left/right of a sql.join(). All tests pass without that. I'd rather not have that there as if we have a join(A, B) where A, B are mapped classes, we want them inside of the _annotations. The rationale seems to be performance, but this performance can be illustrated to be on the compile side which we hope is cached in the normal case. CTEs now accommodate for text selects including recursive. Get typing to accommodate "util.preloaded" cleanly; add "preloaded" as a real module. This seemed like we would have needed pep562 `__getattr__()` but we don't, just set names in globals() as we import them. References: #6810 Change-Id: I34d17f617de2fe2c086fc556bd55748dc782faf0
* pep-484: the pep-484ening, SQL part threeMike Bayer2022-03-301-43/+67
| | | | | | | | | | | | | | | hitting DML which is causing us to open up the ColumnCollection structure a bit, as we do put anonymous column expressions with None here. However, we still want Table /TableClause to have named column collections that don't return None, so parametrize the "key" in this collection also. * rename some "immutable" elements to "readonly". we change the contents of immutablecolumncollection underneath, so it's not "immutable" Change-Id: I2593995a4e5c6eae874bed5bf76117198be8ae97
* pep484 - SQL internalsMike Bayer2022-03-241-96/+173
| | | | | | | non-strict checking for mostly internal or semi-internal code Change-Id: Ib91b47f1a8ccc15e666b94bad1ce78c4ab15b0ec
* pep 484 for typesMike Bayer2022-03-191-2/+4
| | | | | | | strict types type_api.py, including TypeDecorator, NativeForEmulated, etc. Change-Id: Ib2eba26de0981324a83733954cb7044a29bbd7db
* pep-484 - SQL column operationsMike Bayer2022-03-151-10/+27
| | | | | | | | | note we are taking out the ColumnOperartors[SQLCoreOperations] thing; not really clear why that was needed and at the moment it seems I was likely confused. Change-Id: I834b75f9b44f91b97e29f2e1a7b1029bd910e0a1
* pep-484: sqlalchemy.sql pass oneMike Bayer2022-03-131-6/+11
| | | | | | | | | | | | | | | | | | sqlalchemy.sql will require many passes to get all modules even gradually typed. Will have to pick and choose what modules can be strictly typed vs. which can be gradual. in this patch, emphasis is on visitors.py, cache_key.py, annotations.py for strict typing, compiler.py is on gradual typing but has much more structure, in particular where it connects with the outside world. The work within compiler.py also reached back out to engine/cursor.py , default.py quite a bit. References: #6810 Change-Id: I6e8a29f6013fd216e43d45091bc193f8be0368fd
* pep-484 for engineMike Bayer2022-03-011-2/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All modules in sqlalchemy.engine are strictly typed with the exception of cursor, default, and reflection. cursor and default pass with non-strict typing, reflection is waiting on the multi-reflection refactor. Behavioral changes: * create_connect_args() methods return a tuple of list, dict, rather than a list of list, dict * removed allow_chars parameter from pyodbc connector ._get_server_version_info() method * the parameter list passed to do_executemany is now a list in all cases. previously, this was being run through dialect.execute_sequence_format, which defaults to tuple and was only intended for individual tuple params. * broke up dialect.dbapi into dialect.import_dbapi class method and dialect.dbapi module object. added a deprecation path for legacy dialects. it's not really feasible to type a single attr as a classmethod vs. module type. The "type_compiler" attribute also has this problem with greater ability to work around, left that one for now. * lots of constants changing to be Enum, so that we can type them. for fixed tuple-position constants in cursor.py / compiler.py (which are used to avoid the speed overhead of namedtuple), using Literal[value] which seems to work well * some tightening up in Row regarding __getitem__, which we can do since we are on full 2.0 style result use * altered the set_connection_execution_options and set_engine_execution_options event flows so that the dictionary of options may be mutated within the event hook, where it will then take effect as the actual options used. Previously, changing the dict would be silently ignored which seems counter-intuitive and not very useful. * A lot of DefaultDialect/DefaultExecutionContext methods and attributes, including underscored ones, move to interfaces. This is not fully ideal as it means the Dialect/ExecutionContext interfaces aren't publicly subclassable directly, but their current purpose is more of documentation for dialect authors who should (and certainly are) still be subclassing the DefaultXYZ versions in all cases Overall, Result was the most extremely difficult class hierarchy to type here as this hierarchy passes through largely amorphous "row" datatypes throughout, which can in fact by all kinds of different things, like raw DBAPI rows, or Row objects, or "scalar"/Any, but at the same time these types have meaning so I tried still maintaining some level of semantic markings for these, it highlights how complex Result is now, as it's trying to be extremely efficient and inlined while also being very open-ended and extensible. Change-Id: I98b75c0c09eab5355fc7a33ba41dd9874274f12a
* pep-484 for sqlalchemy.event; use future annotationsMike Bayer2022-02-151-0/+2
| | | | | | | | | | __future__.annotations mode allows us to use non-string annotations for argument and return types in most cases, but more importantly it removes a large amount of runtime overhead that would be spent in evaluating the annotations. Change-Id: I2f5b6126fe0019713fc50001be3627b664019ede References: #6810
* establish mypy / typing approach for v2.0Mike Bayer2022-02-131-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | large patch to get ORM / typing efforts started. this is to support adding new test cases to mypy, support dropping sqlalchemy2-stubs entirely from the test suite, validate major ORM typing reorganization to eliminate the need for the mypy plugin. * New declarative approach which uses annotation introspection, fixes: #7535 * Mapped[] is now at the base of all ORM constructs that find themselves in classes, to support direct typing without plugins * Mypy plugin updated for new typing structures * Mypy test suite broken out into "plugin" tests vs. "plain" tests, and enhanced to better support test structures where we assert that various objects are introspected by the type checker as we expect. as we go forward with typing, we will add new use cases to "plain" where we can assert that types are introspected as we expect. * For typing support, users will be much more exposed to the class names of things. Add these all to "sqlalchemy" import space. * Column(ForeignKey()) no longer needs to be `@declared_attr` if the FK refers to a remote table * composite() attributes mapped to a dataclass no longer need to implement a `__composite_values__()` method * with_variant() accepts multiple dialect names Change-Id: I22797c0be73a8fbbd2d6f5e0c0b7258b17fe145d Fixes: #7535 Fixes: #7551 References: #6810
* Fix overlapping slots, base classes without slotsArie Bovenberg2022-02-011-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some `__slots__` were not in order. Fixes #7527 ### Description I'm fixing two types of slots mistakes: - [x] remove overlapping slots (i.e. slots already defined on a base class) - [x] fix broken inheritance (i.e. slots class inheriting from a non-slots class) - [x] slots added to base class `TransactionalContext`. It seemed to use two attributes, which I've added as slots. - [x] empty slots removed from `ORMOption`. Its base class explicitly makes use of `__dict__` so empty slots don't add anything. - [x] empty slots added to `PostLoader`. It doesn't appear to use any slots not already defined on its base classes. - [x] empty slots added to `IterateMappersMixin`. It doesn't appear to use any slots not already defined on its subclasses. - [x] empty slots added to `ImmutableContainer`. It doesn't use any fields. - [x] empty slots added to `OperatorType`. It's a protocol. - [x] empty slots added to `InternalTraversal`, `_HasTraversalDispatch`. They don't seem to use attributes on their own. ### Checklist This pull request is: - [x] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. **Have a nice day!** Closes: #7589 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7589 Pull-request-sha: 70a9c4d46916b7c6907eb1d3ad4f7033ec964191 Change-Id: I6c6e3e69c3c34d0f3bdda7f0684849834fdd1863
* Add new infrastructure to support greater use of __slots__Mike Bayer2022-01-211-1/+9
| | | | | | | | | | | | | | | | | | | * Changed AliasedInsp to use __slots__ * Migrated all of strategy_options to use __slots__ for objects. Adds new infrastructure to traversals to support shallow copy, to dict and from dict based on internal traversal attributes. Load / _LoadElement then leverage this to provide clone / generative / getstate without the need for __dict__ or explicit attribute lists. Doing this change revealed that there are lots of things that trigger off of whether or not a class has a __visit_name__ attribute. so to suit that we've gone back to having Visitable, which is a better name than Traversible at this point (I think Traversible is mis-spelled too). Change-Id: I13d04e494339fac9dbda0b8e78153418abebaf72 References: #7527
* Merge "track item schema names to identify name collisions w/ default ↵mike bayer2022-01-141-1/+1
|\ | | | | | | schema" into main
| * track item schema names to identify name collisions w/ default schemaMike Bayer2022-01-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added an additional lookup step to the compiler which will track all FROM clauses which are tables, that may have the same name shared in multiple schemas where one of the schemas is the implicit "default" schema; in this case, the table name when referring to that name without a schema qualification will be rendered with an anonymous alias name at the compiler level in order to disambiguate the two (or more) names. The approach of schema-qualifying the normally unqualified name with the server-detected "default schema name" value was also considered, however this approach doesn't apply to Oracle nor is it accepted by SQL Server, nor would it work with multiple entries in the PostgreSQL search path. The name collision issue resolved here has been identified as affecting at least Oracle, PostgreSQL, SQL Server, MySQL and MariaDB. Fixes: #7471 Change-Id: Id65e7ca8c43fe8d95777084e8d5ec140ebcd784d
* | Initial ORM typing layoutMike Bayer2022-01-141-3/+4
|/ | | | | | | | | | | | | | | | | | | | | | | | | | introduces: 1. new mapped_column() helper 2. DeclarativeBase helper 3. declared_attr has been re-typed 4. rework of Mapped[] to return InstrumentedAtribute for class get, so works without Mapped itself having expression methods 5. ORM constructs now generic on [_T] also includes some early typing work, most of which will be in later commits: 1. URL and History become typing.NamedTuple 2. come up with type-checking friendly way of type checking cy extensions, where type checking will be applied to the py versions, just needed to come up with a succinct conditional pattern for the imports References: #6810 References: #7535 References: #7562 Change-Id: Ie5d9a44631626c021d130ca4ce395aba623c71fb
* remove internal use of metaclassesMike Bayer2022-01-111-14/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All but one metaclass used internally can now be replaced using __init_subclass__(). Within this patch we remove: * events._EventMeta * sql.visitors.TraversibleType * sql.visitors.InternalTraversibleType * testing.fixtures.FindFixture * testing.fixtures.FindFixtureDeclarative * langhelpers.EnsureKWArgType * sql.functions._GenericMeta * sql.type_api.VisitableCheckKWArg (was a mixture of TraversibleType and EnsureKWArgType) The remaining internal class is MetaOptions used by the sql.Options object which is in turn currently mostly for ORM internal use, as this type implements class level overrides for the ``+`` operator. For declarative, removing DeclarativeMeta in place of an `__init_subclass__()` class would not be fully feasible as it would break backwards compatibility with applications that refer to this class explicitly, but also DeclarativeMeta intercepts class-level attribute set and delete operations which is a widely used pattern. An option for declarative base to use `__init_subclass__()` should be provided but this is out of scope for this particular change. Change-Id: I8aa898c7ab59d887739037d34b1cbab36521ab78 References: #6810
* happy new year 2022Mike Bayer2022-01-061-1/+1
| | | | Change-Id: I49abf2607e0eb0623650efdf0091b1fb3db737ea
* Properly type _generative, decorator, public_factoryFederico Caselli2021-12-301-10/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Good new is that pylance likes it and copies over the singature and everything. Bad news is that mypy does not support this yet https://github.com/python/mypy/issues/8645 Other minor bad news is that non_generative is not typed. I've tried using a protocol like the one in the comment but the signature is not ported over by pylance, so it's probably best to just live without it to have the correct signature. notes from mike: these three decorators are at the core of getting the library to be typed, more good news is that pylance will do all the things we like re: public_factory, see https://github.com/microsoft/pyright/issues/2758#issuecomment-1002788656 . For @_generative, we will likely move to using pep 673 once mypy supports it which may be soon. but overall having the explicit "return self" in the methods, while a little inconvenient, makes the typing more straightforward and locally present in the files rather than being decided at a distance. having "return self" present, or not, both have problems, so maybe we will be able to change it again if things change as far as decorator support. As it is, I feel like we are barely squeaking by with our decorators, the typing is already pretty out there. Change-Id: Ic77e13fc861def76a5925331df85c0aa48d77807 References: #6810
* Replace raise_ with raise fromFederico Caselli2021-12-271-17/+10
| | | | | Change-Id: I7aaeb5bc130271624335b79cf586581d6c6c34c7 References: #4600
* Merge "factor out UnboundLoad and rearchitect strategy_options.py" into mainmike bayer2021-12-271-2/+4
|\
| * factor out UnboundLoad and rearchitect strategy_options.pyMike Bayer2021-12-271-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The architecture of Load is mostly rewritten here. The change includes removal of the "pluggable" aspect of the loader options, which would patch new methods onto Load. This has been replaced by normal methods that respond normally to typing annotations. As part of this change, the bake_loaders() and unbake_loaders() options, which have no effect since 1.4 and were unlikely to be in any common use, have been removed. Additionally, to support annotations for methods that make use of @decorator, @generative etc., modified format_argspec_plus to no longer return "args", instead returns "grouped_args" which is always grouped and allows return annotations to format correctly. Fixes: #6986 Change-Id: I6117c642345cdde65a64389bba6057ddd5374427
View Lorry Controller Status