summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/sql/selectable.py
Commit message (Collapse)AuthorAgeFilesLines
...
* TableValuedAlias generation fixesMike Bayer2022-04-031-2/+17
| | | | | | | | | | | | | | | | | | | | | | | | Fixed bug in newly implemented :paramref:`.FunctionElement.table_valued.joins_implicitly` feature where the parameter would not automatically propagate from the original :class:`.TableValuedAlias` object to the secondary object produced when calling upon :meth:`.TableValuedAlias.render_derived` or :meth:`.TableValuedAlias.alias`. Additionally repaired these issues in :class:`.TableValuedAlias`: * repaired a potential memory issue which could occur when repeatedly calling :meth:`.TableValuedAlias.render_derived` against successive copies of the same object (for .alias(), we currently have to still continue chaining from the previous element. not sure if this can be improved but this is standard behavior for .alias() elsewhere) * repaired issue where the individual element types would be lost when calling upon :meth:`.TableValuedAlias.render_derived` or :meth:`.TableValuedAlias.alias`. Fixes: #7890 Change-Id: Ie5120c7ff1e5c1bba5aaf77c782a51c637860208
* pep-484: the pep-484ening, SQL part threeMike Bayer2022-03-301-96/+142
| | | | | | | | | | | | | | | hitting DML which is causing us to open up the ColumnCollection structure a bit, as we do put anonymous column expressions with None here. However, we still want Table /TableClause to have named column collections that don't return None, so parametrize the "key" in this collection also. * rename some "immutable" elements to "readonly". we change the contents of immutablecolumncollection underneath, so it's not "immutable" Change-Id: I2593995a4e5c6eae874bed5bf76117198be8ae97
* column_descriptions or equiv for DML, core selectMike Bayer2022-03-281-3/+47
| | | | | | | | | | | | Added new attributes :attr:`.ValuesBase.returning_column_descriptions` and :attr:`.ValuesBase.entity_description` to allow for inspection of ORM attributes and entities that are installed as part of an :class:`.Insert`, :class:`.Update`, or :class:`.Delete` construct. The :attr:`.Select.column_descriptions` accessor is also now implemented for Core-only selectables. Fixes: #7861 Change-Id: Ia6a1cd24c798ba61f4e8e8eac90a0fd00d738342
* Merge "pep484 - SQL internals" into mainmike bayer2022-03-241-4/+6
|\
| * pep484 - SQL internalsMike Bayer2022-03-241-4/+6
| | | | | | | | | | | | | | non-strict checking for mostly internal or semi-internal code Change-Id: Ib91b47f1a8ccc15e666b94bad1ce78c4ab15b0ec
* | Add option to disable from linting for table valued functionMike Bayer2022-03-231-1/+9
|/ | | | | | | | | | | | | Added new parameter :paramref:`.FunctionElement.table_valued.joins_implicitly`, for the :meth:`.FunctionElement.table_valued` construct. This parameter indicates that the given table-valued function implicitly joins to the table it refers towards, essentially disabling the "from linting" feature, i.e. the "cartesian product" warning, from taking effect due to the presence of this parameter. May be used for functions such as ``func.json_each()``. Fixes: #7845 Change-Id: I80edcb74efbd4417172132c0db4d9c756fdd5eae
* remove intermediary _is_clone_of entries when cloningMike Bayer2022-03-171-1/+2
| | | | | | | | | | | | | | | | Improvements in memory usage by the ORM, removing a significant set of intermediary expression objects that are typically stored when a copy of an expression object is created. These clones have been greatly reduced, reducing the number of total expression objects stored in memory by ORM mappings by about 30%. note this change causes the tests to have a bit of a harder time with GC, which we would assume is because mappings now have a lot more garbage to clean up after mappers are configured. it remains to be seen what the long term effects of this are. Fixes: #7823 Change-Id: If8729747ffb9bf27e8974f069a994b5a823ee095
* pep-484 - SQL column operationsMike Bayer2022-03-151-18/+32
| | | | | | | | | note we are taking out the ColumnOperartors[SQLCoreOperations] thing; not really clear why that was needed and at the moment it seems I was likely confused. Change-Id: I834b75f9b44f91b97e29f2e1a7b1029bd910e0a1
* pep-484: sqlalchemy.sql pass oneMike Bayer2022-03-131-10/+19
| | | | | | | | | | | | | | | | | | sqlalchemy.sql will require many passes to get all modules even gradually typed. Will have to pick and choose what modules can be strictly typed vs. which can be gradual. in this patch, emphasis is on visitors.py, cache_key.py, annotations.py for strict typing, compiler.py is on gradual typing but has much more structure, in particular where it connects with the outside world. The work within compiler.py also reached back out to engine/cursor.py , default.py quite a bit. References: #6810 Change-Id: I6e8a29f6013fd216e43d45091bc193f8be0368fd
* additional mypy strictnessMike Bayer2022-03-121-1/+2
| | | | | | | | | | | | | | | | | | enable type checking within untyped defs. This allowed some more internals to be fixed up with assertions etc. some internals that were unnecessary or not even used at all were removed. BaseCursorResult was no longer necessary since we only have one kind of CursorResult now. The different ResultProxy subclasses that had alternate "strategies" dont appear to be used at all even in 1.4.x, as there's no code that accesses the _cursor_strategy_cls attribute, which is also removed. As these were mostly private constructs that weren't even functioning correctly in any case, it's fine to remove these over the 2.0 boundary. Change-Id: Ifd536987d104b1cd8b546cefdbd5c1e5d1801082
* pep484 + abc bases for assocaitionproxyMike Bayer2022-03-011-1/+5
| | | | | | | | | | went to this one next as it was going to be hard, and also exercises the ORM expression hierarchy a bit. made some adjustments to SQLCoreOperations etc. Change-Id: Ie5dde9218dc1318252826b766d3e70b17dd24ea7 References: #6810 References: #7774
* Add more nesting features to add_cte()Mike Bayer2022-02-241-11/+85
| | | | | | | | | | | | | | Added new parameter :paramref:`.HasCTE.add_cte.nest_here` to :meth:`.HasCTE.add_cte` which will "nest" a given :class:`.CTE` at the level of the parent statement. This parameter is equivalent to using the :paramref:`.HasCTE.cte.nesting` parameter, but may be more intuitive in some scenarios as it allows the nesting attribute to be set simultaneously along with the explicit level of the CTE. The :meth:`.HasCTE.add_cte` method also accepts multiple CTE objects. Fixes: #7759 Change-Id: I263c015f5a3f452cb54819aee12bc9bf2953a7bb
* pep-484 for sqlalchemy.event; use future annotationsMike Bayer2022-02-151-0/+2
| | | | | | | | | | __future__.annotations mode allows us to use non-string annotations for argument and return types in most cases, but more importantly it removes a large amount of runtime overhead that would be spent in evaluating the annotations. Change-Id: I2f5b6126fe0019713fc50001be3627b664019ede References: #6810
* establish mypy / typing approach for v2.0Mike Bayer2022-02-131-29/+38
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | large patch to get ORM / typing efforts started. this is to support adding new test cases to mypy, support dropping sqlalchemy2-stubs entirely from the test suite, validate major ORM typing reorganization to eliminate the need for the mypy plugin. * New declarative approach which uses annotation introspection, fixes: #7535 * Mapped[] is now at the base of all ORM constructs that find themselves in classes, to support direct typing without plugins * Mypy plugin updated for new typing structures * Mypy test suite broken out into "plugin" tests vs. "plain" tests, and enhanced to better support test structures where we assert that various objects are introspected by the type checker as we expect. as we go forward with typing, we will add new use cases to "plain" where we can assert that types are introspected as we expect. * For typing support, users will be much more exposed to the class names of things. Add these all to "sqlalchemy" import space. * Column(ForeignKey()) no longer needs to be `@declared_attr` if the FK refers to a remote table * composite() attributes mapped to a dataclass no longer need to implement a `__composite_values__()` method * with_variant() accepts multiple dialect names Change-Id: I22797c0be73a8fbbd2d6f5e0c0b7258b17fe145d Fixes: #7535 Fixes: #7551 References: #6810
* Merge "track item schema names to identify name collisions w/ default ↵mike bayer2022-01-141-1/+27
|\ | | | | | | schema" into main
| * track item schema names to identify name collisions w/ default schemaMike Bayer2022-01-141-1/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added an additional lookup step to the compiler which will track all FROM clauses which are tables, that may have the same name shared in multiple schemas where one of the schemas is the implicit "default" schema; in this case, the table name when referring to that name without a schema qualification will be rendered with an anonymous alias name at the compiler level in order to disambiguate the two (or more) names. The approach of schema-qualifying the normally unqualified name with the server-detected "default schema name" value was also considered, however this approach doesn't apply to Oracle nor is it accepted by SQL Server, nor would it work with multiple entries in the PostgreSQL search path. The name collision issue resolved here has been identified as affecting at least Oracle, PostgreSQL, SQL Server, MySQL and MariaDB. Fixes: #7471 Change-Id: Id65e7ca8c43fe8d95777084e8d5ec140ebcd784d
* | Initial ORM typing layoutMike Bayer2022-01-141-12/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | introduces: 1. new mapped_column() helper 2. DeclarativeBase helper 3. declared_attr has been re-typed 4. rework of Mapped[] to return InstrumentedAtribute for class get, so works without Mapped itself having expression methods 5. ORM constructs now generic on [_T] also includes some early typing work, most of which will be in later commits: 1. URL and History become typing.NamedTuple 2. come up with type-checking friendly way of type checking cy extensions, where type checking will be applied to the py versions, just needed to come up with a succinct conditional pattern for the imports References: #6810 References: #7535 References: #7562 Change-Id: Ie5d9a44631626c021d130ca4ce395aba623c71fb
* | initial reorganize for static typingMike Bayer2022-01-121-424/+26
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | start applying foundational annotations to key elements. two main elements addressed here: 1. removal of public_factory() and replacement with explicit functions. this just works much better with typing. 2. typing support for column expressions and operators. The biggest part of this involves stubbing out all the ColumnOperators methods under ColumnElement in a TYPE_CHECKING section. Took me a while to see this method vs. much more complicated things I thought I needed. Also for this version implementing #7519, ColumnElement types against the Python type and not TypeEngine. it is hoped this leads to easier transferrence between ORM/Core as well as eventual support for result set typing. Not clear yet how well this approach will work and what new issues it may introduce. given the current approach we now get full, rich typing for scenarios like this: from sqlalchemy import column, Integer, String, Boolean c1 = column('a', String) c2 = column('a', Integer) expr1 = c2.in_([1, 2, 3]) expr2 = c2 / 5 expr3 = -c2 expr4_a = ~(c2 == 5) expr4_b = ~column('q', Boolean) expr5 = c1 + 'x' expr6 = c2 + 10 Fixes: #7519 Fixes: #6810 Change-Id: I078d9f57955549f6f7868314287175f6c61c44cb
* remove internal use of metaclassesMike Bayer2022-01-111-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All but one metaclass used internally can now be replaced using __init_subclass__(). Within this patch we remove: * events._EventMeta * sql.visitors.TraversibleType * sql.visitors.InternalTraversibleType * testing.fixtures.FindFixture * testing.fixtures.FindFixtureDeclarative * langhelpers.EnsureKWArgType * sql.functions._GenericMeta * sql.type_api.VisitableCheckKWArg (was a mixture of TraversibleType and EnsureKWArgType) The remaining internal class is MetaOptions used by the sql.Options object which is in turn currently mostly for ORM internal use, as this type implements class level overrides for the ``+`` operator. For declarative, removing DeclarativeMeta in place of an `__init_subclass__()` class would not be fully feasible as it would break backwards compatibility with applications that refer to this class explicitly, but also DeclarativeMeta intercepts class-level attribute set and delete operations which is a widely used pattern. An option for declarative base to use `__init_subclass__()` should be provided but this is out of scope for this particular change. Change-Id: I8aa898c7ab59d887739037d34b1cbab36521ab78 References: #6810
* happy new year 2022Mike Bayer2022-01-061-1/+1
| | | | Change-Id: I49abf2607e0eb0623650efdf0091b1fb3db737ea
* Remove all remaining removed_in_20 warnings slated for removalMike Bayer2022-01-051-388/+26
| | | | | | | | | | | | | | | | | | | | | | | | | Finalize all remaining removed-in-2.0 changes so that we can begin doing pep-484 typing without old things getting in the way (we will also have to do public_factory). note there are a few "moved_in_20()" and "became_legacy_in_20()" warnings still in place. The SQLALCHEMY_WARN_20 variable is now removed. Also removed here are the legacy "in place mutators" for Select statements, and some keyword-only argument signatures in Core have been added. Also in the big change department, the ORM mapper() function is removed entirely; the Mapper class is otherwise unchanged, just the public-facing API function. Mappers are now always given a registry in which to participate, however the argument signature of Mapper is not changed. ideally "registry" would be the first positional argument. Fixes: #7257 Change-Id: Ic70c57b9f1cf7eb996338af5183b11bdeb3e1623
* Update Black's target-version to py37Hugo van Kemenade2022-01-051-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | <!-- Provide a general summary of your proposed changes in the Title field above --> ### Description <!-- Describe your changes in detail --> Black's `target-version` was still set to `['py27', 'py36']`. Set it to `[py37]` instead. Also update Black and other pre-commit hooks and re-format with Black. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [ ] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #7536 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7536 Pull-request-sha: b3aedf5570d7e0ba6c354e5989835260d0591b08 Change-Id: I8be85636fd2c9449b07a8626050c8bd35bd119d5
* remove 2.0-removed Query elementsMike Bayer2022-01-011-16/+7
| | | | | | | | | | | | | | * :meth:`_orm.Query.join` no longer accepts the "aliased" and "from_joinpoint" arguments * :meth:`_orm.Query.join` no longer accepts chains of multiple join targets in one method call. * ``Query.from_self()`` and ``Query.with_polymorphic()`` are removed. Change-Id: I534d04b53a538a4fc374966eb2bc8eb98a16497d References: #7257
* Properly type _generative, decorator, public_factoryFederico Caselli2021-12-301-32/+110
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Good new is that pylance likes it and copies over the singature and everything. Bad news is that mypy does not support this yet https://github.com/python/mypy/issues/8645 Other minor bad news is that non_generative is not typed. I've tried using a protocol like the one in the comment but the signature is not ported over by pylance, so it's probably best to just live without it to have the correct signature. notes from mike: these three decorators are at the core of getting the library to be typed, more good news is that pylance will do all the things we like re: public_factory, see https://github.com/microsoft/pyright/issues/2758#issuecomment-1002788656 . For @_generative, we will likely move to using pep 673 once mypy supports it which may be soon. but overall having the explicit "return self" in the methods, while a little inconvenient, makes the typing more straightforward and locally present in the files rather than being decided at a distance. having "return self" present, or not, both have problems, so maybe we will be able to change it again if things change as far as decorator support. As it is, I feel like we are barely squeaking by with our decorators, the typing is already pretty out there. Change-Id: Ic77e13fc861def76a5925331df85c0aa48d77807 References: #6810
* remove legacy select patternsMike Bayer2021-12-291-439/+8
| | | | | Change-Id: If6e521a1eb461e08748a0432943b938528a2619e References: #7257
* Replace raise_ with raise fromFederico Caselli2021-12-271-16/+10
| | | | | Change-Id: I7aaeb5bc130271624335b79cf586581d6c6c34c7 References: #4600
* Fix typo in docstringLele Gaifax2021-12-091-1/+1
|
* fix typo in exists documentationFederico Caselli2021-12-051-1/+1
| | | | Change-Id: I44d8d6ee4816052a8dda64c00905b17b7ad8698e
* The where method of exists now accepts multiple cluase.Federico Caselli2021-12-041-2/+8
| | | | | | | | | Support multiple clause elements in the :meth:`_sql.Exists.where` method, unifying the api with the on presented by a normal :func:`_sql.select` construct. Fixes: #7386 Change-Id: I5df20478008cd5167053d357cbfad8a641c62b44
* Removals: MetaData.bind, Table.bind, all other .bindMike Bayer2021-12-021-102/+0
| | | | | Change-Id: I1ef2eb2018f4b68825fe40a2a8d99084cf217b35 References: #7257
* Clean up most py3k compatFederico Caselli2021-11-241-11/+3
| | | | Change-Id: I8172fdcc3103ff92aa049827728484c8779af6b7
* Remove object in class definitionFederico Caselli2021-11-221-7/+7
| | | | | References: #4600 Change-Id: I2a62ddfe00bc562720f0eae700a497495d7a987a
* Add Non linear CTE supportEric Masseran2021-11-161-29/+155
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "Compound select" methods like :meth:`_sql.Select.union`, :meth:`_sql.Select.intersect_all` etc. now accept ``*other`` as an argument rather than ``other`` to allow for multiple additional SELECTs to be compounded with the parent statement at once. In particular, the change as applied to :meth:`_sql.CTE.union` and :meth:`_sql.CTE.union_all` now allow for a so-called "non-linear CTE" to be created with the :class:`_sql.CTE` construct, whereas previously there was no way to have more than two CTE sub-elements in a UNION together while still correctly calling upon the CTE in recursive fashion. Pull request courtesy Eric Masseran. Allow: ```sql WITH RECURSIVE nodes(x) AS ( SELECT 59 UNION SELECT aa FROM edge JOIN nodes ON bb=x UNION SELECT bb FROM edge JOIN nodes ON aa=x ) SELECT x FROM nodes; ``` Based on @zzzeek suggestion: https://github.com/sqlalchemy/sqlalchemy/pull/7133#issuecomment-933882348 Fixes: #7259 Closes: #7260 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7260 Pull-request-sha: 2565a5fd4b1940e92125e53aeaa731cc682f49bb Change-Id: I685c8379762b5fb6ab4107ff8f4d8a4de70c0ca6
* fully implement future engine and remove legacyMike Bayer2021-11-071-3/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | The major action here is to lift and move future.Connection and future.Engine fully into sqlalchemy.engine.base. This removes lots of engine concepts, including: * autocommit * Connection running without a transaction, autobegin is now present in all cases * most "autorollback" is obsolete * Core-level subtransactions (i.e. MarkerTransaction) * "branched" connections, copies of connections * execution_options() returns self, not a new connection * old argument formats, distill_params(), simplifies calling scheme between engine methods * before/after_execute() events (oriented towards compiled constructs) don't emit for exec_driver_sql(). before/after_cursor_execute() is still included for this * old helper methods superseded by context managers, connection.transaction(), engine.transaction() engine.run_callable() * ancient engine-level reflection methods has_table(), table_names() * sqlalchemy.testing.engines.proxying_engine References: #7257 Change-Id: Ib20ed816642d873b84221378a9ec34480e01e82c
* add missing info from groupby documentationFederico Caselli2021-11-031-0/+2
| | | | | Change-Id: Icfaf242353c23a579fe79f9d72500a08d90fcb77 Signed-off-by: Federico Caselli <cfederico87@gmail.com>
* Remove deprecated dialects and driversFederico Caselli2021-10-311-1/+1
| | | | | Fixes: #7258 Change-Id: I3577f665eca04f2632b69bcb090f0a4ec9271db9
* clarify order_by(None) for Core; improve wordingMike Bayer2021-10-291-1/+8
| | | | | | | the order_by(None) convention was documented for orm.Query but not Core select. Change-Id: I0c1ad76c3eefba1cb54b1649cfd09169c17e2bba
* remove _resolve_label and related attributesMike Bayer2021-10-201-1/+1
| | | | | | | | | | | | | these seem to date back to 0.9 and revs like #3148 but none of it seems to be affecting things now, try removing it all and seeing what fails. we've noted that _resolve_label does not appear to be needed, however allow_label_resolve remains relevant both for non-column elements like text() as well as that it is used explicitly by the joined eager loader. Change-Id: Ic8a5d8001ef2a4133360f51a92a6f7b0cc389095
* Fix recursive CTE to support nestingEric Masseran2021-10-121-4/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | Repaired issue in new :paramref:`_sql.HasCTE.cte.nesting` parameter introduced with :ticket:`4123` where a recursive :class:`_sql.CTE` using :paramref:`_sql.HasCTE.cte.recursive` in typical conjunction with UNION would not compile correctly. Additionally makes some adjustments so that the :class:`_sql.CTE` construct creates a correct cache key. Pull request courtesy Eric Masseran. Fixes: #4123 > This has not been caught by the tests because the nesting recursive queries there did not union against itself, eg there was only the i root clause... - Now tests are real recursive queries - Add tests on aliased nested CTEs (recursive or not) - Adapt the `_restates` attribute to use it as a reference - Add some docs around to explain some variables usage Closes: #7133 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7133 Pull-request-sha: 2633f34f7f5336a4a85bd3f71d07bca33ce27a2c Change-Id: I15512c94e1bc1f52afc619d82057ca647d274e92
* fixes for usage of the null() and similar constantsMike Bayer2021-10-081-6/+24
| | | | | | | | | | | | | | | | | | | | | | | | Adjusted the "column disambiguation" logic that's new in 1.4, where the same expression repeated gets an "extra anonymous" label, so that the logic more aggressively deduplicates those labels when the repeated element is the same Python expression object each time, as occurs in cases like when using "singleton" values like :func:`_sql.null`. This is based on the observation that at least some databases (e.g. MySQL, but not SQLite) will raise an error if the same label is repeated inside of a subquery. Related to :ticket:`7153`, fixed an issue where result column lookups would fail for "adapted" SELECT statements that selected for "constant" value expressions most typically the NULL expression, as would occur in such places as joined eager loading in conjunction with limit/offset. This was overall a regression due to issue :ticket:`6259` which removed all "adaption" for constants like NULL, "true", and "false", but this broke the case where the same adaption logic were used to match the constant to a labeled expression referring to the constant in a subquery. Fixes: #7153 Fixes: #7154 Change-Id: I43823343721b9e70524ea3f5e8f39dd543a3e92b
* Account for `schema` in `table()` `fullname` attribute.Federico Caselli2021-10-011-1/+5
| | | | | Fixes: #7061 Change-Id: I715da89591c03d40d77734473bd42b34b9c4e1dc
* include setup_joins targets when scanning for FROM objects to cloneMike Bayer2021-09-201-0/+34
| | | | | | | | | | | | | | Fixed a two issues where combinations of ``select()`` and ``join()`` when adapted to form a copy of the element would not completely copy the state of all column objects associated with subqueries. A key problem this caused is that usage of the :meth:`_sql.ClauseElement.params` method (which should probably be moved into a legacy category as it is inefficient and error prone) would leave copies of the old :class:`_sql.BindParameter` objects around, leading to issues in correctly setting the parameters at execution time. Fixes: #7055 Change-Id: Ib822a978a99561b4402da3fb727b370f5c58210b
* Implement nesting CTEEric Masseran2021-09-171-2/+43
| | | | | | | | | | | | | | | | | | Added new parameter :meth:`_sql.HasCte.cte.nesting` to the :class:`_sql.CTE` constructor and :meth:`_sql.HasCTE.cte` method, which flags the CTE as one which should remain nested within an enclosing CTE, rather than being moved to the top level of the outermost SELECT. While in the vast majority of cases there is no difference in SQL functionality, users have identified various edge-cases where true nesting of CTE constructs is desirable. Much thanks to Eric Masseran for lots of work on this intricate feature. Fixes: #4123 Closes: #6709 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/6709 Pull-request-sha: 64ab2f6ea269f2dcf37376a13ea38c48c5226fb6 Change-Id: Ic4dc25ab763af96d96632369e01527d48a654149
* Fix various lib / test / examples typos (#7017)Kevin Kirsche2021-09-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * fix: lib/sqlalchemy/sql/lambdas.py * fix: lib/sqlalchemy/sql/compiler.py * fix: lib/sqlalchemy/sql/selectable.py * fix: lib/sqlalchemy/orm/relationships.py * fix: lib/sqlalchemy/dialects/mssql/base.py * fix: lib/sql/test_compiler.py * fix: lib/sqlalchemy/testing/requirements.py * fix: lib/sqlalchemy/orm/path_registry.py * fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py * fix: lib/sqlalchemy/cextension/immutabledict.c * fix: lib/sqlalchemy/cextension/resultproxy.c * fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py * fix: examples/versioned_rows/versioned_rows_w_versionid.py * fix: examples/elementtree/optimized_al.py * fix: test/orm/test_attribute.py * fix: test/sql/test_compare.py * fix: test/sql/test_type_expression.py * fix: capitalization in test/dialect/mysql/test_compiler.py * fix: typos in test/dialect/postgresql/test_reflection.py * fix: typo in tox.ini comment * fix: typo in /lib/sqlalchemy/orm/decl_api.py * fix: typo in test/orm/test_update_delete.py * fix: self-induced typo * fix: typo in test/orm/test_query.py * fix: typos in test/dialect/mssql/test_types.py * fix: typo in test/sql/test_types.py
* remove lambda caching from loader strategiesMike Bayer2021-08-171-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adjusted ORM loader internals to no longer use the "lambda caching" system that was added in 1.4, as well as repaired one location that was still using the previous "baked query" system for a query. The lambda caching system remains an effective way to reduce the overhead of building up queries that have relatively fixed usage patterns. In the case of loader strategies, the queries used are responsible for moving through lots of arbitrary options and criteria, which is both generated and sometimes consumed by end-user code, that make the lambda cache concept not any more efficient than not using it, at the cost of more complexity. In particular the problems noted by :ticket:`6881` and :ticket:`6887` are made considerably less complicated by removing this feature internally. Fixed an issue where the :class:`_orm.Bundle` construct would not create proper cache keys, leading to inefficient use of the query cache. This had some impact on the "selectinload" strategy and was identified as part of :ticket:`6889`. Added a Select._create_raw_select() method which essentially performs ``__new__`` and then populates ``__dict__`` directly, with no coercions. This saves most of the overhead time that the lambda caching system otherwise seeks to avoid. Includes removal of bakedquery from mapper->_subclass_load_via_in() which was overlooked from the 1.4 refactor. Fixes: #6079 Fixes: #6889 Change-Id: Ieac2d9d709b71ec4270e5c121fbac6ac870e2bb1
* add columns_clause_froms and related use casesMike Bayer2021-08-071-11/+117
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added new attribute :attr:`_sql.Select.columns_clause_froms` that will retrieve the FROM list implied by the columns clause of the :class:`_sql.Select` statement. This differs from the old :attr:`_sql.Select.froms` collection in that it does not perform any ORM compilation steps, which necessarily deannotate the FROM elements and do things like compute joinedloads etc., which makes it not an appropriate candidate for the :meth:`_sql.Select.select_from` method. Additionally adds a new parameter :paramref:`_sql.Select.with_only_columns.maintain_column_froms` that transfers this collection to :meth:`_sql.Select.select_from` before replacing the columns collection. In addition, the :attr:`_sql.Select.froms` is renamed to :meth:`_sql.Select.get_final_froms`, to stress that this collection is not a simple accessor and is instead calculated given the full state of the object, which can be an expensive call when used in an ORM context. Additionally fixes a regression involving the :func:`_orm.with_only_columns` function to support applying criteria to column elements that were replaced with either :meth:`_sql.Select.with_only_columns` or :meth:`_orm.Query.with_entities` , which had broken as part of :ticket:`6503` released in 1.4.19. Fixes: #6808 Change-Id: Ib5d66cce488bbaca06dab4f68fb5cdaa73e8823e
* Adjust CTE recrusive col list to accommodate dupe col namesMike Bayer2021-07-131-1/+4
| | | | | | | | | | | | | | | | | | | Fixed issue in CTE constructs where a recursive CTE that referred to a SELECT that has duplicate column names, which are typically deduplicated using labeling logic in 1.4, would fail to refer to the deduplicated label name correctly within the WITH clause. As part of this change we are also attempting to remove the behavior of SelectStatementGrouping forcing off the "asfrom" contextual flag, which will have the result of additional labeling being applied to some UNION and similar statements when they are interpreted as subqueries. To maintain compatibility with "grouping", the Grouping/SelectStatementGrouping are now broken out into two separate compiler cases, as the "asfrom" logic appears to be tailored towards table valued SELECTS as column expressions. Fixes: #6710 Change-Id: I8af07a5c670dbe5736cd9f16084ef82f5e4c8642
* Merge "labeling refactor"mike bayer2021-07-131-123/+140
|\
| * labeling refactorMike Bayer2021-07-121-123/+140
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To service #6718 and #6710, the system by which columns are given labels in a SELECT statement as well as the system that gives them keys in a .c or .selected_columns collection have been refactored to provide a single source of truth for both, in constrast to the previous approach that included similar logic repeated in slightly different ways. Main ideas: 1. ColumnElement attributes ._label, ._anon_label, ._key_label are renamed to include the letters "tq", meaning "table-qualified" - these labels are only used when rendering a SELECT that has LABEL_STYLE_TABLENAME_PLUS_COL for its label style; as this label style is primarily legacy, the "tq" names should be isolated so that in a 2.0 style application these aren't being used at all 2. The means by which the "labels" and "proxy keys" for the elements of a SELECT has been centralized to a single source of truth; previously, the three of _generate_columns_plus_names, _generate_fromclause_column_proxies, and _column_naming_convention all had duplicated rules between them, as well as that there were a little bit of labeling rules in compiler._label_select_column as well; by this we mean that the various "anon_label" "anon_key" methods on ColumnElement were called by all four of these methods, where there were many cases where it was necessary that one method comes up with the same answer as another of the methods. This has all been centralized into _generate_columns_plus_names for all the names except the "proxy key", which is generated by _column_naming_convention. 3. compiler._label_select_column has been rewritten to both not make any naming decisions nor any "proxy key" decisions, only whether to label or not to label; the _generate_columns_plus_names method gives it the information, where the proxy keys come from _column_naming_convention; previously, these proxy keys were matched based on restatement of similar (but not really the same) logic in two places. The heuristics of "whether to label or not to label" are also reorganized to be much easier to read and understand. 4. a new method compiler._label_returning_column is added for dialects to use in their "generate returning columns" methods. A github search reveals a small number of third party dialects also doing this using the prior _label_select_column method so we try to make sure _label_select_column continues to work the exact same way for that specific use case; for the "SELECT" use case it now needs 5. After some attempts to do it different ways, for the case where _proxy_key is giving us some kind of anon label, we are hard changing it to "_no_label" right now, as there's not currently a way to fully match anonymized labels from stmt.c or stmt.selected_columns to what will be in the result map. The idea of "_no_label" is to encourage the user to use label('name') for columns they want to be able to target by string name that don't have a natural name. Change-Id: I7a92a66f3a7e459ccf32587ac0a3c306650daf11
* | Merge "implement independent CTEs"mike bayer2021-07-121-0/+75
|\ \
View Lorry Controller Status