summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/engine
Commit message (Collapse)AuthorAgeFilesLines
* provide connectionfairy on initializeMike Bayer2021-11-298-307/+759
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is so that dialect methods that are called within init can assume the same argument structure as when they are called in other places; we can nail down the type of object as well. This change seems to mostly impact the isolation level routines in the dialects, as these are called during initialize() as well as on established connections. these methods can now assume a non-proxied DBAPI connection object in all cases, as it is commonly required that attributes like ".autocommit" are set on the object which don't work well in a proxied situation. Other changes: * adds an interface for the "connectionfairy" concept called PoolProxiedConnection. * Removes ``Connectable`` superclass of Connection. ``Connectable`` was originally meant to provide for the "method which accepts connection or engine" theme. As this pattern is greatly reduced in 2.0 and Engine no longer extends from it, the ``Connectable`` superclass doesnt serve any real purpose. Leading from that, to set this in I also applied pep 484 annotations to the Dialect base, and then in the interests of seeing some of the typing information show up in my IDE did a little bit for Engine, Connection and others. I hope that it's feasible that we can add annotations to specific classes and attributes ahead of when we actually try to mass-populate the whole library. This was the original spirit of pep-484 that we can apply annotations gradually. I do of course want to try to do a mass-populate although i think even in that case we will end up doing a lot of manual work anyway (in particular for the changes here which are distinct from what the stubs have). Fixes: #7122 Change-Id: I5dd7fbff8a7ae520a81c165091af12a6a68826db
* Added support for ``psycopg`` dialect.Federico Caselli2021-11-265-3/+42
| | | | | | | Both sync and async versions are supported. Fixes: #6842 Change-Id: I57751c5028acebfc6f9c43572562405453a2f2a4
* Merge "propose emulated setinputsizes embedded in the compiler" into mainmike bayer2021-11-254-12/+99
|\
| * propose emulated setinputsizes embedded in the compilerMike Bayer2021-11-234-12/+99
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a new system so that PostgreSQL and other dialects have a reliable way to add casts to bound parameters in SQL statements, replacing previous use of setinputsizes() for PG dialects. rationale: 1. psycopg3 will be using the same SQLAlchemy-side "setinputsizes" as asyncpg, so we will be seeing a lot more of this 2. the full rendering that SQLAlchemy's compilation is performing is in the engine log as well as error messages. Without this, we introduce three levels of SQL rendering, the compiler, the hidden "setinputsizes" in SQLAlchemy, and then whatever the DBAPI driver does. With this new approach, users reporting bugs etc. will be less confused that there are as many as two separate layers of "hidden rendering"; SQLAlchemy's rendering is again fully transparent 3. calling upon a setinputsizes() method for every statement execution is expensive. this way, the work is done behind the caching layer 4. for "fast insertmany()", I also want there to be a fast approach towards setinputsizes. As it was, we were going to be taking a SQL INSERT with thousands of bound parameter placeholders and running a whole second pass on it to apply typecasts. this way, we will at least be able to build the SQL string once without a huge second pass over the whole string 5. psycopg2 can use this same system for its ARRAY casts 6. the general need for PostgreSQL to have lots of type casts is now mostly in the base PostgreSQL dialect and works independently of a DBAPI being present. dependence on DBAPI symbols that aren't complete / consistent / hashable is removed I was originally going to try to build this into bind_expression(), but it was revealed this worked poorly with custom bind_expression() as well as empty sets. the current impl also doesn't need to run a second expression pass over the POSTCOMPILE sections, which came out better than I originally thought it would. Change-Id: I363e6d593d059add7bcc6d1f6c3f91dd2e683c0c
* | Clean up most py3k compatFederico Caselli2021-11-249-28/+25
|/ | | | Change-Id: I8172fdcc3103ff92aa049827728484c8779af6b7
* Remove object in class definitionFederico Caselli2021-11-229-16/+16
| | | | | References: #4600 Change-Id: I2a62ddfe00bc562720f0eae700a497495d7a987a
* Merge "fully support isolation_level parameter in base dialect" into mainmike bayer2021-11-185-36/+130
|\
| * fully support isolation_level parameter in base dialectMike Bayer2021-11-185-36/+130
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Generalized the :paramref:`_sa.create_engine.isolation_level` parameter to the base dialect so that it is no longer dependent on individual dialects to be present. This parameter sets up the "isolation level" setting to occur for all new database connections as soon as they are created by the connection pool, where the value then stays set without being reset on every checkin. The :paramref:`_sa.create_engine.isolation_level` parameter is essentially equivalent in functionality to using the :paramref:`_engine.Engine.execution_options.isolation_level` parameter via :meth:`_engine.Engine.execution_options` for an engine-wide setting. The difference is in that the former setting assigns the isolation level just once when a connection is created, the latter sets and resets the given level on each connection checkout. Fixes: #6342 Change-Id: Id81d6b1c1a94371d901ada728a610696e09e9741
* | Merge "Deprecate create_engine.implicit_returning" into mainmike bayer2021-11-182-12/+5
|\ \ | |/ |/|
| * Deprecate create_engine.implicit_returningjonathan vanasco2021-11-092-12/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The :paramref:`_sa.create_engine.implicit_returning` parameter is deprecated on the :func:`_sa.create_engine` function only; the parameter remains available on the :class:`_schema.Table` object. This parameter was originally intended to enable the "implicit returning" feature of SQLAlchemy when it was first developed and was not enabled by default. Under modern use, there's no reason this parameter should be disabled, and it has been observed to cause confusion as it degrades performance and makes it more difficult for the ORM to retrieve recently inserted server defaults. The parameter remains available on :class:`_schema.Table` to specifically suit database-level edge cases which make RETURNING infeasible, the sole example currently being SQL Server's limitation that INSERT RETURNING may not be used on a table that has INSERT triggers on it. Also removed from the Oracle dialect some logic that would upgrade an Oracle 8/8i server version to use implicit returning if the parameter were explictly passed; these versions of Oracle still support RETURNING so the feature is now enabled for all Oracle versions. Fixes: #6962 Change-Id: Ib338e300cd7c8026c3083043f645084a8211aed8
* | removals: all unicode encoding / decodingMike Bayer2021-11-102-165/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Removed here includes: * convert_unicode parameters * encoding create_engine() parameter * description encoding support * "non-unicode fallback" modes under Python 2 * String symbols regarding Python 2 non-unicode fallbacks * any concept of DBAPIs that don't accept unicode statements, unicode bound parameters, or that return bytes for strings anywhere except an explicit Binary / BLOB type * unicode processors in Python / C Risk factors: * Whether all DBAPIs do in fact return Unicode objects for all entries in cursor.description now * There was logic for mysql-connector trying to determine description encoding. A quick test shows Unicode coming back but it's not clear if there are still edge cases where they return bytes. if so, these are bugs in that driver, and at most we would only work around it in the mysql-connector DBAPI itself (but we won't do that either). * It seems like Oracle 8 was not expecting unicode bound parameters. I'm assuming this was all Python 2 stuff and does not apply for modern cx_Oracle under Python 3. * third party dialects relying upon built in unicode encoding/decoding but it's hard to imagine any non-SQLAlchemy database driver not dealing exclusively in Python unicode strings in Python 3 Change-Id: I97d762ef6d4dd836487b714d57d8136d0310f28a References: #7257
* | De-emphasize notion of "default driver" (DBAPI)Gord Thompson2021-11-095-15/+15
|/ | | | | | | | | | | Fixes: #6960 Even though a default driver still exists for each dialect, remove most usages of `dialect://` to encourage users to explicitly specify `dialect+driver://` Change-Id: I0ad42167582df509138fca64996bbb53e379b1af
* fully implement future engine and remove legacyMike Bayer2021-11-076-1257/+487
| | | | | | | | | | | | | | | | | | | | | | | | | | | The major action here is to lift and move future.Connection and future.Engine fully into sqlalchemy.engine.base. This removes lots of engine concepts, including: * autocommit * Connection running without a transaction, autobegin is now present in all cases * most "autorollback" is obsolete * Core-level subtransactions (i.e. MarkerTransaction) * "branched" connections, copies of connections * execution_options() returns self, not a new connection * old argument formats, distill_params(), simplifies calling scheme between engine methods * before/after_execute() events (oriented towards compiled constructs) don't emit for exec_driver_sql(). before/after_cursor_execute() is still included for this * old helper methods superseded by context managers, connection.transaction(), engine.transaction() engine.run_callable() * ancient engine-level reflection methods has_table(), table_names() * sqlalchemy.testing.engines.proxying_engine References: #7257 Change-Id: Ib20ed816642d873b84221378a9ec34480e01e82c
* Check for Mapping explicitly in 2.0 paramsMike Bayer2021-11-041-3/+3
| | | | | | | | | | Fixed issue in future :class:`_future.Connection` object where the :meth:`_future.Connection.execute` method would not accept a non-dict mapping object, such as SQLAlchemy's own :class:`.RowMapping` or other ``abc.collections.Mapping`` object as a parameter dictionary. Fixes: #7291 Change-Id: I819f079d86d19d1d81c570e0680f987e51e34b84
* Update "transaction has already begun" languageMike Bayer2021-11-042-3/+8
| | | | | | | | | | | | | | | | | | | As future connections will now be autobeginning, there will be more cases where begin() can't be called as well as where isolation level can't be set, which will be surprising as this is a behavioral change for 2.0; additionally, when DBAPI autocommit is set, there isn't actually a DBAPI level transaction in effect even though Connection has a Transaction object. Clarify the language in these two error messages to make it clear that begin() and autobegin are tracking a SQLAlchemy-level Transaction() object, whether or not the DBAPI has actually started a transaction, and that this is the reason rollback() or commit() is required before performing the requsted operation. Additionally make sure the error message mentions "autobegin" as a likely reason this error is being encountered along with what Connection needs the user to do in order to resolve. Change-Id: If8763939eeabc46aa9d9209a56d05ad82b892c5c
* Merge "ensure soft_close occurs for fetchmany with server side cursor" into mainmike bayer2021-11-022-8/+58
|\
| * ensure soft_close occurs for fetchmany with server side cursorMike Bayer2021-11-022-8/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression where the :meth:`_engine.CursorResult.fetchmany` method would fail to autoclose a server-side cursor (i.e. when ``stream_results`` or ``yield_per`` is in use, either Core or ORM oriented results) when the results were fully exhausted. All :class:`_result.Result` objects will now consistently raise :class:`_exc.ResourceClosedError` if they are used after a hard close, which includes the "hard close" that occurs after calling "single row or value" methods like :meth:`_result.Result.first` and :meth:`_result.Result.scalar`. This was already the behavior of the most common class of result objects returned for Core statement executions, i.e. those based on :class:`_engine.CursorResult`, so this behavior is not new. However, the change has been extended to properly accommodate for the ORM "filtering" result objects returned when using 2.0 style ORM queries, which would previously behave in "soft closed" style of returning empty results, or wouldn't actually "soft close" at all and would continue yielding from the underlying cursor. As part of this change, also added :meth:`_result.Result.close` to the base :class:`_result.Result` class and implemented it for the filtered result implementations that are used by the ORM, so that it is possible to call the :meth:`_engine.CursorResult.close` method on the underlying :class:`_engine.CursorResult` when the the ``yield_per`` execution option is in use to close a server side cursor before remaining ORM results have been fetched. This was again already available for Core result sets but the change makes it available for 2.0 style ORM results as well. Fixes: #7274 Change-Id: Id4acdfedbcab891582a7f8edd2e2e7d20d868e53
* | First round of removal of python 2Federico Caselli2021-11-014-51/+3
|/ | | | | References: #4600 Change-Id: I61e35bc93fe95610ae75b31c18a3282558cd4ffe
* remove case_sensitive create_engine parameterMike Bayer2021-11-013-55/+2
| | | | | | | | | | | | Removed the previously deprecated ``case_sensitive`` parameter from :func:`_sa.create_engine`, which would impact only the lookup of string column names in Core-only result set rows; it had no effect on the behavior of the ORM. The effective behavior of what ``case_sensitive`` refers towards remains at its default value of ``True``, meaning that string names looked up in ``row._mapping`` will match case-sensitively, just like any other Python mapping. Change-Id: I0dc4be3fac37d30202b1603db26fa10a110b618d
* Merge "2.0 removals: LegacyRow, connectionless execution, close_with_result" ↵mike bayer2021-10-319-640/+104
|\ | | | | | | into main
| * 2.0 removals: LegacyRow, connectionless execution, close_with_resultMike Bayer2021-10-319-640/+104
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | in order to remove LegacyRow / LegacyResult, we have to also lose close_with_result, which connectionless execution relies upon. also includes a new profiles.txt file that's all against py310, as that's what CI is on now. some result counts changed by one function call which was enough to fail the low-count result tests. Replaces Connectable as the common interface between Connection and Engine with EngineEventsTarget. Engine is no longer Connectable. Connection and MockConnection still are. References: #7257 Change-Id: Iad5eba0313836d347e65490349a22b061356896a
* | Merge "Remove deprecated dialects and drivers" into mainmike bayer2021-10-312-3/+2
|\ \
| * | Remove deprecated dialects and driversFederico Caselli2021-10-312-3/+2
| |/ | | | | | | | | Fixes: #7258 Change-Id: I3577f665eca04f2632b69bcb090f0a4ec9271db9
* | The ``has_table`` method now also checks viewsFederico Caselli2021-10-312-3/+14
|/ | | | | | | | | | | | The :meth:`_engine.Inspector.has_table` method will now consistently check for views of the given name as well as tables. Previously this behavior was dialect dependent, with PostgreSQL, MySQL/MariaDB and SQLite supporting it, and Oracle and SQL Server not supporting it. Third party dialects should also seek to ensure their :meth:`_engine.Inspector.has_table` method searches for views as well as tables for the given name. Fixes: #7161 Change-Id: I9e523c76741b19596c81ef577dc6f0823e44183b
* support bind expressions w/ expanding IN; apply to psycopg2Mike Bayer2021-10-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Fixed issue where "expanding IN" would fail to function correctly with datatypes that use the :meth:`_types.TypeEngine.bind_expression` method, where the method would need to be applied to each element of the IN expression rather than the overall IN expression itself. Fixed issue where IN expressions against a series of array elements, as can be done with PostgreSQL, would fail to function correctly due to multiple issues within the "expanding IN" feature of SQLAlchemy Core that was standardized in version 1.4. The psycopg2 dialect now makes use of the :meth:`_types.TypeEngine.bind_expression` method with :class:`_types.ARRAY` to portably apply the correct casts to elements. The asyncpg dialect was not affected by this issue as it applies bind-level casts at the driver level rather than at the compiler level. as part of this commit the "bind translate" feature has been simplified and also applies to the names in the POSTCOMPILE tag to accommodate for brackets. Fixes: #7177 Change-Id: I08c703adb0a9bd6f5aeee5de3ff6f03cccdccdc5
* Reflect table can reflect table with no columnsSumit Khanna2021-10-141-2/+3
| | | | | | | | | | | | | | | The :meth:`_engine.Inspector.reflect_table` method now supports reflecting tables that do not have user defined columns. This allows :meth:`_schema.MetaData.reflect` to properly complete reflection on databases that contain such tables. Currently, only PostgreSQL is known to support such a construct among the common database backends. Fixes: #3247 Closes: #7118 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7118 Pull-request-sha: cb8ce01957e9a1453290a7c2728af8c60ef55fa1 Change-Id: I906cebe17d13554d79086b92f3e1e51ffba3e818
* indicate private use of URL.__new__ privately onlyMike Bayer2021-10-011-1/+3
| | | | | | | | | | | Fixed issue where the deprecation warning for the :class:`.URL` constructor which indicates that the :meth:`.URL.create` method should be used would not emit if a full positional argument list of seven arguments were passed; additionally, validation of URL arguments will now occur if the constructor is called in this way, which was being skipped previously. Fixes: #7130 Change-Id: I8c8491d8aa7774afaf67c22b4f8e9859f780f2d9
* changelog, doc editsMike Bayer2021-09-221-8/+12
| | | | Change-Id: Iafb50de7e28626d9cee755db9c05ac7189b4d963
* clarify "encoding" parameter furtherMike Bayer2021-09-201-8/+16
| | | | | | | | | Prevent any reading of this parameter that would omit that it is not used under Python 3 and in Python 2 is not used very much either. Fixes: #7050 Change-Id: Iaf619f1ee164fc58afe710d11627ed6368d74343
* Surface driver connection object when using a proxied dialectFederico Caselli2021-09-174-2/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | Improve the interface used by adapted drivers, like the asyncio ones, to access the actual connection object returned by the driver. The :class:`_engine._ConnectionRecord` and :class:`_engine._ConnectionFairy` now have two new attributes: * ``dbapi_connection`` always represents a DBAPI compatible object. For pep-249 drivers, this is the DBAPI connection as it always has been, previously accessed under the ``.connection`` attribute. For asyncio drivers that SQLAlchemy adapts into a pep-249 interface, the returned object will normally be a SQLAlchemy adaption object called :class:`_engine.AdaptedConnection`. * ``driver_connection`` always represents the actual connection object maintained by the third party pep-249 DBAPI or async driver in use. For standard pep-249 DBAPIs, this will always be the same object as that of the ``dbapi_connection``. For an asyncio driver, it will be the underlying asyncio-only connection object. The ``.connection`` attribute remains available and is now a legacy alias of ``.dbapi_connection``. Fixes: #6832 Change-Id: Ib72f97deefca96dce4e61e7c38ba430068d6a82e
* Merge "Add scalars method to connection and session classes"mike bayer2021-09-151-0/+18
|\
| * Add scalars method to connection and session classesMiguel Grinberg2021-09-141-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added new methods :meth:`_orm.Session.scalars`, :meth:`_engine.Connection.scalars`, :meth:`_asyncio.AsyncSession.scalars` and :meth:`_asyncio.AsyncSession.stream_scalars`, which provide a short cut to the use case of receiving a row-oriented :class:`_result.Result` object and converting it to a :class:`_result.ScalarResult` object via the :meth:`_engine.Result.scalars` method, to return a list of values rather than a list of rows. The new methods are analogous to the long existing :meth:`_orm.Session.scalar` and :meth:`_engine.Connection.scalar` methods used to return a single value from the first row only. Pull request courtesy Miguel Grinberg. Fixes: #6990 Closes: #6991 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/6991 Pull-request-sha: b3e0bb3042c55b0cc5af6a25cb3f31b929f88a47 Change-Id: Ia445775e24ca964b0162c2c8e5ca67dd1e39199f
* | Ensure str is callect on the URL password.Federico Caselli2021-09-132-14/+48
|/ | | | | | | | | | | Ensure that ``str()`` is called on the an ``URL.password`` argument, allowing usage of objects that implement the ``__str__()`` method as password attributes. Also clarified that one such object is not appropriate to dynamically change the password. Fixes: #6958 Change-Id: Id0690990a64b9e0935537b7b8f5a73efe6a9e3dc
* remove erroneous None check from _assert_strMike Bayer2021-09-041-3/+0
| | | | | | | | Fixed issue in ``URL`` where validation of "drivername" would not appropriately respond to the ``None`` value where a string were expected. Fixes: #6983 Change-Id: If546c373a60533779595a9e393ea9a59a9b8a96f
* Fix and test sequences w/ executemany in pre-exec scenariosMike Bayer2021-09-021-1/+2
| | | | | | | | | | | | | | Fixed issue where an engine that had ``implicit_returning`` set to False would fail to function when PostgreSQL's "fast insertmany" feature were used in conjunction with a ``Sequence``, as well as if any kind of "executemany" with "return_defaults()" were used in conjunction with a ``Sequence``. Note that PostgreSQL "fast insertmany" uses "RETURNING" by definition, when the SQL statement is passed to the driver; overall, the ``implicit_returning`` flag is legacy and has no real use in modern SQLAlchemy, and will be deprecated in a separate change. Fixes: #6963 Change-Id: Id8e3dd50a21b9124f338067b0fdb57b8f608dca8
* Avoid mutable object as default valuestsimafeip2021-08-241-1/+1
| | | | | | | | | Fixes: #6915 Closes: #6916 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/6916 Pull-request-sha: 6ec484d3d14b7dd7053d10a5d550bd74eb524c8b Change-Id: I2c87fbed44870110e35a69ee9a9e678671eeb8f0
* Merge "restore statement substitution to before_execute()"mike bayer2021-08-231-1/+6
|\
| * restore statement substitution to before_execute()Mike Bayer2021-08-201-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed issue where the ability of the :meth:`_engine.ConnectionEvents.before_execute` method to alter the SQL statement object passed, returning the new object to be invoked, was inadvertently removed. This behavior has been restored. The refactor in a1939719a652774a437f69f8d4788b3f08650089 removed this feature for some reason and there were no tests in place to detect it. I don't see any indication this was planned. Fixes: #6913 Change-Id: Ia77ca08aa91ab9403f19a8eb61e2a0e41aad138a
* | heads up that execute(query).first() can't apply LIMIT 1Mike Bayer2021-08-201-0/+11
|/ | | | | Fixes: #6914 Change-Id: I5de9843dd3723c017b94b705fc009b883737ede1
* Merge "Documentation improvements"mike bayer2021-07-201-0/+9
|\
| * Documentation improvementsFederico Caselli2021-07-201-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Also remove deprecated usage: - load_only does not accept strings - case.whens is positional only Ref #6712 Ref #5994 Ref #6121 Ref #6785 Ref https://groups.google.com/g/sqlalchemy/c/-cnhThEu3kk Change-Id: I5db49a075b9d3d332518b9d189a24b13b502e2af
* | labeling refactorMike Bayer2021-07-121-3/+9
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To service #6718 and #6710, the system by which columns are given labels in a SELECT statement as well as the system that gives them keys in a .c or .selected_columns collection have been refactored to provide a single source of truth for both, in constrast to the previous approach that included similar logic repeated in slightly different ways. Main ideas: 1. ColumnElement attributes ._label, ._anon_label, ._key_label are renamed to include the letters "tq", meaning "table-qualified" - these labels are only used when rendering a SELECT that has LABEL_STYLE_TABLENAME_PLUS_COL for its label style; as this label style is primarily legacy, the "tq" names should be isolated so that in a 2.0 style application these aren't being used at all 2. The means by which the "labels" and "proxy keys" for the elements of a SELECT has been centralized to a single source of truth; previously, the three of _generate_columns_plus_names, _generate_fromclause_column_proxies, and _column_naming_convention all had duplicated rules between them, as well as that there were a little bit of labeling rules in compiler._label_select_column as well; by this we mean that the various "anon_label" "anon_key" methods on ColumnElement were called by all four of these methods, where there were many cases where it was necessary that one method comes up with the same answer as another of the methods. This has all been centralized into _generate_columns_plus_names for all the names except the "proxy key", which is generated by _column_naming_convention. 3. compiler._label_select_column has been rewritten to both not make any naming decisions nor any "proxy key" decisions, only whether to label or not to label; the _generate_columns_plus_names method gives it the information, where the proxy keys come from _column_naming_convention; previously, these proxy keys were matched based on restatement of similar (but not really the same) logic in two places. The heuristics of "whether to label or not to label" are also reorganized to be much easier to read and understand. 4. a new method compiler._label_returning_column is added for dialects to use in their "generate returning columns" methods. A github search reveals a small number of third party dialects also doing this using the prior _label_select_column method so we try to make sure _label_select_column continues to work the exact same way for that specific use case; for the "SELECT" use case it now needs 5. After some attempts to do it different ways, for the case where _proxy_key is giving us some kind of anon label, we are hard changing it to "_no_label" right now, as there's not currently a way to fully match anonymized labels from stmt.c or stmt.selected_columns to what will be in the result map. The idea of "_no_label" is to encourage the user to use label('name') for columns they want to be able to target by string name that don't have a natural name. Change-Id: I7a92a66f3a7e459ccf32587ac0a3c306650daf11
* Replace all http:// links to https://Federico Caselli2021-07-0414-17/+18
| | | | | | Also replace http://pypi.python.org/pypi with https://pypi.org/project Change-Id: I84b5005c39969a82140706472989f2a30b0c7685
* Fix typo in _warn_for_legacy_exec_formatgordthompson2021-06-241-1/+1
| | | | Change-Id: Idc24b5ab4b5a25fcfb7115c5d7be4c2ece520674
* Add Executable to DefaultGeneratorMike Bayer2021-06-231-1/+1
| | | | | | | | | | | | | | Fixed the class hierarchy for the :class:`_schema.Sequence` and the more general :class:`_schema.DefaultGenerator` base, as these are "executable" as statements they need to include :class:`_sql.Executable` in their hierarchy, not just :class:`_roles.StatementRole` as was applied arbitrarily to :class:`_schema.Sequence` previously. The fix allows :class:`_schema.Sequence` to work in all ``.execute()`` methods including with :meth:`_orm.Session.execute` which was not working in the case that a ``do_orm_execute()`` handler was also established. Fixes: #6668 Change-Id: I0d192258c7cbd1bce2552f9e748e8fdd680dc45f
* Export deferred columns but not col props; fix CTE labelingMike Bayer2021-06-221-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refined the behavior of ORM subquery rendering with regards to deferred columns and column properties to be more compatible with that of 1.3 while also providing for 1.4's newer features. As a subquery in 1.4 does not make use of loader options, including :func:`_orm.deferred`, a subquery that is against an ORM entity with deferred attributes will now render those deferred attributes that refer directly to mapped table columns, as these are needed in the outer SELECT if that outer SELECT makes use of these columns; however a deferred attribute that refers to a composed SQL expression as we normally do with :func:`_orm.column_property` will not be part of the subquery, as these can be selected explicitly if needed in the subquery. If the entity is being SELECTed from this subquery, the column expression can still render on "the outside" in terms of the derived subquery columns. This produces essentially the same behavior as when working with 1.3. However in this case the fix has to also make sure that the ``.selected_columns`` collection of an ORM-enabled :func:`_sql.select` also follows these rules, which in particular allows recursive CTEs to render correctly in this scenario, which were previously failing to render correctly due to this issue. As part of this change the _exported_columns_iterator() method has been removed and logic simplified to use ._all_selected_columns from any SelectBase object where _exported_columns_iterator() was used before. Additionally sets up UpdateBase to include ReturnsRows in its hierarchy; the literal point of ReturnsRows was to be a common base for UpdateBase and SelectBase so it was kind of weird it wasn't there. Fixes: #6661 Fixed issue in CTE constructs mostly relevant to ORM use cases where a recursive CTE against "anonymous" labels such as those seen in ORM ``column_property()`` mappings would render in the ``WITH RECURSIVE xyz(...)`` section as their raw internal label and not a cleanly anonymized name. Fixes: #6663 Change-Id: I26219d4d8e6c0915b641426e9885540f74fae4d2
* Merge "restore adapter logic in ORM loading"mike bayer2021-06-081-0/+4
|\
| * restore adapter logic in ORM loadingMike Bayer2021-06-081-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression involving how the ORM would resolve a given mapped column to a result row, where under cases such as joined eager loading, a slightly more expensive "fallback" could take place to set up this resolution due to some logic that was removed since 1.3. The issue could also cause deprecation warnings involving column resolution to be emitted when using a 1.4 style query with joined eager loading. In order to ensure we don't look up columns by string name in the ORM, we've turned on future_result=True in all cases, which I thought was already the assumption here, but apparently not. That in turn led to the issue that Session autocommit relies on close_with_result=True, which is legacy result only. This was also hard to figure out. So a new exception is raised if one tries to use future_result=True along with close_with_result, and the Session now has an explicit path for "autocommit" that sets these flags to their legacy values. This does leave the possibility of some of these fallback cases emitting warnings for users using session in autocommit along with joined inheritance and column properties, as this patch identifies that joined inheritance + column properties produce the fallback logic when looking up in the result via the adapted column, which in those tests is actually a Label object that doesn't adapt nicely. Fixes: #6596 Change-Id: I107a47e873ae05ab50853bb00a9ea0e1a88d5aee
* | Add asyncio.TimeoutError as an exit exceptionMike Bayer2021-06-071-2/+1
|/ | | | | | | | | | | | Added ``asyncio.exceptions.TimeoutError``, ``asyncio.exceptions.CancelledError`` as so-called "exit exceptions", a class of exceptions that include things like ``GreenletExit`` and ``KeyboardInterrupt``, which are considered to be events that warrant considering a DBAPI connection to be in an unusable state where it should be recycled. Fixes: #6592 Change-Id: Idcfa7aaa2d7660838b907388db9c6457afa6edbd
* Pass URL object, not the string, to on_connect_urlMike Bayer2021-06-061-2/+1
| | | | | | | | | | | | The fix for pysqlcipher released in version 1.4.3 :ticket:`5848` was unfortunately non-working, in that the new ``on_connect_url`` hook was erroneously not receiving a ``URL`` object under normal usage of :func:`_sa.create_engine` and instead received a string that was unhandled; the test suite failed to fully set up the actual conditions under which this hook is called. This has been fixed. Fixes: #6586 Change-Id: I3bf738daec35877a10fdad740f08dca9e7420829