Fixed a long-standing race condition in the connection pool which could occur under eventlet/gevent monkeypatching schemes in conjunction with the use of eventlet/gevent ``Timeout`` conditions, where a connection pool checkout that's interrupted due to the timeout would fail to clean up the failed state, causing the underlying connection record and sometimes the database connection itself to "leak", leaving the pool in an invalid state with unreachable entries. This issue was first identified and fixed in SQLAlchemy 1.2 for :ticket:`4225`, however the failure modes detected in that fix failed to accommodate for ``BaseException``, rather than ``Exception``, which prevented eventlet/gevent ``Timeout`` from being caught. In addition, a block within initial pool connect has also been identified and hardened with a ``BaseException`` -> "clean failed connect" block to accommodate for the same condition in this location. Big thanks to Github user @niklaus for their tenacious efforts in identifying and describing this intricate issue. Fixes: #8974 Change-Id: I95a0e1f080d0cee6f1a66977432a586fdf87f686 (cherry picked from commit a71917204dcf12a93d957a0fa29c9df97d0411ee)
Fixed issue where the :meth:`.PoolEvents.reset` event hook would not be called when a :class:`.Connection` were closed which already closed its own transaction. Logic that bypasses the "rollback on return" behavior of the pool was also skipping the event hook being emitted, preventing custom pool reset schemes from being used within this hook. This was a regression that appeared in version 1.4. For version 1.4, the hook is still not called in the case of an asyncio connection that is being discarded due to garbage collection. Version 2.0 will feature an improved version of :meth:`.PoolEvents.reset` which also receives contextual information about the reset, so that comprehensive "custom connection reset" schemes can be devised. Existing custom reset schemes that make use of :meth:`.PoolEvents.checkin` remain usable as they typically only need to deal with connections that are to be re-used. Change-Id: Ie17c4f55d02beb6f570b9de6b3044baffa7d6df6 Fixes: #8717 (cherry picked from commit bb8c36c5d2622e6e7033dc59dc98da0926ba7c00)
Integrated support for asyncpg's ``terminate()`` method call for cases where the connection pool is recycling a possibly timed-out connection, where a connection is being garbage collected that wasn't gracefully closed, as well as when the connection has been invalidated. This allows asyncpg to abandon the connection without waiting for a response that may incur long timeouts. Fixes: #8419 Change-Id: Ia575af779d5733b483a72dff3690b8bbbad2bb05 (cherry picked from commit 3b7e621aa728d9b01dbac4150e13ea2ef6af35a3)
Fixed bugs involving the :paramref:`.Table.include_columns` and the :paramref:`.Table.resolve_fks` parameters on :class:`.Table`; these little-used parameters were apparently not working for columns that refer to foreign key constraints. In the first case, not-included columns that refer to foreign keys would still attempt to create a :class:`.ForeignKey` object, producing errors when attempting to resolve the columns for the foreign key constraint within reflection; foreign key constraints that refer to skipped columns are now omitted from the table reflection process in the same way as occurs for :class:`.Index` and :class:`.UniqueConstraint` objects with the same conditions. No warning is produced however, as we likely want to remove the include_columns warnings for all constraints in 2.0. In the latter case, the production of table aliases or subqueries would fail on an FK related table not found despite the presence of ``resolve_fks=False``; the logic has been repaired so that if a related table is not found, the :class:`.ForeignKey` object is still proxied to the aliased table or subquery (these :class:`.ForeignKey` objects are normally used in the production of join conditions), but it is sent with a flag that it's not resolvable. The aliased table / subquery will then work normally, with the exception that it cannot be used to generate a join condition automatically, as the foreign key information is missing. This was already the behavior for such foreign key constraints produced using non-reflection methods, such as joining :class:`.Table` objects from different :class:`.MetaData` collections. Fixes: #8100 Fixes: #8101 Change-Id: Ifa37a91bd1f1785fca85ef163eec031660d9ea4d (cherry picked from commit 40e3c0da5be7dd526866bfc63590fc5621a9bd6e)
ROLLBACK TO SAVEPOINT is re-enabled in https://github.com/tlocke/pg8000/issues/111. we still have to add savepoint support to our fixture that deletes from tables without checking for them. this is inconvenient but not incorrect. Change-Id: I2f4a0a3e18db93c3e6794ade9b0fee33d2e4b7dc (cherry picked from commit c0612f8166b7cd07895e7302bb59192abbb68c43)
Change-Id: I42ed77f559e3ee5b8c600d98457ee37803ef0ea6 (cherry picked from commit 139c6ec0fb1f930be9b64545262d2580f6cbc83e)
Added new parameter :paramref:`.Engine.dispose.close`, defaulting to True. When False, the engine disposal does not touch the connections in the old pool at all, simply dropping the pool and replacing it. This use case is so that when the original pool is transferred from a parent process, the parent process may continue to use those connections. Fixes: #7877 Change-Id: I88b0808442381ba5e50674787cdb64f0e77d8b54 (cherry picked from commit 87a0f7183de4e8454483c7348bf486265bfe1c4d)
Further clarified connection-level logging to indicate the BEGIN, ROLLBACK and COMMIT log messages do not actually indicate a real transaction when the AUTOCOMMIT isolation level is in use; messaging has been extended to include the BEGIN message itself, and the messaging has also been fixed to accommodate when the :class:`.Engine` level :paramref:`.create_engine.isolation_level` parameter was used directly. Fixes: #7853 Change-Id: Iafc78070737ad117f84262e4bde84b81a81e4ea1 (cherry picked from commit 56366924673f88e51c74d94058c11132a057ecfa)
Fixed regression in mariadbconnector dialect as of mariadb connector 1.0.10 where the DBAPI no longer pre-buffers cursor.lastrowid. The dialect now fetches this value proactively for situations where it applies. test_invalidate_on_results seems to pass for mariadbconnector now. the driver has likely changed how it buffers result sets. This is a major change for them to make in a point release so we might want to watch this in case they reverse course again. Fixes: #7738 Change-Id: I9610aae01d1ae42fa92ffbc7123a6948e40ec9dd (cherry picked from commit e120837b682a3a822c2dff136ad48b1ca9fb6ce2)
Adjusted the logging for key SQLAlchemy components including :class:`_engine.Engine`, :class:`_engine.Connection` to establish an appropriate stack level parameter, so that the Python logging tokens ``funcName`` and ``lineno`` when used in custom logging formatters will report the correct information, which can be useful when filtering log output; supported on Python 3.8 and above. Pull request courtesy Markus Gerstel. Fixes: #7612 Closes: #7615 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7615 Pull-request-sha: cf9567beb06680df320cb12dde1f15baa68e1eb5 Change-Id: Iff23c92ef3453ac93cbd0d190e7efbf8ea4457a2 (cherry picked from commit f24a34140f6007cada900a8ae5ed03fe40ce2631)