summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/dialects
Commit message (Collapse)AuthorAgeFilesLines
* Replace c extension with cython versions.workflow_test_cythonFederico Caselli2021-12-176-6/+6
| | | | | | | | | | | | | | | Re-implement c version immutabledict / processors / resultproxy / utils with cython. Performance is in general in par or better than the c version Added a collection module that has cython version of OrderedSet and IdentitySet Added a new test/perf file to compare the implementations. Run ``python test/perf/compiled_extensions.py all`` to execute the comparison test. See results here: https://docs.google.com/document/d/1nOcDGojHRtXEkuy4vNXcW_XOJd9gqKhSeALGG3kYr6A/edit?usp=sharing Fixes: #7256 Change-Id: I2930ef1894b5048210384728118e586e813f6a76 Signed-off-by: Federico Caselli <cfederico87@gmail.com>
* include InterfaceError for mariadb disconnect checkMike Bayer2021-12-161-9/+14
| | | | | | | | | | | | | | | | | Corrected the error classes inspected for the "is_disconnect" check for the ``mariadbconnector`` dialect, which was failing for disconnects that occurred due to common MySQL/MariaDB error codes such as 2006; the DBAPI appears to currently use the ``mariadb.InterfaceError`` exception class for disconnect errors such as error code 2006, which has been added to the list of classes checked. For the current "real reconnect test", shutting down the mariadb connection from the client side produces ProgrammingError("Connection isn't valid anymore") which we also continue to intercept. Fixes: #7457 Change-Id: I0b37cd7a73359a23ad756ff2af0a9333c841221b
* Update references to tables in PostgreSQL documentationLeo Singer2021-12-101-4/+4
| | | | | Since the table numbers change from one version of PostgreSQL to the next, refer to the tables by name rather than by number.
* Warn when caching is disabled / documentMike Bayer2021-12-067-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds new warnings for all elements that don't indicate their caching behavior, including user-defined ClauseElement subclasses and third party dialects. it additionally adds new documentation to discuss an apparent performance degradation in 1.4 when caching is disabled as a result in the significant expense incurred by ORM lazy loaders, which in 1.3 used BakedQuery so were actually cached. As a result of adding the warnings, a fair degree of lesser used SQL expression objects identified that they did not define caching behavior so would have been producing ``[no key]``, including PostgreSQL constructs ``hstore`` and ``array``. These have been amended to use inherit cache where appropriate. "on conflict" constructs in PostgreSQL, MySQL, SQLite still explicitly don't generate a cache key at this time. The change also adds a test for all constructs via assert_compile() to assert they will not generate cache warnings. Fixes: #7394 Change-Id: I85958affbb99bfad0f5efa21bc8f2a95e7e46981
* Merge "provide connectionfairy on initialize" into mainmike bayer2021-11-2914-112/+95
|\
| * provide connectionfairy on initializeMike Bayer2021-11-2914-112/+95
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is so that dialect methods that are called within init can assume the same argument structure as when they are called in other places; we can nail down the type of object as well. This change seems to mostly impact the isolation level routines in the dialects, as these are called during initialize() as well as on established connections. these methods can now assume a non-proxied DBAPI connection object in all cases, as it is commonly required that attributes like ".autocommit" are set on the object which don't work well in a proxied situation. Other changes: * adds an interface for the "connectionfairy" concept called PoolProxiedConnection. * Removes ``Connectable`` superclass of Connection. ``Connectable`` was originally meant to provide for the "method which accepts connection or engine" theme. As this pattern is greatly reduced in 2.0 and Engine no longer extends from it, the ``Connectable`` superclass doesnt serve any real purpose. Leading from that, to set this in I also applied pep 484 annotations to the Dialect base, and then in the interests of seeing some of the typing information show up in my IDE did a little bit for Engine, Connection and others. I hope that it's feasible that we can add annotations to specific classes and attributes ahead of when we actually try to mass-populate the whole library. This was the original spirit of pep-484 that we can apply annotations gradually. I do of course want to try to do a mass-populate although i think even in that case we will end up doing a lot of manual work anyway (in particular for the changes here which are distinct from what the stubs have). Fixes: #7122 Change-Id: I5dd7fbff8a7ae520a81c165091af12a6a68826db
* | Fix a typo in psycopg_async module name (#7369)Denis Laxalde2021-11-271-2/+2
|/
* Added support for ``psycopg`` dialect.Federico Caselli2021-11-267-158/+853
| | | | | | | Both sync and async versions are supported. Fixes: #6842 Change-Id: I57751c5028acebfc6f9c43572562405453a2f2a4
* Merge "propose emulated setinputsizes embedded in the compiler" into mainmike bayer2021-11-256-193/+105
|\
| * propose emulated setinputsizes embedded in the compilerMike Bayer2021-11-236-193/+105
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a new system so that PostgreSQL and other dialects have a reliable way to add casts to bound parameters in SQL statements, replacing previous use of setinputsizes() for PG dialects. rationale: 1. psycopg3 will be using the same SQLAlchemy-side "setinputsizes" as asyncpg, so we will be seeing a lot more of this 2. the full rendering that SQLAlchemy's compilation is performing is in the engine log as well as error messages. Without this, we introduce three levels of SQL rendering, the compiler, the hidden "setinputsizes" in SQLAlchemy, and then whatever the DBAPI driver does. With this new approach, users reporting bugs etc. will be less confused that there are as many as two separate layers of "hidden rendering"; SQLAlchemy's rendering is again fully transparent 3. calling upon a setinputsizes() method for every statement execution is expensive. this way, the work is done behind the caching layer 4. for "fast insertmany()", I also want there to be a fast approach towards setinputsizes. As it was, we were going to be taking a SQL INSERT with thousands of bound parameter placeholders and running a whole second pass on it to apply typecasts. this way, we will at least be able to build the SQL string once without a huge second pass over the whole string 5. psycopg2 can use this same system for its ARRAY casts 6. the general need for PostgreSQL to have lots of type casts is now mostly in the base PostgreSQL dialect and works independently of a DBAPI being present. dependence on DBAPI symbols that aren't complete / consistent / hashable is removed I was originally going to try to build this into bind_expression(), but it was revealed this worked poorly with custom bind_expression() as well as empty sets. the current impl also doesn't need to run a second expression pass over the POSTCOMPILE sections, which came out better than I originally thought it would. Change-Id: I363e6d593d059add7bcc6d1f6c3f91dd2e683c0c
* | Clean up most py3k compatFederico Caselli2021-11-2425-151/+88
|/ | | | Change-Id: I8172fdcc3103ff92aa049827728484c8779af6b7
* Support lightweight compiler column elements w/ slotsMike Bayer2021-11-221-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | the _CompileLabel class included ``__slots__`` but these weren't used as the superclasses included slots. Create a ``__slots__`` superclass for ``ClauseElement``, creating a new class of compilable SQL elements that don't include heavier features like caching, annotations and cloning, which are meant to be used only in an ad-hoc compiler fashion. Create new ``CompilerColumnElement`` from that which serves in column-oriented contexts, but similarly does not include any expression operator support as it is intended to be used only to generate a string. Apply this to both ``_CompileLabel`` as well as PostgreSQL ``_ColonCast``, which does not actually subclass ``ColumnElement`` as this class has memoized attributes that aren't worth changing, and does not include SQL operator capabilities as these are not needed for these compiler-only objects. this allows us to more inexpensively add new ad-hoc labels / casts etc. at compile time, as we will be seeking to expand out the typecasts that are needed for PostgreSQL dialects in a subsequent patch. Change-Id: I52973ae3295cb6e2eb0d7adc816c678a626643ed
* Remove object in class definitionFederico Caselli2021-11-2210-14/+14
| | | | | References: #4600 Change-Id: I2a62ddfe00bc562720f0eae700a497495d7a987a
* Merge "fully support isolation_level parameter in base dialect" into mainmike bayer2021-11-1812-187/+53
|\
| * fully support isolation_level parameter in base dialectMike Bayer2021-11-1812-187/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Generalized the :paramref:`_sa.create_engine.isolation_level` parameter to the base dialect so that it is no longer dependent on individual dialects to be present. This parameter sets up the "isolation level" setting to occur for all new database connections as soon as they are created by the connection pool, where the value then stays set without being reset on every checkin. The :paramref:`_sa.create_engine.isolation_level` parameter is essentially equivalent in functionality to using the :paramref:`_engine.Engine.execution_options.isolation_level` parameter via :meth:`_engine.Engine.execution_options` for an engine-wide setting. The difference is in that the former setting assigns the isolation level just once when a connection is created, the latter sets and resets the given level on each connection checkout. Fixes: #6342 Change-Id: Id81d6b1c1a94371d901ada728a610696e09e9741
* | Merge "Add new sections regarding schemas and reflection" into mainmike bayer2021-11-181-25/+25
|\ \
| * | Add new sections regarding schemas and reflectionjonathan vanasco2021-11-171-25/+25
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | * add a new section to reflection.rst `Schemas and Reflection`. * this contains some text from the ticket * migrate some text from `Specifying the Schema Name` to new section * migrate some text from PostgreSQL dialect to new section * target text is made more generic * cross-reference the postgres and new sections to one another, to avoid duplication of docs * update some docs 'meta' to 'metadata_obj' Fixes: #4387 Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Change-Id: I2b08672753fb2575d30ada07ead77587468fdade
* | Merge "Deprecate create_engine.implicit_returning" into mainmike bayer2021-11-181-3/+7
|\ \ | |/ |/|
| * Deprecate create_engine.implicit_returningjonathan vanasco2021-11-091-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The :paramref:`_sa.create_engine.implicit_returning` parameter is deprecated on the :func:`_sa.create_engine` function only; the parameter remains available on the :class:`_schema.Table` object. This parameter was originally intended to enable the "implicit returning" feature of SQLAlchemy when it was first developed and was not enabled by default. Under modern use, there's no reason this parameter should be disabled, and it has been observed to cause confusion as it degrades performance and makes it more difficult for the ORM to retrieve recently inserted server defaults. The parameter remains available on :class:`_schema.Table` to specifically suit database-level edge cases which make RETURNING infeasible, the sole example currently being SQL Server's limitation that INSERT RETURNING may not be used on a table that has INSERT triggers on it. Also removed from the Oracle dialect some logic that would upgrade an Oracle 8/8i server version to use implicit returning if the parameter were explictly passed; these versions of Oracle still support RETURNING so the feature is now enabled for all Oracle versions. Fixes: #6962 Change-Id: Ib338e300cd7c8026c3083043f645084a8211aed8
* | Merge "removals: all unicode encoding / decoding" into mainmike bayer2021-11-118-158/+28
|\ \
| * | removals: all unicode encoding / decodingMike Bayer2021-11-108-158/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Removed here includes: * convert_unicode parameters * encoding create_engine() parameter * description encoding support * "non-unicode fallback" modes under Python 2 * String symbols regarding Python 2 non-unicode fallbacks * any concept of DBAPIs that don't accept unicode statements, unicode bound parameters, or that return bytes for strings anywhere except an explicit Binary / BLOB type * unicode processors in Python / C Risk factors: * Whether all DBAPIs do in fact return Unicode objects for all entries in cursor.description now * There was logic for mysql-connector trying to determine description encoding. A quick test shows Unicode coming back but it's not clear if there are still edge cases where they return bytes. if so, these are bugs in that driver, and at most we would only work around it in the mysql-connector DBAPI itself (but we won't do that either). * It seems like Oracle 8 was not expecting unicode bound parameters. I'm assuming this was all Python 2 stuff and does not apply for modern cx_Oracle under Python 3. * third party dialects relying upon built in unicode encoding/decoding but it's hard to imagine any non-SQLAlchemy database driver not dealing exclusively in Python unicode strings in Python 3 Change-Id: I97d762ef6d4dd836487b714d57d8136d0310f28a References: #7257
* | | Merge "change the POSTCOMPILE/ SCHEMA symbols to not conflict w mssql ↵mike bayer2021-11-091-0/+3
|\ \ \ | |/ / |/| | | | | quoting" into main
| * | change the POSTCOMPILE/ SCHEMA symbols to not conflict w mssql quotingMike Bayer2021-11-091-0/+3
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adjusted the compiler's generation of "post compile" symbols including those used for "expanding IN" as well as for the "schema translate map" to not be based directly on plain bracketed strings with underscores, as this conflicts directly with SQL Server's quoting format of also using brackets, which produces false matches when the compiler replaces "post compile" and "schema translate" symbols. The issue created easy to reproduce examples both with the :meth:`.Inspector.get_schema_names` method when used in conjunction with the :paramref:`_engine.Connection.execution_options.schema_translate_map` feature, as well in the unlikely case that a symbol overlapping with the internal name "POSTCOMPILE" would be used with a feature like "expanding in". Fixes: #7300 Change-Id: I6255c850b140522a4aba95085216d0bca18ce230
* | De-emphasize notion of "default driver" (DBAPI)Gord Thompson2021-11-095-9/+9
|/ | | | | | | | | | | Fixes: #6960 Even though a default driver still exists for each dialect, remove most usages of `dialect://` to encourage users to explicitly specify `dialect+driver://` Change-Id: I0ad42167582df509138fca64996bbb53e379b1af
* fully implement future engine and remove legacyMike Bayer2021-11-073-17/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | The major action here is to lift and move future.Connection and future.Engine fully into sqlalchemy.engine.base. This removes lots of engine concepts, including: * autocommit * Connection running without a transaction, autobegin is now present in all cases * most "autorollback" is obsolete * Core-level subtransactions (i.e. MarkerTransaction) * "branched" connections, copies of connections * execution_options() returns self, not a new connection * old argument formats, distill_params(), simplifies calling scheme between engine methods * before/after_execute() events (oriented towards compiled constructs) don't emit for exec_driver_sql(). before/after_cursor_execute() is still included for this * old helper methods superseded by context managers, connection.transaction(), engine.transaction() engine.run_callable() * ancient engine-level reflection methods has_table(), table_names() * sqlalchemy.testing.engines.proxying_engine References: #7257 Change-Id: Ib20ed816642d873b84221378a9ec34480e01e82c
* simplify and publicize the asyncpg JSON(B) codec registrsationMike Bayer2021-11-031-29/+55
| | | | | | | | | | | | Added overridable methods ``PGDialect_asyncpg.setup_asyncpg_json_codec`` and ``PGDialect_asyncpg.setup_asyncpg_jsonb_codec`` codec, which handle the required task of registering JSON/JSONB codecs for these datatypes when using asyncpg. The change is that methods are broken out as individual, overridable methods to support third party dialects that need to alter or disable how these particular codecs are set up. Fixes: #7284 Change-Id: I3eac258fea61f3975bd03c428747f788813ce45e
* Revert "Gracefully degrade unsupported types with asyncpg"mike bayer2021-11-031-29/+14
| | | | | | | | This reverts commit 96c294da8a50d692b3f0b8e508dbbca5d9c22f1b. I have another approach that is more obvious, easier to override explicitly and also I can test it more easily. Change-Id: I11a3be7700dbc6f25d436e450b6fb8e8f6c4fd16
* formatting updatesMike Bayer2021-11-031-2/+3
| | | | Change-Id: I7352bed0115b8fcdb4708e012d83e81d1ae494ed
* Merge "Gracefully degrade unsupported types with asyncpg" into mainmike bayer2021-11-031-14/+29
|\
| * Gracefully degrade unsupported types with asyncpgGord Thompson2021-11-021-14/+29
| | | | | | | | | | | | | | | | | | | | | | | | Fixes: #7284 Modify the on_connect() method of PGDialect_asyncpg to gracefully degrade unsupported types instead of throwing a ValueError. Useful for third-party dialects that derive from PGDialect_asyncpg but whose databases do not support all types (e.g., CockroachDB supports JSONB but not JSON). Change-Id: Ibb7cc8c3de632d27b9716a93d83956a590b2a2b0
* | Merge "Fixed issue in visit_on_duplicate_key_update within a composed ↵mike bayer2021-11-031-1/+1
|\ \ | | | | | | | | | expression" into main
| * | Fixed issue in visit_on_duplicate_key_update within a composed expressionCristian Sabaila2021-11-021-1/+1
| |/ | | | | | | | | | | | | | | | | | | | | | | | | Fixed issue in MySQL :meth:`_mysql.Insert.on_duplicate_key_update` which would render the wrong column name when an expression were used in a VALUES expression. Pull request courtesy Cristian Sabaila. Fixes: #7281 Closes: #7285 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7285 Pull-request-sha: 3e6ad6f2fecc6ae36a10a5a34b5d3d393483edbb Change-Id: I83377c20eae6358fead9e7e361127938e538a71c
* | map Float to asyncpg.FLOAT, test for infinityMike Bayer2021-11-021-0/+9
|/ | | | | Fixes: #7283 Change-Id: I5402a72617b7f9bc366d64bc5ce8669374839984
* First round of removal of python 2Federico Caselli2021-11-016-158/+27
| | | | | References: #4600 Change-Id: I61e35bc93fe95610ae75b31c18a3282558cd4ffe
* Update psycopg2 dialect to use the DBAPI interface to executeFederico Caselli2021-10-312-10/+41
| | | | | | | two phase transactions Fixes: #7238 Change-Id: Ie4f5cf59d29b5bfc142ec2dfdecffb896ee7d708
* Merge "Remove deprecated dialects and drivers" into mainmike bayer2021-10-3118-3597/+5
|\
| * Remove deprecated dialects and driversFederico Caselli2021-10-3118-3597/+5
| | | | | | | | | | Fixes: #7258 Change-Id: I3577f665eca04f2632b69bcb090f0a4ec9271db9
* | The ``has_table`` method now also checks viewsFederico Caselli2021-10-312-4/+14
|/ | | | | | | | | | | | The :meth:`_engine.Inspector.has_table` method will now consistently check for views of the given name as well as tables. Previously this behavior was dialect dependent, with PostgreSQL, MySQL/MariaDB and SQLite supporting it, and Oracle and SQL Server not supporting it. Third party dialects should also seek to ensure their :meth:`_engine.Inspector.has_table` method searches for views as well as tables for the given name. Fixes: #7161 Change-Id: I9e523c76741b19596c81ef577dc6f0823e44183b
* Merge "Improve array support on pg8000" into mainmike bayer2021-10-282-1/+9
|\
| * Improve array support on pg8000Federico Caselli2021-10-272-1/+9
| | | | | | | | | | | | References: #6023 Change-Id: I0f6cbc34b3c0bfc0b8c86b3ebe4531e23039b6c0
* | Fixes: #5020jonathan vanasco2021-10-271-7/+68
| | | | | | | | | | | | Add mike's example to docs Change-Id: I96a79084cccca5c792bee697338422f3de0884fb
* | add additional "oracle mode" reserved wordsMike Bayer2021-10-261-3/+19
|/ | | | | | | | | | despite mariadb's docs, the word "system" must be quoted in plain mariadb 10.5, not sure if that's "oracle mode" but it is > 10.3. observed keystone tests failing on a column of this name. Fixes: #7167 Change-Id: I153b79809e7da007bcceecec96c1340d1641f0de
* fix: Update reserved words list of MySQL / MariaDB dialectKevin Kirsche2021-10-205-308/+563
| | | | | | | | | | | | | | | | | | | | | | | Reorganized the list of reserved words into two separate lists, one for MySQL and one for MariaDB, so that these diverging sets of words can be managed more accurately; adjusted the MySQL/MariaDB dialect to switch among these lists based on either explicitly configured or server-version-detected "MySQL" or "MariaDB" backend. Added all current reserved words through MySQL 8 and current MariaDB versions including recently added keywords like "lead" . Pull request courtesy Kevin Kirsche. 1. Move reserved words to it's own file. 2. Add missing reserved words from https://mariadb.com/kb/en/reserved-words/ * Note: this only adds MariaDB though links to MySQL, it also does not include the reserved words for Oracle mode, as listed in the link. Fixes: #7167 Supercedes: #7197 Closes: #7207 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7207 Pull-request-sha: 5a682e331069520ccad9e6bf3cc5e4a77a889ef0 Change-Id: Ib25be8148568899f56b5c9b42d4f530ade8a04e3
* re-enable asnycmy >= 0.2.3; disable aiomysql totallyMike Bayer2021-10-151-3/+6
| | | | Change-Id: I94bc22db0462509003c8241d0ccef915fe6513c6
* Merge "support bind expressions w/ expanding IN; apply to psycopg2" into mainmike bayer2021-10-153-21/+24
|\
| * support bind expressions w/ expanding IN; apply to psycopg2Mike Bayer2021-10-153-21/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed issue where "expanding IN" would fail to function correctly with datatypes that use the :meth:`_types.TypeEngine.bind_expression` method, where the method would need to be applied to each element of the IN expression rather than the overall IN expression itself. Fixed issue where IN expressions against a series of array elements, as can be done with PostgreSQL, would fail to function correctly due to multiple issues within the "expanding IN" feature of SQLAlchemy Core that was standardized in version 1.4. The psycopg2 dialect now makes use of the :meth:`_types.TypeEngine.bind_expression` method with :class:`_types.ARRAY` to portably apply the correct casts to elements. The asyncpg dialect was not affected by this issue as it applies bind-level casts at the driver level rather than at the compiler level. as part of this commit the "bind translate" feature has been simplified and also applies to the names in the POSTCOMPILE tag to accommodate for brackets. Fixes: #7177 Change-Id: I08c703adb0a9bd6f5aeee5de3ff6f03cccdccdc5
* | Fix reflection of FK against a unique indexGord Thompson2021-10-141-24/+135
|/ | | | | | | | Also implement reflection of ON DELETE, ON UPDATE as the data is right there. Fixes: #7160 Change-Id: Ifff871a8cb1d1bea235616042e16ed3b5c5f19f9
* Fix has_table() false negative for #temp tablesGord Thompson2021-10-121-8/+12
| | | | | | | | | | | Fixed issue with :meth:`.Inspector.has_table` where it would return False if a local temp table with the same name from a different session happened to be returned first when querying tempdb. This is a continuation of :ticket:`6910` which accounted for the temp table existing only in the alternate session and not the current one. Fixes: #7168 Change-Id: I19dbb71a63184c6d41822b0e882b7b284ac83786
* Merge "Add autocommit documentation for Azure SQL DW"mike bayer2021-10-061-3/+42
|\
| * Add autocommit documentation for Azure SQL DWGord Thompson2021-09-281-3/+42
| | | | | | | | | | Fixes: #7065 Change-Id: I3666e16f9a8c47f9784ea11f38972cda6b08c146
View Lorry Controller Status