summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/testing
Commit message (Collapse)AuthorAgeFilesLines
...
| * don't invoke fromclause.c when creating an annotatedMike Bayer2022-11-151-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The ``aliased()`` constructor calls upon ``__clause_element__()``, which internally annotates a ``FromClause``, like a subquery. This became expensive as ``AnnotatedFromClause`` has for many years called upon ``element.c`` so that the full ``.c`` collection is transferred to the Annotated. Taking this out proved to be challenging. A straight remove seemed to not break any tests except for the one that tested the exact condition. Nevertheless this seemed "spooky" so I instead moved the get of ``.c`` to be in a memoized proxy method. However, that then exposed a recursion issue related to loader_criteria; so the source of that behavior, which was an accidental behavioral artifact, is now made into an explcicit option that loader_criteria uses directly. The accidental behavioral artifact in question is still kind of strange since I was not able to fully trace out how it works, but the end result is that fixing the artifact to be "correct" causes loader_criteria, within the particular test for #7491, creates a select/ subquery structure with a cycle in it, so compilation fails with recursion overflow. The "solution" is to cause the artifact to occur in this case, which is that the ``AnnotatedFromClause`` will have a different ``.c`` collection than its element, which is a subquery. It's not totally clear how a cycle is generated when this is not done. This is commit one of two, which goes through some hoops to make essentially a one-line change. The next commit will rework ColumnCollection to optimize the corresponding_column() method significantly. Fixes: #8796 Change-Id: Id58ae6554db62139462c11a8be7313a3677456ad
* | Repair test in 32bit python builds.Federico Caselli2022-11-121-0/+4
|/ | | | Change-Id: I8287f3e1a975534c8a01a41c9dcc7e5e9f08bb52
* Merge "ensure anon_map is passed for most annotated traversals" into mainmike bayer2022-11-113-0/+75
|\
| * ensure anon_map is passed for most annotated traversalsMike Bayer2022-11-113-0/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We can cache the annotated cache key for Table, but for selectables it's not safe, as it fails to pass the anon_map along and creates many redudant structures in observed test scenario. It is likely safe for a Column that's mapped to a Table also, however this is not implemented here. Will have to see if that part needs adjusting. Fixed critical memory issue identified in cache key generation, where for very large and complex ORM statements that make use of lots of ORM aliases with subqueries, cache key generation could produce excessively large keys that were orders of magnitude bigger than the statement itself. Much thanks to Rollo Konig Brock for their very patient, long term help in finally identifying this issue. Also within TypeEngine objects, when we generate elements for instance variables, skip the None elements at least. this also saves on tuple complexity. Fixes: #8790 Change-Id: I448ddbfb45ae0a648815be8dad4faad7d1977427
* | repair --disable-asyncio parameterMike Bayer2022-11-112-1/+5
|/ | | | | | | | | | | Fixed issue where the ``--disable-asyncio`` parameter to the test suite would fail to not actually run greenlet tests and would also not prevent the suite from using a "wrapping" greenlet for the whole suite. This parameter now ensures that no greenlet or asyncio use will occur within the entire run when set. Fixes: #8793 Change-Id: I87b510846b2cc24413cd54e7b7136e91aad3c309
* Improve typings of execution optionsFederico Caselli2022-11-021-0/+6
| | | | | Fixes: #8605 Change-Id: I4aec83b9f321462427c3f4ac941c3b272255c088
* use only object_id() function for temp tablesMike Barry2022-10-282-13/+69
| | | | | | | | | | | | | | | | | | Fixed issue with :meth:`.Inspector.has_table` when used against a temporary table for the SQL Server dialect would fail an invalid object name error on some Azure variants, due to an unnecessary information schema query that is not supported on those server versions. Pull request courtesy Mike Barry. the patch also fills out test support for has_table() against temp tables, temp views, adding to the has_table() support just added for views in #8700. Fixes: #8714 Closes: #8716 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8716 Pull-request-sha: e2ac7a52e2b09a349a703ba1e1a2911f4d3c0912 Change-Id: Ia73e4e9e977a2d6b7e100abd2f81a8c8777dc9bb
* add Oracle-specific parameter escapes for expanding paramsMike Bayer2022-10-241-0/+9
| | | | | | | | | | | | | | Fixed issue where bound parameter names, including those automatically derived from similarly-named database columns, which contained characters that normally require quoting with Oracle would not be escaped when using "expanding parameters" with the Oracle dialect, causing execution errors. The usual "quoting" for bound parameters used by the Oracle dialect is not used with the "expanding parameters" architecture, so escaping for a large range of characters is used instead, now using a list of characters/escapes that are specific to Oracle. Fixes: #8708 Change-Id: I90c24e48534e1b3a4c222b3022da58159784d91a
* test support for has_table()->view; backport to 1.4Mike Bayer2022-10-231-17/+24
| | | | | | | | | | | | | For 1.4 only; in 2.0 this just refines the test suite a bit. Fixed regression which occurred throughout the 1.4 series where the :meth:`.Inspector.has_table` method, which historically reported on views as well, stopped working for SQL Server. The issue is not present in the 2.0 series which uses a different reflection architecture. Test support is added to ensure ``has_table()`` remains working per spec re: views. Fixes: #8700 Change-Id: I119a91ec07911edb08cf0799234827fec9ea1195
* Merge "further qualify pyodbc setinputsizes types for long stirngs" into mainmike bayer2022-10-192-8/+25
|\
| * further qualify pyodbc setinputsizes types for long stirngsMike Bayer2022-10-182-8/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression caused by SQL Server pyodbc change :ticket:`8177` where we now use ``setinputsizes()`` by default; for VARCHAR, this fails if the character size is greater than 4000 (or 2000, depending on data) characters as the incoming datatype is NVARCHAR, which has a limit of 4000 characters, despite the fact that VARCHAR can handle unlimited characters. Additional pyodbc-specific typing information is now passed to ``setinputsizes()`` when the datatype's size is > 2000 characters. The change is also applied to the :class:`.JSON` type which was also impacted by this issue for large JSON serializations. Fixes: #8661 Change-Id: I07fa873e95dbd2c94f3d286e93e8b3229c3a9807
* | Revert automatic set of sequence start to 1Federico Caselli2022-10-172-16/+53
|/ | | | | | | | | | | | | | | | | The :class:`.Sequence` construct restores itself to the DDL behavior it had prior to the 1.4 series, where creating a :class:`.Sequence` with no additional arguments will emit a simple ``CREATE SEQUENCE`` instruction **without** any additional parameters for "start value". For most backends, this is how things worked previously in any case; **however**, for MS SQL Server, the default value on this database is ``-2**63``; to prevent this generally impractical default from taking effect on SQL Server, the :paramref:`.Sequence.start` parameter should be provided. As usage of :class:`.Sequence` is unusual for SQL Server which for many years has standardized on ``IDENTITY``, it is hoped that this change has minimal impact. Fixes: #7211 Change-Id: I1207ea10c8cb1528a1519a0fb3581d9621c27b31
* implement write-only colletions, typing for dynamicMike Bayer2022-10-062-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For 2.0, we provide a truly "larger than memory collection" implementation, a write-only collection that will never under any circumstances implicitly load the entire collection, even during flush. This is essentially a much more "strict" version of the "dynamic" loader, which in fact has a lot of scenarios that it loads the full backing collection into memory, mostly defeating its purpose. Typing constructs are added that support both the new feature WriteOnlyMapping as well as the legacy feature DynamicMapping. These have been integrated with "annotion based mapping" so that relationship() uses these annotations to configure the loader strategy as well. additional changes: * the docs triggered a conflict in hybrid's "transformers" section, this section is hard-coded to Query using a pattern that doesnt seem to have any use and isn't part of the current select() interface, so just removed this section * As the docs for WriteOnlyMapping are very long, collections.rst is broken up into two pages now. Fixes: #6229 Fixes: #7123 Change-Id: I6929f3da6e441cad92285e7309030a9bac4e429d
* Merge "`aggregate_order_by` now supports cache generation." into mainmike bayer2022-09-261-0/+108
|\
| * `aggregate_order_by` now supports cache generation.Federico Caselli2022-09-251-0/+108
| | | | | | | | | | | | | | | | | | also adjusted CacheKeyFixture to be a general purpose fixture so that sub-components / dialects can run their own cache key tests. Fixes: #8574 Change-Id: I6c66107856aee11e548d357cea77bceee3e316a0
* | New ORM Query Guide featuring DML supportMike Bayer2022-09-251-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | reviewers: these docs publish periodically at: https://docs.sqlalchemy.org/en/gerrit/4042/orm/queryguide/index.html See the "last generated" timestamp near the bottom of the page to ensure the latest version is up Change includes some other adjustments: * small typing fixes for end-user benefit * removal of a bunch of old examples for patterns that nobody uses or aren't really what we promote now * modernization of some examples, including inheritance Change-Id: I9929daab7797be9515f71c888b28af1209e789ff
* | ORM bulk insert via executeMike Bayer2022-09-243-13/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * ORM Insert now includes "bulk" mode that will run essentially the same process as session.bulk_insert_mappings; interprets the given list of values as ORM attributes for key names * ORM UPDATE has a similar feature, without RETURNING support, for session.bulk_update_mappings * Added support for upserts to do RETURNING ORM objects as well * ORM UPDATE/DELETE with list of parameters + WHERE criteria is a not implemented; use connection * ORM UPDATE/DELETE defaults to "auto" synchronize_session; use fetch if RETURNING is present, evaluate if not, as "fetch" is much more efficient (no expired object SELECT problem) and less error prone if RETURNING is available UPDATE: howver this is inefficient! please continue to use evaluate for simple cases, auto can move to fetch if criteria not evaluable * "Evaluate" criteria will now not preemptively unexpire and SELECT attributes that were individually expired. Instead, if evaluation of the criteria indicates that the necessary attrs were expired, we expire the object completely (delete) or expire the SET attrs unconditionally (update). This keeps the object in the same unloaded state where it will refresh those attrs on the next pass, for this generally unusual case. (originally #5664) * Core change! update/delete rowcount comes from len(rows) if RETURNING was used. SQLite at least otherwise did not support this. adjusted test_rowcount accordingly * ORM DELETE with a list of parameters at all is also a not implemented as this would imply "bulk", and there is no bulk_delete_mappings (could be, but we dont have that) * ORM insert().values() with single or multi-values translates key names based on ORM attribute names * ORM returning() implemented for insert, update, delete; explcit returning clauses now interpret rows in an ORM context, with support for qualifying loader options as well * session.bulk_insert_mappings() assigns polymorphic identity if not set. * explicit RETURNING + synchronize_session='fetch' is now supported with UPDATE and DELETE. * expanded return_defaults() to work with DELETE also. * added support for composite attributes to be present in the dictionaries used by bulk_insert_mappings and bulk_update_mappings, which is also the new ORM bulk insert/update feature, that will expand the composite values into their individual mapped attributes the way they'd be on a mapped instance. * bulk UPDATE supports "synchronize_session=evaluate", is the default. this does not apply to session.bulk_update_mappings, just the new version * both bulk UPDATE and bulk INSERT, the latter with or without RETURNING, support *heterogenous* parameter sets. session.bulk_insert/update_mappings did this, so this feature is maintained. now cursor result can be both horizontally and vertically spliced :) This is now a long story with a lot of options, which in itself is a problem to be able to document all of this in some way that makes sense. raising exceptions for use cases we haven't supported is pretty important here too, the tradition of letting unsupported things just not work is likely not a good idea at this point, though there are still many cases that aren't easily avoidable Fixes: #8360 Fixes: #7864 Fixes: #7865 Change-Id: Idf28379f8705e403a3c6a937f6a798a042ef2540
* | implement batched INSERT..VALUES () () for executemanyMike Bayer2022-09-245-0/+220
|/ | | | | | | | | | | | | | | | | | | | the feature is enabled for all built in backends when RETURNING is used, except for Oracle that doesn't need it, and on psycopg2 and mssql+pyodbc it is used for all INSERT statements, not just those that use RETURNING. third party dialects would need to opt in to the new feature by setting use_insertmanyvalues to True. Also adds dialect-level guards against using returning with executemany where we dont have an implementation to suit it. execute single w/ returning still defers to the server without us checking. Fixes: #6047 Fixes: #7907 Change-Id: I3936d3c00003f02e322f2e43fb949d0e6e568304
* Tighten password security by removing `URL.__str__`Yassen Damyanov2022-09-232-3/+6
| | | | | | | | | | | | | | | For improved security, the :class:`_url.URL` object will now use password obfuscation by default when ``str(url)`` is called. To stringify a URL with cleartext password, the :meth:`_url.URL.render_as_string` may be used, passing the :paramref:`_url.URL.render_as_string.hide_password` parameter as ``False``. Thanks to our contributors for this pull request. Fixes: #8567 Closes: #8563 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8563 Pull-request-sha: d1f1127f753849eb70b8d6cc64badf34e1b9219b Change-Id: If756c8073ff99ac83876d9833c8fe1d7c76211f9
* Improve array_agg and Array processingFederico Caselli2022-09-171-0/+11
| | | | | | | | | The :class:`_functions.array_agg` will now set the array dimensions to 1. Improved :class:`_types.ARRAY` processing to accept ``None`` values as value of a multi-array. Fixes: #7083 Change-Id: Iafec4f77fde9719ccc7c8535bf6235dbfbc62102
* Use ``;`` instead of ``select 1`` to ping PostgreSQLFederico Caselli2022-09-151-0/+11
| | | | | Fixes: #8491 Change-Id: I941d2a3cf92e5609e2045a53cec94522340951db
* Use cibuildwheel to create wheelsTrevor Gross2022-08-292-4/+13
| | | | | | | | | | | | | | | | Using cibuildwheel the following wheels are created: - windows x64 and x84 - macos x64 and arm - linux x64 and arm on manylinux and mosulinux (for alpine) - create a pure python wheel (for pypy and other archs) Fixes: #6702 Fixes: #7607 Closes: #7992 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7992 Pull-request-sha: 61d5e24e5b76c97db73aa2507af7f5c2d3a948fa Change-Id: If0c0b353766e0b61d421789d619eb2b940c08ad0
* include mssql_clustered dialect_options when reflecting - issue #8288John Lennox2022-08-231-4/+9
| | | | | | | | | | | | Implemented reflection of the "clustered index" flag ``mssql_clustered`` for the SQL Server dialect. Pull request courtesy John Lennox. Fixes: #8288 Closes: #8289 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8289 Pull-request-sha: 1bb57352e3e31d8fb7de69ab5e60e5464949f640 Change-Id: Ife367066328f9e47ad823e4098647964a18e21e8
* Update to flake8 5.Federico Caselli2022-07-311-3/+2
| | | | Change-Id: I5a241a70efba68bcea9819ddce6aebc25703e68d
* Reflect expression-based indexes on PostgreSQLFederico Caselli2022-07-284-13/+83
| | | | | | | | | | | The PostgreSQL dialect now supports reflection of expression based indexes. The reflection is supported both when using :meth:`_engine.Inspector.get_indexes` and when reflecting a :class:`_schema.Table` using :paramref:`_schema.Table.autoload_with`. Thanks to immerrr and Aidan Kane for the help on this ticket. Fixes: #7442 Change-Id: I3e36d557235286c0f7f6d8276272ff9225058d48
* update ORM declarative docs for new featuresMike Bayer2022-07-161-0/+6
| | | | | | | I screwed up a rebase or something so this was temporarily in Ic51a12de3358f3a451bd7cf3542b375569499fc1 Change-Id: I847ee1336381221c0112b67854df022edf596b25
* repair yield_per for non-SS dialects and add new optionsMike Bayer2022-07-011-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implemented new :paramref:`_engine.Connection.execution_options.yield_per` execution option for :class:`_engine.Connection` in Core, to mirror that of the same :ref:`yield_per <orm_queryguide_yield_per>` option available in the ORM. The option sets both the :paramref:`_engine.Connection.execution_options.stream_results` option at the same time as invoking :meth:`_engine.Result.yield_per`, to provide the most common streaming result configuration which also mirrors that of the ORM use case in its usage pattern. Fixed bug in :class:`_engine.Result` where the usage of a buffered result strategy would not be used if the dialect in use did not support an explicit "server side cursor" setting, when using :paramref:`_engine.Connection.execution_options.stream_results`. This is in error as DBAPIs such as that of SQLite and Oracle already use a non-buffered result fetching scheme, which still benefits from usage of partial result fetching. The "buffered" strategy is now used in all cases where :paramref:`_engine.Connection.execution_options.stream_results` is set. Added :meth:`.FilterResult.yield_per` so that result implementations such as :class:`.MappingResult`, :class:`.ScalarResult` and :class:`.AsyncResult` have access to this method. Fixes: #8199 Change-Id: I6dde3cbe483a1bf81e945561b60f4b7d1c434750
* Comments on (named) constraintscheremnov2022-06-292-44/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adds support for comments on named constraints, including `ForeignKeyConstraint`, `PrimaryKeyConstraint`, `CheckConstraint`, `UniqueConstraint`, solving the [Issue 5667](https://github.com/sqlalchemy/sqlalchemy/issues/5667). Supports only PostgreSQL backend. ### Description Following the example of [Issue 1546](https://github.com/sqlalchemy/sqlalchemy/issues/1546), supports comments on constraints. Specifically, enables comments on _named_ ones — as I get it, PostgreSQL prohibits comments on unnamed constraints. Enables setting the comments for named constraints like this: ``` Table( 'example', metadata, Column('id', Integer), Column('data', sa.String(30)), PrimaryKeyConstraint( "id", name="id_pk", comment="id_pk comment" ), CheckConstraint('id < 100', name="cc1", comment="Id value can't exceed 100"), UniqueConstraint(['data'], name="uc1", comment="Must have unique data field"), ) ``` Provides the DDL representation for constraint comments and routines to create and drop them. Class `.Inspector` reflects constraint comments via methods like `get_check_constraints` . ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - [ ] A short code fix - [x] A new feature implementation - Solves the issue 5667. - The commit message includes `Fixes: 5667`. - Includes tests based on comment reflection. **Have a nice day!** Fixes: #5667 Closes: #7742 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7742 Pull-request-sha: 42a5d3c3e9ccf9a9d5397fd007aeab0854f66130 Change-Id: Ia60f578595afdbd6089541c9a00e37997ef78ad3
* merge column args from Annotated left sideMike Bayer2022-06-281-3/+19
| | | | | | | | | | | | because we are forced by pep-681 to use the argument "default", we need a way to have client Column default separate from a dataclasses level default. Also, pep-681 does not support deriving the descriptor function from Annotated, so allow a brief right side mapped_column() to be present that will have more column-centric arguments from the left side Annotated to be merged. Change-Id: I039be1628d498486ba013b2798e1392ed1cd7f9f
* Add ability to test using thick mode with oracledbFederico Caselli2022-06-202-13/+31
| | | | | Change-Id: Iee14750ba20422931bde4d61eaa570af482c7d8b References: #8147
* Merge "create new approach for deeply nested post loader options" into mainmike bayer2022-06-181-1/+1
|\
| * create new approach for deeply nested post loader optionsMike Bayer2022-06-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added very experimental feature to the :func:`_orm.selectinload` and :func:`_orm.immediateload` loader options called :paramref:`_orm.selectinload.recursion_depth` / :paramref:`_orm.immediateload.recursion_depth` , which allows a single loader option to automatically recurse into self-referential relationships. Is set to an integer indicating depth, and may also be set to -1 to indicate to continue loading until no more levels deep are found. Major internal changes to :func:`_orm.selectinload` and :func:`_orm.immediateload` allow this feature to work while continuing to make correct use of the compilation cache, as well as not using arbitrary recursion, so any level of depth is supported (though would emit that many queries). This may be useful for self-referential structures that must be loaded fully eagerly, such as when using asyncio. A warning is also emitted when loader options are connected together with arbitrary lengths (that is, without using the new ``recursion_depth`` option) when excessive recursion depth is detected in related object loading. This operation continues to use huge amounts of memory and performs extremely poorly; the cache is disabled when this condition is detected to protect the cache from being flooded with arbitrary statements. Fixes: #8126 Change-Id: I9f162e0a09c1ed327dd19498aac193f649333a01
* | Merge "rearchitect reflection for batched performance" into mainFederico Caselli2022-06-188-301/+1492
|\ \ | |/ |/|
| * rearchitect reflection for batched performanceFederico Caselli2022-06-188-301/+1492
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rearchitected the schema reflection API to allow some dialects to make use of high performing batch queries to reflect the schemas of many tables at once using much fewer queries. The new performance features are targeted first at the PostgreSQL and Oracle backends, and may be applied to any dialect that makes use of SELECT queries against system catalog tables to reflect tables (currently this omits the MySQL and SQLite dialects which instead make use of parsing the "CREATE TABLE" statement, however these dialects do not have a pre-existing performance issue with reflection. MS SQL Server is still a TODO). The new API is backwards compatible with the previous system, and should require no changes to third party dialects to retain compatibility; third party dialects can also opt into the new system by implementing batched queries for schema reflection. Along with this change is an updated reflection API that is fully :pep:`484` typed, features many new methods and some changes. Fixes: #4379 Change-Id: I897ec09843543aa7012bcdce758792ed3d415d08
* | Revert "add auto_recurse option to selectinload, immediateload"Mike Bayer2022-06-161-1/+1
| | | | | | | | | | | | | | | | | | | | this option works very badly with caching and the API is likely not what we want either. Work continues for #8126 including that the additional work in I9f162e0a09c1ed327dd19498aac193f649333a01 tries to add new recursive features. This reverts commit b3a1162553879d1369154e920f3f4129020bb88e.
* | implement literal stringification for arraysMike Bayer2022-06-151-0/+57
|/ | | | | | | | | | | | | | | as we already implement stringification for the contents, provide a bracketed syntax for default and ARRAY literal for PG specifically. ARRAY literal seems much simpler to render than their quoted syntax which requires double quotes for strings. also open up testing for pg8000 which has likely been fine with arrays for awhile now, bump the version pin also. Fixes: #8138 Change-Id: Id85b052b0a9564d6aa1489160e58b7359f130fdd
* add auto_recurse option to selectinload, immediateloadMike Bayer2022-06-111-1/+1
| | | | | | | | | | | | | | Added very experimental feature to the :func:`_orm.selectinload` and :func:`_orm.immediateload` loader options called :paramref:`_orm.selectinload.auto_recurse` / :paramref:`_orm.immediateload.auto_recurse` , which when set to True will cause a self-referential relationship load to continue loading with arbitrary depth until no further objects are found. This may be useful for self-referential structures that must be loaded fully eagerly, such as when using asyncio. Fixes: #8126 Change-Id: I5bbd00bd0ca43f4649b44680fea1e84680f0a5db
* resolve large ints to BigIntegerMike Bayer2022-06-101-2/+31
| | | | | | | | | | | | The in-place type detection for Python integers, as occurs with an expression such as ``literal(25)``, will now apply value-based adaption as well to accommodate Python large integers, where the datatype determined will be :class:`.BigInteger` rather than :class:`.Integer`. This accommodates for dialects such as that of asyncpg which both sends implicit typing information to the driver as well as is sensitive to numeric scale. Fixes: #7909 Change-Id: I1cd3ec2676c9bb03ffedb600695252bd0037ba02
* handle non-mapped mixins for with_loader_criteria reduceMike Bayer2022-06-091-0/+10
| | | | | | | | | special handling is needed for a with_loader_criteria() against a non-mapped mixin class. added that to test coverage Fixes: #8109 Change-Id: Ia599361c8faab008e92095eb4607d02820f590d5
* run test_update_rowcount_return_defaults only w/ returningMike Bayer2022-06-061-0/+1
| | | | | | | | | | | | | | | sane_rowcount_w_returning asserts failure, which will only occur here if the DBAPI actually uses RETURNING. as SQLite conditionally supports RETURNING which breaks rowcount support only if present, limit this test to that case. Additionally, newer pysqlites will likely fix the issue so we will probably want to put a sqlite3_version check as well once that fix is released. Change-Id: I065aa181eb48363c1024550ae3622486ae0b4a6e
* Generalize RETURNING and suppor for MariaDB / SQLiteDaniel Black2022-06-024-32/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As almost every dialect supports RETURNING now, RETURNING is also made more of a default assumption. * the default compiler generates a RETURNING clause now when specified; CompileError is no longer raised. * The dialect-level implicit_returning parameter now has no effect. It's not fully clear if there are real world cases relying on the dialect-level parameter, so we will see once 2.0 is released. ORM-level RETURNING can be disabled at the table level, and perhaps "implicit returning" should become an ORM-level option at some point as that's where it applies. * Altered ORM update() / delete() to respect table-level implicit returning for fetch. * Since MariaDB doesnt support UPDATE returning, "full_returning" is now split into insert_returning, update_returning, delete_returning * Crazy new thing. Dialects that have *both* cursor.lastrowid *and* returning. so now we can pick between them for SQLite and mariadb. so, we are trying to keep it on .lastrowid for simple inserts with an autoincrement column, this helps with some edge case test scenarios and i bet .lastrowid is faster anyway. any return_defaults() / multiparams etc then we use returning * SQLite decided they dont want to return rows that match in ON CONFLICT. this is flat out wrong, but for now we need to work with it. Fixes: #6195 Fixes: #7011 Closes: #7047 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7047 Pull-request-sha: d25d5ea3abe094f282c53c7dd87f5f53a9e85248 Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Change-Id: I9908ce0ff7bdc50bd5b27722081767c31c19a950
* add backend agnostic UUID datatypeMike Bayer2022-06-013-16/+166
| | | | | | | | | | | | | | | | | | | | Added new backend-agnostic :class:`_types.Uuid` datatype generalized from the PostgreSQL dialects to now be a core type, as well as migrated :class:`_types.UUID` from the PostgreSQL dialect. Thanks to Trevor Gross for the help on this. also includes: * corrects some missing behaviors in the suite literal fixtures test where row round trips weren't being correctly asserted. * fixes some of the ISO literal date rendering added in 952383f9ee0 for #5052 to truncate datetime strings for date/time datatypes in the same way that drivers typically do for bound parameters; this was not working fully and wasn't caught by the broken test fixture Fixes: #7212 Change-Id: I981ac6d34d278c18281c144430a528764c241b04
* move bindparam quote application from compiler to defaultMike Bayer2022-05-291-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | in 296c84313ab29bf9599634f3 for #5653 we generalized Oracle's parameter escaping feature into the compiler, so that it could also work for PostgreSQL. The compiler used quoted names within parameter dictionaries, which then led to the complexity that all functions which interpreted keys from the compiled_params dict had to also quote the param names to use the dictionary. This extra complexity was not added to the ORM peristence.py however, which led to the versioning id feature being broken as well as other areas where persistence.py relies on naming schemes present in context.compiled_params. It also was not added to the "processors" lookup which led to #8053, that added this escaping to that part of the compiler. To both solve the whole problem as well as simplify the compiler quite a bit, move the actual application of the escaped names to be as late as possible, when default.py builds the final list of parameters. This is more similar to how it worked previously where OracleExecutionContext would be late-applying these escaped names. This re-establishes context.compiled_params as deterministically named regardless of dialect in use and moves out the complexity of the quoted param names to be only at the cursor.execute stage. Fixed bug, likely a regression from 1.3, where usage of column names that require bound parameter escaping, more concretely when using Oracle with column names that require quoting such as those that start with an underscore, or in less common cases with some PostgreSQL drivers when using column names that contain percent signs, would cause the ORM versioning feature to not work correctly if the versioning column itself had such a name, as the ORM assumes certain bound parameter naming conventions that were being interfered with via the quotes. This issue is related to :ticket:`8053` and essentially revises the approach towards fixing this, revising the original issue :ticket:`5653` that created the initial implementation for generalized bound-parameter name quoting. Fixes: #8056 Change-Id: I57b064e8f0d070e328b65789c30076f6a0ca0fef
* enable pg8000 for 1.29.1 and aboveMike Bayer2022-05-231-1/+9
| | | | | | | | | | | ROLLBACK TO SAVEPOINT is re-enabled in https://github.com/tlocke/pg8000/issues/111. we still have to add savepoint support to our fixture that deletes from tables without checking for them. this is inconvenient but not incorrect. Change-Id: I2f4a0a3e18db93c3e6794ade9b0fee33d2e4b7dc
* Merge "implement dataclass_transforms" into mainmike bayer2022-05-201-1/+24
|\
| * implement dataclass_transformsMike Bayer2022-05-201-1/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement a new means of creating a mapped dataclass where instead of applying the `@dataclass` decorator distinctly, the declarative process itself can create the dataclass. MapperProperty and MappedColumn objects themselves take the place of the dataclasses.Field object when constructing the class. The overall approach is made possible at the typing level using pep-681 dataclass transforms [1]. This new approach should be able to completely supersede the previous "dataclasses" approach of embedding metadata into Field() objects, which remains a mutually exclusive declarative setup style (mixing them introduces new issues that are not worth solving). [1] https://peps.python.org/pep-0681/#transform-descriptor-types-example Fixes: #7642 Change-Id: I6ba88a87c5df38270317b4faf085904d91c8a63c
* | Fix warnings raised by the testing moduleFederico Caselli2022-05-182-4/+8
|/ | | | | | | Adjust the automatic stacklevel counter to ignore sqlalchemy.testing Properly apply warning filters Change-Id: Ib3d2eb6269af5fc72881df4d39194b3b0cbb1353
* adjust log stacklevel for py3.11.0b1; enable greenletMike Bayer2022-05-151-0/+4
| | | | | | | | | | | | | | | | Fixed issue where support for logging "stacklevel" implemented in :ticket:`7612` required adjustment to work with recently released Python 3.11.0b1, also repairs the unit tests which tested this feature. Install greenlet from a py311 compat patch. re: the stacklevel thing, this is going to be very inconvenient if we have to keep hardcoding numbers everywhere for every new python version Change-Id: I0c8f7293e98c0ca5cc544538284bfd1d3020cb1f References: https://github.com/python-greenlet/greenlet/issues/288 Fixes: #8019
* Use tuple instead of raw url in string formatting (#7987)dzcode2022-05-031-3/+5
| | | | | * Fixes: #7902 - Use tuple instead of raw url in string formatting * Fix lint error
* inline mypy config; files ignoring type errors for the momentMike Bayer2022-04-2832-4/+61
| | | | | | | | | | | | | | | | | | | to simplify pyproject.toml change the remaining files that aren't going to be typed on this first pass (unless of course someone wants to type some of these) to include # mypy: ignore-errors. for the moment, only a handful of ORM modules are to have more type checking implemented. It's important that ignore-errors is used and not "# type: ignore", as in the latter case, mypy doesn't even read the existing types in the file, which makes it impossible to type any files that refer to those modules at all. to simplify ongoing typing work use inline mypy config for remaining files that are "done" for now, indicating the level of type checking they currently have. Change-Id: I98669c1a305c2f0adba85d10b5425541f3fe9533
View Lorry Controller Status