summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/testing
Commit message (Collapse)AuthorAgeFilesLines
* ORM bulk insert via executeMike Bayer2022-09-243-13/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * ORM Insert now includes "bulk" mode that will run essentially the same process as session.bulk_insert_mappings; interprets the given list of values as ORM attributes for key names * ORM UPDATE has a similar feature, without RETURNING support, for session.bulk_update_mappings * Added support for upserts to do RETURNING ORM objects as well * ORM UPDATE/DELETE with list of parameters + WHERE criteria is a not implemented; use connection * ORM UPDATE/DELETE defaults to "auto" synchronize_session; use fetch if RETURNING is present, evaluate if not, as "fetch" is much more efficient (no expired object SELECT problem) and less error prone if RETURNING is available UPDATE: howver this is inefficient! please continue to use evaluate for simple cases, auto can move to fetch if criteria not evaluable * "Evaluate" criteria will now not preemptively unexpire and SELECT attributes that were individually expired. Instead, if evaluation of the criteria indicates that the necessary attrs were expired, we expire the object completely (delete) or expire the SET attrs unconditionally (update). This keeps the object in the same unloaded state where it will refresh those attrs on the next pass, for this generally unusual case. (originally #5664) * Core change! update/delete rowcount comes from len(rows) if RETURNING was used. SQLite at least otherwise did not support this. adjusted test_rowcount accordingly * ORM DELETE with a list of parameters at all is also a not implemented as this would imply "bulk", and there is no bulk_delete_mappings (could be, but we dont have that) * ORM insert().values() with single or multi-values translates key names based on ORM attribute names * ORM returning() implemented for insert, update, delete; explcit returning clauses now interpret rows in an ORM context, with support for qualifying loader options as well * session.bulk_insert_mappings() assigns polymorphic identity if not set. * explicit RETURNING + synchronize_session='fetch' is now supported with UPDATE and DELETE. * expanded return_defaults() to work with DELETE also. * added support for composite attributes to be present in the dictionaries used by bulk_insert_mappings and bulk_update_mappings, which is also the new ORM bulk insert/update feature, that will expand the composite values into their individual mapped attributes the way they'd be on a mapped instance. * bulk UPDATE supports "synchronize_session=evaluate", is the default. this does not apply to session.bulk_update_mappings, just the new version * both bulk UPDATE and bulk INSERT, the latter with or without RETURNING, support *heterogenous* parameter sets. session.bulk_insert/update_mappings did this, so this feature is maintained. now cursor result can be both horizontally and vertically spliced :) This is now a long story with a lot of options, which in itself is a problem to be able to document all of this in some way that makes sense. raising exceptions for use cases we haven't supported is pretty important here too, the tradition of letting unsupported things just not work is likely not a good idea at this point, though there are still many cases that aren't easily avoidable Fixes: #8360 Fixes: #7864 Fixes: #7865 Change-Id: Idf28379f8705e403a3c6a937f6a798a042ef2540
* implement batched INSERT..VALUES () () for executemanyMike Bayer2022-09-245-0/+220
| | | | | | | | | | | | | | | | | | | | the feature is enabled for all built in backends when RETURNING is used, except for Oracle that doesn't need it, and on psycopg2 and mssql+pyodbc it is used for all INSERT statements, not just those that use RETURNING. third party dialects would need to opt in to the new feature by setting use_insertmanyvalues to True. Also adds dialect-level guards against using returning with executemany where we dont have an implementation to suit it. execute single w/ returning still defers to the server without us checking. Fixes: #6047 Fixes: #7907 Change-Id: I3936d3c00003f02e322f2e43fb949d0e6e568304
* Tighten password security by removing `URL.__str__`Yassen Damyanov2022-09-232-3/+6
| | | | | | | | | | | | | | | For improved security, the :class:`_url.URL` object will now use password obfuscation by default when ``str(url)`` is called. To stringify a URL with cleartext password, the :meth:`_url.URL.render_as_string` may be used, passing the :paramref:`_url.URL.render_as_string.hide_password` parameter as ``False``. Thanks to our contributors for this pull request. Fixes: #8567 Closes: #8563 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8563 Pull-request-sha: d1f1127f753849eb70b8d6cc64badf34e1b9219b Change-Id: If756c8073ff99ac83876d9833c8fe1d7c76211f9
* Improve array_agg and Array processingFederico Caselli2022-09-171-0/+11
| | | | | | | | | The :class:`_functions.array_agg` will now set the array dimensions to 1. Improved :class:`_types.ARRAY` processing to accept ``None`` values as value of a multi-array. Fixes: #7083 Change-Id: Iafec4f77fde9719ccc7c8535bf6235dbfbc62102
* Use ``;`` instead of ``select 1`` to ping PostgreSQLFederico Caselli2022-09-151-0/+11
| | | | | Fixes: #8491 Change-Id: I941d2a3cf92e5609e2045a53cec94522340951db
* Use cibuildwheel to create wheelsTrevor Gross2022-08-292-4/+13
| | | | | | | | | | | | | | | | Using cibuildwheel the following wheels are created: - windows x64 and x84 - macos x64 and arm - linux x64 and arm on manylinux and mosulinux (for alpine) - create a pure python wheel (for pypy and other archs) Fixes: #6702 Fixes: #7607 Closes: #7992 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7992 Pull-request-sha: 61d5e24e5b76c97db73aa2507af7f5c2d3a948fa Change-Id: If0c0b353766e0b61d421789d619eb2b940c08ad0
* include mssql_clustered dialect_options when reflecting - issue #8288John Lennox2022-08-231-4/+9
| | | | | | | | | | | | Implemented reflection of the "clustered index" flag ``mssql_clustered`` for the SQL Server dialect. Pull request courtesy John Lennox. Fixes: #8288 Closes: #8289 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8289 Pull-request-sha: 1bb57352e3e31d8fb7de69ab5e60e5464949f640 Change-Id: Ife367066328f9e47ad823e4098647964a18e21e8
* Update to flake8 5.Federico Caselli2022-07-311-3/+2
| | | | Change-Id: I5a241a70efba68bcea9819ddce6aebc25703e68d
* Reflect expression-based indexes on PostgreSQLFederico Caselli2022-07-284-13/+83
| | | | | | | | | | | The PostgreSQL dialect now supports reflection of expression based indexes. The reflection is supported both when using :meth:`_engine.Inspector.get_indexes` and when reflecting a :class:`_schema.Table` using :paramref:`_schema.Table.autoload_with`. Thanks to immerrr and Aidan Kane for the help on this ticket. Fixes: #7442 Change-Id: I3e36d557235286c0f7f6d8276272ff9225058d48
* update ORM declarative docs for new featuresMike Bayer2022-07-161-0/+6
| | | | | | | I screwed up a rebase or something so this was temporarily in Ic51a12de3358f3a451bd7cf3542b375569499fc1 Change-Id: I847ee1336381221c0112b67854df022edf596b25
* repair yield_per for non-SS dialects and add new optionsMike Bayer2022-07-011-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implemented new :paramref:`_engine.Connection.execution_options.yield_per` execution option for :class:`_engine.Connection` in Core, to mirror that of the same :ref:`yield_per <orm_queryguide_yield_per>` option available in the ORM. The option sets both the :paramref:`_engine.Connection.execution_options.stream_results` option at the same time as invoking :meth:`_engine.Result.yield_per`, to provide the most common streaming result configuration which also mirrors that of the ORM use case in its usage pattern. Fixed bug in :class:`_engine.Result` where the usage of a buffered result strategy would not be used if the dialect in use did not support an explicit "server side cursor" setting, when using :paramref:`_engine.Connection.execution_options.stream_results`. This is in error as DBAPIs such as that of SQLite and Oracle already use a non-buffered result fetching scheme, which still benefits from usage of partial result fetching. The "buffered" strategy is now used in all cases where :paramref:`_engine.Connection.execution_options.stream_results` is set. Added :meth:`.FilterResult.yield_per` so that result implementations such as :class:`.MappingResult`, :class:`.ScalarResult` and :class:`.AsyncResult` have access to this method. Fixes: #8199 Change-Id: I6dde3cbe483a1bf81e945561b60f4b7d1c434750
* Comments on (named) constraintscheremnov2022-06-292-44/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Adds support for comments on named constraints, including `ForeignKeyConstraint`, `PrimaryKeyConstraint`, `CheckConstraint`, `UniqueConstraint`, solving the [Issue 5667](https://github.com/sqlalchemy/sqlalchemy/issues/5667). Supports only PostgreSQL backend. ### Description Following the example of [Issue 1546](https://github.com/sqlalchemy/sqlalchemy/issues/1546), supports comments on constraints. Specifically, enables comments on _named_ ones — as I get it, PostgreSQL prohibits comments on unnamed constraints. Enables setting the comments for named constraints like this: ``` Table( 'example', metadata, Column('id', Integer), Column('data', sa.String(30)), PrimaryKeyConstraint( "id", name="id_pk", comment="id_pk comment" ), CheckConstraint('id < 100', name="cc1", comment="Id value can't exceed 100"), UniqueConstraint(['data'], name="uc1", comment="Must have unique data field"), ) ``` Provides the DDL representation for constraint comments and routines to create and drop them. Class `.Inspector` reflects constraint comments via methods like `get_check_constraints` . ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - [ ] A short code fix - [x] A new feature implementation - Solves the issue 5667. - The commit message includes `Fixes: 5667`. - Includes tests based on comment reflection. **Have a nice day!** Fixes: #5667 Closes: #7742 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7742 Pull-request-sha: 42a5d3c3e9ccf9a9d5397fd007aeab0854f66130 Change-Id: Ia60f578595afdbd6089541c9a00e37997ef78ad3
* merge column args from Annotated left sideMike Bayer2022-06-281-3/+19
| | | | | | | | | | | | because we are forced by pep-681 to use the argument "default", we need a way to have client Column default separate from a dataclasses level default. Also, pep-681 does not support deriving the descriptor function from Annotated, so allow a brief right side mapped_column() to be present that will have more column-centric arguments from the left side Annotated to be merged. Change-Id: I039be1628d498486ba013b2798e1392ed1cd7f9f
* Add ability to test using thick mode with oracledbFederico Caselli2022-06-202-13/+31
| | | | | Change-Id: Iee14750ba20422931bde4d61eaa570af482c7d8b References: #8147
* Merge "create new approach for deeply nested post loader options" into mainmike bayer2022-06-181-1/+1
|\
| * create new approach for deeply nested post loader optionsMike Bayer2022-06-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added very experimental feature to the :func:`_orm.selectinload` and :func:`_orm.immediateload` loader options called :paramref:`_orm.selectinload.recursion_depth` / :paramref:`_orm.immediateload.recursion_depth` , which allows a single loader option to automatically recurse into self-referential relationships. Is set to an integer indicating depth, and may also be set to -1 to indicate to continue loading until no more levels deep are found. Major internal changes to :func:`_orm.selectinload` and :func:`_orm.immediateload` allow this feature to work while continuing to make correct use of the compilation cache, as well as not using arbitrary recursion, so any level of depth is supported (though would emit that many queries). This may be useful for self-referential structures that must be loaded fully eagerly, such as when using asyncio. A warning is also emitted when loader options are connected together with arbitrary lengths (that is, without using the new ``recursion_depth`` option) when excessive recursion depth is detected in related object loading. This operation continues to use huge amounts of memory and performs extremely poorly; the cache is disabled when this condition is detected to protect the cache from being flooded with arbitrary statements. Fixes: #8126 Change-Id: I9f162e0a09c1ed327dd19498aac193f649333a01
* | Merge "rearchitect reflection for batched performance" into mainFederico Caselli2022-06-188-301/+1492
|\ \ | |/ |/|
| * rearchitect reflection for batched performanceFederico Caselli2022-06-188-301/+1492
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rearchitected the schema reflection API to allow some dialects to make use of high performing batch queries to reflect the schemas of many tables at once using much fewer queries. The new performance features are targeted first at the PostgreSQL and Oracle backends, and may be applied to any dialect that makes use of SELECT queries against system catalog tables to reflect tables (currently this omits the MySQL and SQLite dialects which instead make use of parsing the "CREATE TABLE" statement, however these dialects do not have a pre-existing performance issue with reflection. MS SQL Server is still a TODO). The new API is backwards compatible with the previous system, and should require no changes to third party dialects to retain compatibility; third party dialects can also opt into the new system by implementing batched queries for schema reflection. Along with this change is an updated reflection API that is fully :pep:`484` typed, features many new methods and some changes. Fixes: #4379 Change-Id: I897ec09843543aa7012bcdce758792ed3d415d08
* | Revert "add auto_recurse option to selectinload, immediateload"Mike Bayer2022-06-161-1/+1
| | | | | | | | | | | | | | | | | | | | this option works very badly with caching and the API is likely not what we want either. Work continues for #8126 including that the additional work in I9f162e0a09c1ed327dd19498aac193f649333a01 tries to add new recursive features. This reverts commit b3a1162553879d1369154e920f3f4129020bb88e.
* | implement literal stringification for arraysMike Bayer2022-06-151-0/+57
|/ | | | | | | | | | | | | | | as we already implement stringification for the contents, provide a bracketed syntax for default and ARRAY literal for PG specifically. ARRAY literal seems much simpler to render than their quoted syntax which requires double quotes for strings. also open up testing for pg8000 which has likely been fine with arrays for awhile now, bump the version pin also. Fixes: #8138 Change-Id: Id85b052b0a9564d6aa1489160e58b7359f130fdd
* add auto_recurse option to selectinload, immediateloadMike Bayer2022-06-111-1/+1
| | | | | | | | | | | | | | Added very experimental feature to the :func:`_orm.selectinload` and :func:`_orm.immediateload` loader options called :paramref:`_orm.selectinload.auto_recurse` / :paramref:`_orm.immediateload.auto_recurse` , which when set to True will cause a self-referential relationship load to continue loading with arbitrary depth until no further objects are found. This may be useful for self-referential structures that must be loaded fully eagerly, such as when using asyncio. Fixes: #8126 Change-Id: I5bbd00bd0ca43f4649b44680fea1e84680f0a5db
* resolve large ints to BigIntegerMike Bayer2022-06-101-2/+31
| | | | | | | | | | | | The in-place type detection for Python integers, as occurs with an expression such as ``literal(25)``, will now apply value-based adaption as well to accommodate Python large integers, where the datatype determined will be :class:`.BigInteger` rather than :class:`.Integer`. This accommodates for dialects such as that of asyncpg which both sends implicit typing information to the driver as well as is sensitive to numeric scale. Fixes: #7909 Change-Id: I1cd3ec2676c9bb03ffedb600695252bd0037ba02
* handle non-mapped mixins for with_loader_criteria reduceMike Bayer2022-06-091-0/+10
| | | | | | | | | special handling is needed for a with_loader_criteria() against a non-mapped mixin class. added that to test coverage Fixes: #8109 Change-Id: Ia599361c8faab008e92095eb4607d02820f590d5
* run test_update_rowcount_return_defaults only w/ returningMike Bayer2022-06-061-0/+1
| | | | | | | | | | | | | | | sane_rowcount_w_returning asserts failure, which will only occur here if the DBAPI actually uses RETURNING. as SQLite conditionally supports RETURNING which breaks rowcount support only if present, limit this test to that case. Additionally, newer pysqlites will likely fix the issue so we will probably want to put a sqlite3_version check as well once that fix is released. Change-Id: I065aa181eb48363c1024550ae3622486ae0b4a6e
* Generalize RETURNING and suppor for MariaDB / SQLiteDaniel Black2022-06-024-32/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As almost every dialect supports RETURNING now, RETURNING is also made more of a default assumption. * the default compiler generates a RETURNING clause now when specified; CompileError is no longer raised. * The dialect-level implicit_returning parameter now has no effect. It's not fully clear if there are real world cases relying on the dialect-level parameter, so we will see once 2.0 is released. ORM-level RETURNING can be disabled at the table level, and perhaps "implicit returning" should become an ORM-level option at some point as that's where it applies. * Altered ORM update() / delete() to respect table-level implicit returning for fetch. * Since MariaDB doesnt support UPDATE returning, "full_returning" is now split into insert_returning, update_returning, delete_returning * Crazy new thing. Dialects that have *both* cursor.lastrowid *and* returning. so now we can pick between them for SQLite and mariadb. so, we are trying to keep it on .lastrowid for simple inserts with an autoincrement column, this helps with some edge case test scenarios and i bet .lastrowid is faster anyway. any return_defaults() / multiparams etc then we use returning * SQLite decided they dont want to return rows that match in ON CONFLICT. this is flat out wrong, but for now we need to work with it. Fixes: #6195 Fixes: #7011 Closes: #7047 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7047 Pull-request-sha: d25d5ea3abe094f282c53c7dd87f5f53a9e85248 Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Change-Id: I9908ce0ff7bdc50bd5b27722081767c31c19a950
* add backend agnostic UUID datatypeMike Bayer2022-06-013-16/+166
| | | | | | | | | | | | | | | | | | | | Added new backend-agnostic :class:`_types.Uuid` datatype generalized from the PostgreSQL dialects to now be a core type, as well as migrated :class:`_types.UUID` from the PostgreSQL dialect. Thanks to Trevor Gross for the help on this. also includes: * corrects some missing behaviors in the suite literal fixtures test where row round trips weren't being correctly asserted. * fixes some of the ISO literal date rendering added in 952383f9ee0 for #5052 to truncate datetime strings for date/time datatypes in the same way that drivers typically do for bound parameters; this was not working fully and wasn't caught by the broken test fixture Fixes: #7212 Change-Id: I981ac6d34d278c18281c144430a528764c241b04
* move bindparam quote application from compiler to defaultMike Bayer2022-05-291-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | in 296c84313ab29bf9599634f3 for #5653 we generalized Oracle's parameter escaping feature into the compiler, so that it could also work for PostgreSQL. The compiler used quoted names within parameter dictionaries, which then led to the complexity that all functions which interpreted keys from the compiled_params dict had to also quote the param names to use the dictionary. This extra complexity was not added to the ORM peristence.py however, which led to the versioning id feature being broken as well as other areas where persistence.py relies on naming schemes present in context.compiled_params. It also was not added to the "processors" lookup which led to #8053, that added this escaping to that part of the compiler. To both solve the whole problem as well as simplify the compiler quite a bit, move the actual application of the escaped names to be as late as possible, when default.py builds the final list of parameters. This is more similar to how it worked previously where OracleExecutionContext would be late-applying these escaped names. This re-establishes context.compiled_params as deterministically named regardless of dialect in use and moves out the complexity of the quoted param names to be only at the cursor.execute stage. Fixed bug, likely a regression from 1.3, where usage of column names that require bound parameter escaping, more concretely when using Oracle with column names that require quoting such as those that start with an underscore, or in less common cases with some PostgreSQL drivers when using column names that contain percent signs, would cause the ORM versioning feature to not work correctly if the versioning column itself had such a name, as the ORM assumes certain bound parameter naming conventions that were being interfered with via the quotes. This issue is related to :ticket:`8053` and essentially revises the approach towards fixing this, revising the original issue :ticket:`5653` that created the initial implementation for generalized bound-parameter name quoting. Fixes: #8056 Change-Id: I57b064e8f0d070e328b65789c30076f6a0ca0fef
* enable pg8000 for 1.29.1 and aboveMike Bayer2022-05-231-1/+9
| | | | | | | | | | | ROLLBACK TO SAVEPOINT is re-enabled in https://github.com/tlocke/pg8000/issues/111. we still have to add savepoint support to our fixture that deletes from tables without checking for them. this is inconvenient but not incorrect. Change-Id: I2f4a0a3e18db93c3e6794ade9b0fee33d2e4b7dc
* Merge "implement dataclass_transforms" into mainmike bayer2022-05-201-1/+24
|\
| * implement dataclass_transformsMike Bayer2022-05-201-1/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement a new means of creating a mapped dataclass where instead of applying the `@dataclass` decorator distinctly, the declarative process itself can create the dataclass. MapperProperty and MappedColumn objects themselves take the place of the dataclasses.Field object when constructing the class. The overall approach is made possible at the typing level using pep-681 dataclass transforms [1]. This new approach should be able to completely supersede the previous "dataclasses" approach of embedding metadata into Field() objects, which remains a mutually exclusive declarative setup style (mixing them introduces new issues that are not worth solving). [1] https://peps.python.org/pep-0681/#transform-descriptor-types-example Fixes: #7642 Change-Id: I6ba88a87c5df38270317b4faf085904d91c8a63c
* | Fix warnings raised by the testing moduleFederico Caselli2022-05-182-4/+8
|/ | | | | | | Adjust the automatic stacklevel counter to ignore sqlalchemy.testing Properly apply warning filters Change-Id: Ib3d2eb6269af5fc72881df4d39194b3b0cbb1353
* adjust log stacklevel for py3.11.0b1; enable greenletMike Bayer2022-05-151-0/+4
| | | | | | | | | | | | | | | | Fixed issue where support for logging "stacklevel" implemented in :ticket:`7612` required adjustment to work with recently released Python 3.11.0b1, also repairs the unit tests which tested this feature. Install greenlet from a py311 compat patch. re: the stacklevel thing, this is going to be very inconvenient if we have to keep hardcoding numbers everywhere for every new python version Change-Id: I0c8f7293e98c0ca5cc544538284bfd1d3020cb1f References: https://github.com/python-greenlet/greenlet/issues/288 Fixes: #8019
* Use tuple instead of raw url in string formatting (#7987)dzcode2022-05-031-3/+5
| | | | | * Fixes: #7902 - Use tuple instead of raw url in string formatting * Fix lint error
* inline mypy config; files ignoring type errors for the momentMike Bayer2022-04-2832-4/+61
| | | | | | | | | | | | | | | | | | | to simplify pyproject.toml change the remaining files that aren't going to be typed on this first pass (unless of course someone wants to type some of these) to include # mypy: ignore-errors. for the moment, only a handful of ORM modules are to have more type checking implemented. It's important that ignore-errors is used and not "# type: ignore", as in the latter case, mypy doesn't even read the existing types in the file, which makes it impossible to type any files that refer to those modules at all. to simplify ongoing typing work use inline mypy config for remaining files that are "done" for now, indicating the level of type checking they currently have. Change-Id: I98669c1a305c2f0adba85d10b5425541f3fe9533
* pep-484: ORM public API, constructorsMike Bayer2022-04-201-16/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | for the moment, abandoning using @overload with relationship() and mapped_column(). The overloads are very difficult to get working at all, and the overloads that were there all wouldn't pass on mypy. various techniques of getting them to "work", meaning having right hand side dictate what's legal on the left, have mixed success and wont give consistent results; additionally, it's legal to have Optional / non-optional independent of nullable in any case for columns. relationship cases are less ambiguous but mypy was not going along with things. we have a comprehensive system of allowing left side annotations to drive the right side, in the absense of explicit settings on the right. so type-centric SQLAlchemy will be left-side driven just like dataclasses, and the various flags and switches on the right side will just not be needed very much. in other matters, one surprise, forgot to remove string support from orm.join(A, B, "somename") or do deprecations for it in 1.4. This is a really not-directly-used structure barely mentioned in the docs for many years, the example shows a relationship being used, not a string, so we will just change it to raise the usual error here. Change-Id: Iefbbb8d34548b538023890ab8b7c9a5d9496ec6e
* Merge "pep484: schema API" into mainmike bayer2022-04-151-2/+2
|\
| * pep484: schema APIMike Bayer2022-04-151-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | implement strict typing for schema.py this module has lots of public API, lots of old decisions and very hard to follow construction sequences in many cases, and is also where we get a lot of new feature requests, so strict typing should help keep things clean. among improvements here, fixed the pool .info getters and also figured out how to get ColumnCollection and related to be covariant so that we may set them up as returning Column or ColumnClause without any conflicts. DDL was affected, noting that superclasses of DDLElement (_DDLCompiles, added recently) can now be passed into "ddl_if" callables; reorganized ddl into ExecutableDDLElement as a new name for DDLElement and _DDLCompiles renamed to BaseDDLElement. setting up strict also located an API use case that is completely broken, which is connection.execute(some_default) returns a scalar value. This case has been deprecated and new paths have been set up so that connection.scalar() may be used. This likely wasn't possible in previous versions because scalar() would assume a CursorResult. The scalar() change also impacts Session as we have explicit support (since someone had reported it as a regression) for session.execute(Sequence()) to work. They will get the same deprecation message (which omits the word "Connection", just uses ".execute()" and ".scalar()") and they can then use Session.scalar() as well. Getting this to type correctly while still supporting ORM use cases required some refactoring, and I also set up a keyword only delimeter for Session.execute() and related as execution_options / bind_arguments should always be keyword only, applied these changes to AsyncSession as well. Additionally simpify Table __init__ now that we are Python 3 only, we can have positional plus explicit kwargs finally. Simplify Column.__init__ as well again taking advantage of kw only arguments. Fill in most/all __init__ methods in sqltypes.py as the constructor for types is most of the API. should likely do this for dialect-specific types as well. Apply _InfoType for all info attributes as should have been done originally and update descriptor decorators. Change-Id: I3f9f8ff3f1c8858471ff4545ac83d68c88107527
* | implement multi-element expression constructsMike Bayer2022-04-131-0/+25
|/ | | | | | | | | | | | | | | | | | | | | | Improved the construction of SQL binary expressions to allow for very long expressions against the same associative operator without special steps needed in order to avoid high memory use and excess recursion depth. A particular binary operation ``A op B`` can now be joined against another element ``op C`` and the resulting structure will be "flattened" so that the representation as well as SQL compilation does not require recursion. To implement this more cleanly, the biggest change here is that column-oriented lists of things are broken away from ClauseList in a new class ExpressionClauseList, that also forms the basis of BooleanClauseList. ClauseList is still used for the generic "comma-separated list" of things such as Tuple and things like ORDER BY, as well as in some API endpoints. Also adds __slots__ to the TypeEngine-bound Comparator classes. Still can't really do __slots__ on ClauseElement. Fixes: #7744 Change-Id: I81a8ceb6f8f3bb0fe52d58f3cb42e4b6c2bc9018
* pep-484: session, instancestate, etcMike Bayer2022-04-121-0/+1
| | | | | | | | Also adds some fixes to annotation-based mapping that have come up, as well as starts to add more pep-484 test cases Change-Id: Ia722bbbc7967a11b23b66c8084eb61df9d233fee
* use code generation for scoped_sessionMike Bayer2022-04-122-13/+22
| | | | | | | | | | | | | | | | | | | our decorator thing generates code in any case, so point it at the file itself to generate real code for the blocks rather than doing things dynamically. this will allow typing tools to have no problem whatsoever and we also reduce import time overhead. file size will be a lot bigger though, shrugs. syntax / dupe method / etc. checking will be accomplished by our existing linting / typing / formatting tools. As we are also using "from __future__ import annotations", we also no longer have to apply quotes to generated annotations. Change-Id: I20962cb65bda63ff0fb67357ab346e9b1ef4f108
* update flake8 noqa skips with proper syntaxFederico Caselli2022-04-111-1/+1
| | | | Change-Id: I42ed77f559e3ee5b8c600d98457ee37803ef0ea6
* add sane_rowcount to SimpleUpdateDeleteTestMike Bayer2022-04-091-0/+1
| | | | | | | | | For third party dialects, repaired a missing requirement for the ``SimpleUpdateDeleteTest`` suite test which was not checking for a working "rowcount" function on the target dialect. Fixes: #7919 Change-Id: I2bc68132131eb36c43b8dabec0fac86272e26df5
* maintain complete cloned_set for BindParameterMike Bayer2022-04-061-0/+7
| | | | | | | | | | | Fixed regression caused by :ticket:`7823` which impacted the caching system, such that bound parameters that had been "cloned" within ORM operations, such as polymorphic loading, would in some cases not acquire their correct execution-time value leading to incorrect bind values being rendered. Fixes: #7903 Change-Id: I61c802749b859bebeb127d24e66d6e77d13ce57a
* pep484 - sql.selectableMike Bayer2022-04-042-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the pep484 task becomes more intense as there is mounting pressure to come up with a consistency in how data moves from end-user to instance variable. current thinking is coming into: 1. there are _typing._XYZArgument objects that represent "what the user sent" 2. there's the roles, which represent a kind of "filter" for different kinds of objects. These are mostly important as the argument we pass to coerce(). 3. there's the thing that coerce() returns, which should be what the construct uses as its internal representation of the thing. This is _typing._XYZElement. but there's some controversy over whether or not we should pass actual ClauseElements around by their role or not. I think we shouldn't at the moment, but this makes the "role-ness" of something a little less portable. Like, we have to set DMLTableRole for TableClause, Join, and Alias, but then also we have to repeat those three types in order to set up _DMLTableElement. Other change introduced here, there was a deannotate=True for the left/right of a sql.join(). All tests pass without that. I'd rather not have that there as if we have a join(A, B) where A, B are mapped classes, we want them inside of the _annotations. The rationale seems to be performance, but this performance can be illustrated to be on the compile side which we hope is cached in the normal case. CTEs now accommodate for text selects including recursive. Get typing to accommodate "util.preloaded" cleanly; add "preloaded" as a real module. This seemed like we would have needed pep562 `__getattr__()` but we don't, just set names in globals() as we import them. References: #6810 Change-Id: I34d17f617de2fe2c086fc556bd55748dc782faf0
* use .fromisoformat() for sqlite datetime, date, time parsingMike Bayer2022-04-031-1/+1
| | | | | | | | | | | | SQLite datetime, date, and time datatypes now use Python standard lib ``fromisoformat()`` methods in order to parse incoming datetime, date, and time string values. This improves performance vs. the previous regular expression-based approach, and also automatically accommodates for datetime and time formats that contain either a six-digit "microseconds" format or a three-digit "milliseconds" format. Fixes: #7029 Change-Id: I67aab4fe5ee3055e5996050cf4564981413cc221
* fix quotes regexp for SQLite CHECK constraintsMike Bayer2022-03-281-2/+4
| | | | | | | | | Fixed bug where the name of CHECK constraints under SQLite would not be reflected if the name were created using quotes, as is the case when the name uses mixed case or special characters. Fixes: #5463 Change-Id: Ic3b1e0a0385fb9e727b0880e90815ea2814df313
* test sqlite w/ savepoint workaround in session fixture testMike Bayer2022-03-071-0/+14
| | | | | Fixes: #7795 Change-Id: Ib790581555656c088f86c00080c70d19ca295a03
* Implement generic Double and related fixed typeszeeeeeb2022-02-252-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added :class:`.Double`, :class:`.DOUBLE`, :class:`.DOUBLE_PRECISION` datatypes to the base ``sqlalchemy.`` module namespace, for explicit use of double/double precision as well as generic "double" datatypes. Use :class:`.Double` for generic support that will resolve to DOUBLE/DOUBLE PRECISION/FLOAT as needed for different backends. Implemented DDL and reflection support for ``FLOAT`` datatypes which include an explicit "binary_precision" value. Using the Oracle-specific :class:`_oracle.FLOAT` datatype, the new parameter :paramref:`_oracle.FLOAT.binary_precision` may be specified which will render Oracle's precision for floating point types directly. This value is interpreted during reflection. Upon reflecting back a ``FLOAT`` datatype, the datatype returned is one of :class:`_types.DOUBLE_PRECISION` for a ``FLOAT`` for a precision of 126 (this is also Oracle's default precision for ``FLOAT``), :class:`_types.REAL` for a precision of 63, and :class:`_oracle.FLOAT` for a custom precision, as per Oracle documentation. As part of this change, the generic :paramref:`_sqltypes.Float.precision` value is explicitly rejected when generating DDL for Oracle, as this precision cannot be accurately converted to "binary precision"; instead, an error message encourages the use of :meth:`_sqltypes.TypeEngine.with_variant` so that Oracle's specific form of precision may be chosen exactly. This is a backwards-incompatible change in behavior, as the previous "precision" value was silently ignored for Oracle. Fixes: #5465 Closes: #7674 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7674 Pull-request-sha: 5c68419e5aee2e27bf21a8ac9eb5950d196c77e5 Change-Id: I831f4af3ee3b23fde02e8f6393c83e23dd7cd34d
* Revert SQLAlchemy warnings to warnings.pyFederico Caselli2022-02-211-5/+10
| | | | | | | | | | | | Configuring the warning filters in pyproject breaks tests if no sqlalchemy is installed in the env, since the filters are processed before loading conftest. It also may interfere with coverage. Revises Ia9715533b01f72aa5fdcf6a27ce75b76f829fa43 aba3ab247da4628e4e7baf993702e2efaccbc547 Change-Id: I51448a6a014f31d3088dce54cd20d1e683500f8c
* pep-484 for sqlalchemy.event; use future annotationsMike Bayer2022-02-1514-0/+28
| | | | | | | | | | __future__.annotations mode allows us to use non-string annotations for argument and return types in most cases, but more importantly it removes a large amount of runtime overhead that would be spent in evaluating the annotations. Change-Id: I2f5b6126fe0019713fc50001be3627b664019ede References: #6810
View Lorry Controller Status