summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/sql
Commit message (Collapse)AuthorAgeFilesLines
...
| * Dedicated bitwise operatorsjazzthief2023-02-066-0/+176
| | | | | | | | | | | | | | | | | | | | | | | | | | Added a full suite of new SQL bitwise operators, for performing database-side bitwise expressions on appropriate data values such as integers, bit-strings, and similar. Pull request courtesy Yegor Statkevich. Fixes: #8780 Closes: #9204 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9204 Pull-request-sha: a4541772a6a784f9161ad78ef84d2ea7a62fa8de Change-Id: I4c70e80f9548dcc1b4e3dccd71bd59d51d3ed46e
* | Use correct dialect annotationFederico Caselli2023-02-021-1/+1
|/ | | | | Fixes: #9222 Change-Id: Ife841a5cf5ec896405e84d8570ef63ce6176a5ec
* Merge "Add support for typing.Literal in Mapped" into mainmike bayer2023-02-011-11/+61
|\
| * Add support for typing.Literal in MappedFrederik Aalund2023-01-311-11/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added support for :pep:`586` ``Literal`` to be used in the :paramref:`_orm.registry.type_annotation_map` as well as within :class:`.Mapped` constructs. To use custom types such as these, they must appear explicitly within the :paramref:`_orm.registry.type_annotation_map` to be mapped. Pull request courtesy Frederik Aalund. As part of this change, the support for :class:`.sqltypes.Enum` in the :paramref:`_orm.registry.type_annotation_map` has been expanded to include support for ``Literal[]`` types consisting of string values to be used, in addition to ``enum.Enum`` datatypes. If a ``Literal[]`` datatype is used within ``Mapped[]`` that is not linked in :paramref:`_orm.registry.type_annotation_map` to a specific datatype, a :class:`.sqltypes.Enum` will be used by default. Fixed issue involving the use of :class:`.sqltypes.Enum` within the :paramref:`_orm.registry.type_annotation_map` where the :paramref:`_sqltypes.Enum.native_enum` parameter would not be correctly copied to the mapped column datatype, if it were overridden as stated in the documentation to set this parameter to False. Fixes: #9187 Fixes: #9200 Closes: #9191 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9191 Pull-request-sha: 7d13f705307bf62560fc831f6f049a425d411374 Change-Id: Ife3ba2655f4897f806d6a9cf0041c69fd4f39e9d
* | Merge "Fixed typing of limit, offset and fetch to allow ``None``." into mainFederico Caselli2023-01-313-12/+13
|\ \ | |/ |/|
| * Fixed typing of limit, offset and fetch to allow ``None``.Federico Caselli2023-01-313-12/+13
| | | | | | | | | | Fixes: #9183 Change-Id: I1ac3e3698034826122ea8a0cdc9f8f55a10ed6c1
* | support NewType in type_annotation_mapMike Bayer2023-01-312-8/+11
|/ | | | | | | | | | | | | | | | | Added support for :pep:`484` ``NewType`` to be used in the :paramref:`_orm.registry.type_annotation_map` as well as within :class:`.Mapped` constructs. These types will behave in the same way as custom subclasses of types right now; they must appear explicitly within the :paramref:`_orm.registry.type_annotation_map` to be mapped. Within this change, the lookup between decl_api._resolve_type and TypeEngine._resolve_for_python_type is streamlined to not inspect the given type multiple times, instead passing in from decl_api to TypeEngine the already "flattened" version of a Generic or NewType type. Fixes: #9175 Change-Id: I227cf84b4b88e4567fa2d1d7da0c05b54e00c562
* Merge "don't count / gather INSERT bind names inside of a CTE" into mainmike bayer2023-01-302-43/+105
|\
| * don't count / gather INSERT bind names inside of a CTEMike Bayer2023-01-302-43/+105
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed regression related to the implementation for the new "insertmanyvalues" feature where an internal ``TypeError`` would occur in arrangements where a :func:`_sql.insert` would be referred towards inside of another :func:`_sql.insert` via a CTE; made additional repairs for this use case for positional dialects such as asyncpg when using "insertmanyvalues". at the core here is a change to positional insertmanyvalues where we now get exactly the positions for the "manyvalues" within the larger list, allowing non-"manyvalues" on the left and right sides at the same time, not assuming anything about how RETURNING renders etc., since CTEs are in the mix also. Fixes: #9173 Change-Id: I5ff071fbef0d92a2d6046b9c4e609bb008438afd
* | fix post-production typo for #9174Mike Bayer2023-01-301-1/+1
| | | | | | | | | | | | | | | | merged in cae662a6383d3ae8f3673c70c3118ea3a1a1606e with one typo fix afterwards Fixes: #9174 Change-Id: I5a525da8a95f40c75da627fed49ce828bd498248
* | Revert "fix post-production typo for #9174"Mike Bayer2023-01-301-1/+1
| | | | | | | | | | | | This reverts commit 3b60c3f53eab8ee5896b3fde525bcf31d4233658. some scratch code for isolation levels got pushed :(
* | fix post-production typo for #9174Mike Bayer2023-01-301-1/+1
| | | | | | | | | | | | | | | | merged in cae662a6383d3ae8f3673c70c3118ea3a1a1606e with one typo fix afterwards Fixes: #9174 Change-Id: I5a525da8a95f40c75da627fed49ce828bd498248
* | allow single tables and entities for "of"Mike Bayer2023-01-292-19/+18
|/ | | | | | | | Opened up typing on :meth:`.Select.with_for_update.of` to also accept table and mapped class arguments, as seems to be available for the MySQL dialect. Fixes: #9174 Change-Id: I15659d7084657564bd5a2aa55ef0e4db51b91247
* Merge "Set correct type annotations for ColumnElement.cast" into mainmike bayer2023-01-281-1/+1
|\
| * Set correct type annotations for ColumnElement.castYurii Karabas2023-01-281-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | <!-- Provide a general summary of your proposed changes in the Title field above --> Fixes: #9156 ### Description <!-- Describe your changes in detail --> ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [x] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #9157 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9157 Pull-request-sha: 321dac184ee0d317296a689d7c8e47bf1464bcce Change-Id: I99fe759a21de910f34bae3bb919e82cd08969e81
* | Correct #7664 to include DropSchemaMike Bayer2023-01-281-0/+2
|/ | | | | | | | | | Corrected the fix for :ticket:`7664`, released in version 2.0.0, to also include :class:`.DropSchema` which was inadvertently missed in this fix, allowing stringification without a dialect. The fixes for both constructs is backported to the 1.4 series as of 1.4.47. Fixes: #7664 Change-Id: I509b7500ee496ac1e444ea2096c2a02520167e6d
* add typing to legacy operatorsMike Bayer2023-01-262-23/+201
| | | | | | | | | Added typing to legacy operators such as ``isnot()``, ``notin_()``, etc. which previously were referencing the newer operators but were not themselves typed. Fixes: #9148 Change-Id: I3ad7d75d89ec13c9f45063033ecff69d610c72ca
* fix stringify for CreateSchemaMike Bayer2023-01-241-0/+2
| | | | | | | | Fixed stringify for a the :class:`.CreateSchema` DDL construct, which would fail with an ``AttributeError`` when stringified without a dialect. Fixes: #7664 Change-Id: Ifc1769604bc5219c060f5112f7bdea0f780f1a1c
* generate stubs for func known functionsMike Bayer2023-01-231-0/+149
| | | | | | | | | | Added typing for the built-in generic functions that are available from the :data:`_sql.func` namespace, which accept a particular set of arguments and return a particular type, such as for :class:`_sql.count`, :class:`_sql.current_timestamp`, etc. Fixes: #9129 Change-Id: I1a2e0dcca3048c77e84dc786843a7df05c457dfa
* Merge "typing updates" into mainmike bayer2023-01-203-37/+83
|\
| * typing updatesMike Bayer2023-01-203-37/+83
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The :meth:`_sql.ColumnOperators.in_` and :meth:`_sql.ColumnOperators.not_in_` are typed to include ``Iterable[Any]`` rather than ``Sequence[Any]`` for more flexibility in argument type. The :func:`_sql.or_` and :func:`_sql.and_` from a typing perspective require the first argument to be present, however these functions still accept zero arguments which will emit a deprecation warning at runtime. Typing is also added to support sending the fixed literal ``False`` for :func:`_sql.or_` and ``True`` for :func:`_sql.and_` as the first argument only, however the documentation now indicates sending the :func:`_sql.false` and :func:`_sql.true` constructs in these cases as a more explicit approach. Fixed typing issue where iterating over a :class:`_orm.Query` object was not correctly typed. Fixes: #9122 Fixes: #9123 Fixes: #9125 Change-Id: I500e3e1b826717b3dd49afa1e682c3c8279c9226
* | implement basic typing for lambda elementsMike Bayer2023-01-192-34/+76
|/ | | | | | | | | | | | | | | | | These weren't working at all, so fixed things up and added a test suite. Keeping things very basic with Any returns etc. as having more specific return types starts making it too cumbersome to write end-user code. Corrected the type passed for "lambda statements" so that a plain lambda is accepted by mypy, pyright, others without any errors about argument types. Additionally implemented typing for more of the public API for lambda statements and ensured :class:`.StatementLambdaElement` is part of the :class:`.Executable` hierarchy so it's typed as accepted by :meth:`_engine.Connection.execute`. Fixes: #9120 Change-Id: Ia7fa34e5b6e43fba02c8f94ccc256f3a68a1f445
* apply pep-612 to hybrid_method; accept SQLCoreOperationsMike Bayer2023-01-141-1/+7
| | | | | | | | | | | | Fixes to the annotations within the ``sqlalchemy.ext.hybrid`` extension for more effective typing of user-defined methods. The typing now uses :pep:`612` features, now supported by recent versions of Mypy, to maintain argument signatures for :class:`.hybrid_method`. Return values for hybrid methods are accepted as SQL expressions in contexts such as :meth:`_sql.Select.where` while still supporting SQL methods. Fixes: #9096 Change-Id: Id4e3a38ec50e415220dfc5f022281b11bb262469
* Merge "Type annotations for sqlalchemy.sql.selectable" into mainmike bayer2023-01-137-108/+224
|\
| * Type annotations for sqlalchemy.sql.selectableDzmitar2023-01-137-108/+224
| | | | | | | | | | | | | | | | | | Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Closes: #9028 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9028 Pull-request-sha: e2f8ddeac0b08feaad917285e988acf1e9465a26 Change-Id: I5caad31bfeeed2d224657a55f067ba1d86b8733f
* | Fixes related to improved sql formattingFederico Caselli2023-01-125-48/+118
|/ | | | | | Follow up of I07b72e6620bb64e329d6b641afa27631e91c4f16 Change-Id: I1f61974bf9cdc3da5317e546d4f9b649c2029e4d
* replace @decorated_property decoratorMike Bayer2023-01-122-4/+8
| | | | | | | | | | | | | This decorator is no longer necessary as of Mypy 0.981 [1]. In current mypy versions, we require direct use of `@property` for return types of these methods to be recognized [1] https://github.com/python/mypy/issues/1362 Change-Id: Ibc36083dec854c5f9140a9b621e9bf9d5bb4fb61
* Merge "Improve sql formatting" into mainmike bayer2023-01-125-33/+33
|\
| * Improve sql formattingFederico Caselli2023-01-115-33/+33
| | | | | | | | | | | | change {opensql} to {printsql} in prints, add missing markers Change-Id: I07b72e6620bb64e329d6b641afa27631e91c4f16
* | fix ORM support for column-named bindparam() in crud .values()Mike Bayer2023-01-111-3/+7
|/ | | | | | | | | | | | | | | | | | | Fixed bug / regression where using :func:`.bindparam()` with the same name as a column in the :meth:`.Update.values` method of :class:`.Update`, as well as the :meth:`.Insert.values` method of :class:`.Insert` in 2.0 only, would in some cases silently fail to honor the SQL expression in which the parameter were presented, replacing the expression with a new parameter of the same name and discarding any other elements of the SQL expression, such as SQL functions, etc. The specific case would be statements that were constructed against ORM entities rather than plain :class:`.Table` instances, but would occur if the statement were invoked with a :class:`.Session` or a :class:`.Connection`. :class:`.Update` part of the issue was present in both 2.0 and 1.4 and is backported to 1.4. Fixes: #9075 Change-Id: Ie954bc1f492ec6a566163588182ef4910c7ee452
* Merge "accept TableClause through mapped selectable chain" into mainmike bayer2023-01-091-2/+5
|\
| * accept TableClause through mapped selectable chainMike Bayer2023-01-091-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | type annotation somehow decided that TableClause doesn't have primary key fields which is not the case at all. In particular the "views" recipe relies on TableClause so adding a restriction like this does not make any sense. It seems the issue was to open this up for typing, by allowing TableClause out as far as ddl.sort_tables() typing is passing for now. Support it out in get_bind() etc. Fixes: #9071 Change-Id: If0e22e0e7df7bee0ff4b295b0ffacfbc6b7a0142
* | warn and skip for FKs that refer to invisible cols for OracleMike Bayer2023-01-091-12/+12
|/ | | | | | | | | | | | | | | | | | | | | Supported use case for foreign key constraints where the local column is marked as "invisible". The errors normally generated when a :class:`.ForeignKeyConstraint` is created that check for the target column are disabled when reflecting, and the constraint is skipped with a warning in the same way which already occurs for an :class:`.Index` with a similar issue. tests are added for indexes, unique constraints, and primary key constraints, which were already working; indexes and uniques warn, primary keys don't which we would assume is because we never see those PK columns in the first place. Constraints now raise an informative ConstraintColumnNotFoundError in the general case for strings in the "pending colargs" collection not being resolvable. Fixes: #9059 Change-Id: I400cf0bff6abba0e0c75f38b07617be1a8ec3453
* tutorial updates re: Core /ORM commonalityMike Bayer2023-01-031-3/+2
| | | | | | | | | | | updates for Insert / bulk insert, executemanyvalues, as well as beginning to describe Table / declared class more closely together, mentioning typing support. Fixed a long-standing issue where sphinx would complain about the Insert symbol being ambiguous. Change-Id: Id4cc09b9581e8fa39c9c00bc8f229636e626e9bc
* happy new year 2023Mike Bayer2023-01-0329-29/+29
| | | | Change-Id: I625af65b3fb1815b1af17dc2ef47dd697fdc3fb1
* link to with_variant() narrative from API docMike Bayer2023-01-031-0/+5
| | | | Change-Id: I5d8e4d7cb7871bedebe0fe89758be441e64b94c6
* ensure whereclause, returning copied as tuplesMike Bayer2022-12-281-7/+7
| | | | | | | | | | Fixed issue in the internal SQL traversal for DML statements like :class:`_dml.Update` and :class:`_dml.Delete` which would cause among other potential issues, a specific issue using lambda statements with the ORM update/delete feature. Fixes: #9033 Change-Id: I76428049cb767ba302fbea89555114bf63ab8687
* rename 2.0.0b5 to 2.0.0rc1Mike Bayer2022-12-271-3/+3
| | | | | | it's hoped for 2.0.0 final to be next, in early January Change-Id: If4285f0929f4a2895f2bc93d9e8336599b973bcf
* pass more contextual information to PyWrapper param createMike Bayer2022-12-272-7/+17
| | | | | | | | | | Fixed issue in lambda SQL feature where the calculated type of a literal value would not take into account the type coercion rules of the "compared to type", leading to a lack of typing information for SQL expressions, such as comparisons to :class:`.JSON` elements and similar. Fixes: #9029 Change-Id: I381c8d7458d98ba762313dee9ec47a9c1881f74a
* add joins_implicitly to column_valued()Mike Bayer2022-12-201-2/+12
| | | | | | | | | | | | | Added parameter :paramref:`.FunctionElement.column_valued.joins_implicitly`, which is useful in preventing the "cartesian product" warning when making use of table-valued or column-valued functions. This parameter was already introduced for :meth:`.FunctionElement.table_valued` in :ticket:`7845`, however it failed to be added for :meth:`.FunctionElement.column_valued` as well. Fixes: #9009 Change-Id: Ifb72fbcb4f4d2998e730d6f85ec7280df3bf3d47
* Merge "ensure all visit methods accept **kw" into mainmike bayer2022-12-171-7/+7
|\
| * ensure all visit methods accept **kwMike Bayer2022-12-161-7/+7
| | | | | | | | | | | | | | | | | | | | Added test support to ensure that all compiler ``visit_xyz()`` methods across all :class:`.Compiler` implementations in SQLAlchemy accept a ``**kw`` parameter, so that all compilers accept additional keyword arguments under all circumstances. Fixes: #8988 Change-Id: I1cefc313e4e64a10ee7dd14400137fbe02ce9523
* | make bind escape lookup extensibleMike Bayer2022-12-161-6/+69
|/ | | | | | | | | | | | | | To accommodate for third party dialects with different character escaping needs regarding bound parameters, the system by which SQLAlchemy "escapes" (i.e., replaces with another character in its place) special characters in bound parameter names has been made extensible for third party dialects, using the :attr:`.SQLCompiler.bindname_escape_chars` dictionary which can be overridden at the class declaration level on any :class:`.SQLCompiler` subclass. As part of this change, also added the dot ``"."`` as a default "escaped" character. Fixes: #8994 Change-Id: I52fbbfa8c64497b123f57327113df3f022bd1419
* implement literal_binds with expanding + bind_expressionMike Bayer2022-12-151-8/+56
| | | | | | | | | | Fixed bug where SQL compilation would fail to make use of :meth:`_types.TypeEngine.bind_expression` on a given type when used in the context of an "expanding" (i.e. "IN") parameter with the ``literal_binds`` compiler parameter in use. Fixes: #8989 Change-Id: Ic9fd27b46381b488117295ea5a492d8fc158e39f
* fix construct_params() for render_postcompile; add new APIMike Bayer2022-12-081-36/+141
| | | | | | | | | | | | | | | | | | | | | The :meth:`.SQLCompiler.construct_params` method, as well as the :attr:`.SQLCompiler.params` accessor, will now return the exact parameters that correspond to a compiled statement that used the ``render_postcompile`` parameter to compile. Previously, the method returned a parameter structure that by itself didn't correspond to either the original parameters or the expanded ones. Passing a new dictionary of parameters to :meth:`.SQLCompiler.construct_params` for a :class:`.SQLCompiler` that was constructed with ``render_postcompile`` is now disallowed; instead, to make a new SQL string and parameter set for an alternate set of parameters, a new method :meth:`.SQLCompiler.construct_expanded_state` is added which will produce a new expanded form for the given parameter set, using the :class:`.ExpandedState` container which includes a new SQL statement and new parameter dictionary, as well as a positional parameter tuple. Fixes: #6114 Change-Id: I9874905bb90f86799b82b244d57369558b18fd93
* disable polymorphic adaption in most casesMike Bayer2022-12-072-24/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Improved a fix first made in version 1.4 for :ticket:`8456` which scaled back the usage of internal "polymorphic adapters", that are used to render ORM queries when the :paramref:`_orm.Mapper.with_polymorphic` parameter is used. These adapters, which are very complex and error prone, are now used only in those cases where an explicit user-supplied subquery is used for :paramref:`_orm.Mapper.with_polymorphic`, which includes only the use case of concrete inheritance mappings that use the :func:`_orm.polymorphic_union` helper, as well as the legacy use case of using an aliased subquery for joined inheritance mappings, which is not needed in modern use. For the most common case of joined inheritance mappings that use the built-in polymorphic loading scheme, which includes those which make use of the :paramref:`_orm.Mapper.polymorphic_load` parameter set to ``inline``, polymorphic adapters are now no longer used. This has both a positive performance impact on the construction of queries as well as a substantial simplification of the internal query rendering process. The specific issue targeted was to allow a :func:`_orm.column_property` to refer to joined-inheritance classes within a scalar subquery, which now works as intuitively as is feasible. ORM context, mapper, strategies now use ORMAdapter in all cases instead of straight ColumnAdapter; added some more parameters to ORMAdapter to make this possible. ORMAdapter now includes a "trace" enumeration that identifies the use path for the adapter and can aid in debugging. implement __slots__ for the ExternalTraversal hierarchy up to ORMAdapter. Within this change, we have to change the ClauseAdapter.wrap() method, which is only used in one polymorphic codepath, to use copy.copy() instead of `__dict__` access (apparently `__reduce_ex__` is implemented for objects with `__slots__`), and we also remove pickling ability, which should not be needed for adapters (this might have been needed for 1.3 and earlier in order for Query to be picklable, but none of that state is present within Query / select() / etc. anymore). Fixes: #8168 Change-Id: I3f6593eb02ab5e5964807c53a9fa4894c826d017
* Add tests for issue #8168; slight internal adjustmentsMike Bayer2022-12-051-1/+0
| | | | | | | | | | | | | The issue in #8168 was improved, but not completely fixed, by #8456. This includes some small changes to ORM context that are a prerequisite for getting ORM adaptation to be better. Have these in 2.0.0b4 so that we have at least a better starting point. References: #8168 Change-Id: I51dbe333b156048836d074fbba1d850f9eb67fd2
* Rewrite positional handling, test for "numeric"Federico Caselli2022-12-052-168/+306
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changed how the positional compilation is performed. It's rendered by the compiler the same as the pyformat compilation. The string is then processed to replace the placeholders with the correct ones, and to obtain the correct order of the parameters. This vastly simplifies the computation of the order of the parameters, that in case of nested CTE is very hard to compute correctly. Reworked how numeric paramstyle behavers: - added support for repeated parameter, without duplicating them like in normal positional dialects - implement insertmany support. This requires that the dialect supports out of order placehoders, since all parameters that are not part of the VALUES clauses are placed at the beginning of the parameter tuple - support for different identifiers for a numeric parameter. It's for example possible to use postgresql style placeholder $1, $2, etc Added two new dialect based on sqlite to test "numeric" fully using both :1 style and $1 style. Includes a workaround for SQLite's not-really-correct numeric implementation. Changed parmstyle of asyncpg dialect to use numeric, rendering with its native $ identifiers Fixes: #8926 Fixes: #8849 Change-Id: I7c640467d49adfe6d795cc84296fc7403dcad4d6
* Merge "disallow same-named columns, unchecked replacement in Table" into mainmike bayer2022-12-042-22/+68
|\
| * disallow same-named columns, unchecked replacement in TableMike Bayer2022-12-042-22/+68
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed issue where table reflection using :paramref:`.Table.extend_existing` would fail to deduplicate a same-named column if the existing :class:`.Table` used a separate key. The :paramref:`.Table.autoload_replace` parameter would allow the column to be skipped but under no circumstances should a :class:`.Table` ever have the same-named column twice. Additionally, changed deprecation warnings to exceptions as were implemented in I1d58c8ebe081079cb669e7ead60886ffc1b1a7f5 . Fixes: #8925 Change-Id: I83d0f8658177a7ffbb06e01dbca91377d1a98d49
View Lorry Controller Status