summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/orm
Commit message (Collapse)AuthorAgeFilesLines
* Merge "remove "aliased class pool" caching approach" into mainHEADmainmike bayer2023-05-171-45/+28
|\
| * remove "aliased class pool" caching approachMike Bayer2023-05-131-45/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Modified the ``JoinedLoader`` implementation to use a simpler approach in one particular area where it previously used a cached structure that would be shared among threads. The rationale is to avoid a potential race condition which is suspected of being the cause of a particular crash that's been reported multiple times. The cached structure in question is still ultimately "cached" via the compiled SQL cache, so a performance degradation is not anticipated. The change also modifies the tests for None in context.secondary to ensure no None values are in this list, as this is suspected as being the immediate cause of the issue in #9777. The cached AliasedClass thing is suspected as being the origination of the cause, as under high concurrency many threads might all access that AliasedClass immediately, which seems a reasonable place that the "adapter returning None" symptom might be originating. As of yet we don't have a self-contained reproducer for the issue, some initial attempts with threads are not showing any issue. Fixes: #9777 Change-Id: I967588f280796c413e629b55b8d97d40c1164248
* | limit joinedload exclusion rules to immediate mapped columnsMike Bayer2023-05-141-1/+38
|/ | | | | | | | | | | Fixed issue where using additional relationship criteria with the :func:`_orm.joinedload` loader option, where the additional criteria itself contained correlated subqueries that referred to the joined entities and therefore also required "adaption" to aliased entities, would be excluded from this adaption, producing the wrong ON clause for the joinedload. Fixes: #9779 Change-Id: Idcfec3e760057fbf6a09c10ad67a0bb4bf70f03a
* Merge "return empty MappedColumn() at early pep-593 step" into mainmike bayer2023-05-111-3/+7
|\
| * return empty MappedColumn() at early pep-593 stepMike Bayer2023-05-101-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | Fixed issue in new ORM Annotated Declarative where using a :class:`_schema.ForeignKey` (or other column-level constraint) inside of :func:`_orm.mapped_column` which is then copied out to models via pep-593 ``Annotated`` would apply duplicates of each constraint to the :class:`_schema.Column` as produced in the target :class:`_schema.Table`, leading to incorrect CREATE TABLE DDL as well as migration directives under Alembic. Fixes: #9766 Change-Id: I8a3b2716bf393d1d2b5894f9f72b45fa59df1e08
* | skip ORM loading setups for non-toplevel DMLMike Bayer2023-05-111-26/+40
|/ | | | | | | | | | Fixed regression where use of :func:`_dml.update` or :func:`_dml_delete` within a :class:`_sql.CTE` construct, then used in a :func:`_sql.select`, would raise a :class:`.CompileError` as a result of ORM related rules for performing ORM-level update/delete statements. Fixes: #9767 Change-Id: I4eae9af86752b2e5fd64f7998f8a68754c349e4c
* Merge "add full parameter types for ORM with_for_update" into mainmike bayer2023-05-102-6/+7
|\
| * add full parameter types for ORM with_for_updateMike Bayer2023-05-102-6/+7
| | | | | | | | | | | | | | | | | | | | Fixed typing for the :paramref:`_orm.Session.get.with_for_update` parameter of :meth:`_orm.Session.get` and :meth:`_orm.Session.refresh` (as well as corresponding methods on :class:`_asyncio.AsyncSession`) to accept boolean ``True`` and all other argument forms accepted by the parameter at runtime. Fixes: #9762 Change-Id: Ied4d37a269906b3d9be5ab7d31a2fa863360cced
* | Merge "fix test suite warnings" into mainmike bayer2023-05-102-4/+9
|\ \ | |/ |/|
| * fix test suite warningsMike Bayer2023-05-092-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | fix a handful of warnings that were emitting but not raising, usually because they were inside an "expect_warnings" block. modify "expect_warnings" to always use "raise_on_any_unexpected" behavior; remove this parameter. Fixed issue in semi-private ``await_only()`` and ``await_fallback()`` concurrency functions where the given awaitable would remain un-awaited if the function threw a ``GreenletError``, which could cause "was not awaited" warnings later on if the program continued. In this case, the given awaitable is now cancelled before the exception is thrown. Change-Id: I33668c5e8c670454a3d879e559096fb873b57244
* | allow column named twice warning to take effectMike Bayer2023-05-091-1/+5
| | | | | | | | | | | | | | | | | | | | | | Fixed issue in :func:`_orm.mapped_column` construct where the correct warning for "column X named directly multiple times" would not be emitted when ORM mapped attributes referred to the same :class:`_schema.Column`, if the :func:`_orm.mapped_column` construct were involved, raising an internal assertion instead. Fixes: #9630 Change-Id: I5d9dfaaa225aefb487c9cd981ba3ad78507bb577
* | Fix usage of `Annotated` in `DeclarativeBase` docstring (#9751)Viicos2023-05-081-1/+1
| |
* | add explicit step to set populate_existing for bulk insertMike Bayer2023-05-051-1/+12
| | | | | | | | | | | | | | | | | | | | Fixed issue in new :ref:`orm_queryguide_upsert_returning` feature where the ``populate_existing`` execution option was not being propagated to the loading option, preventing existing attributes from being refreshed in-place. Fixes: #9746 Change-Id: I3efcab644e2b5874c6b265d5313f353c051db629
* | Merge "improve natural_path usage in two places" into mainmike bayer2023-05-043-39/+27
|\ \ | |/ |/|
| * improve natural_path usage in two placesMike Bayer2023-04-293-39/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed loader strategy pathing issues where eager loaders such as :func:`_orm.joinedload` / :func:`_orm.selectinload` would fail to traverse fully for many-levels deep following a load that had a :func:`_orm.with_polymorphic` or similar construct as an interim member. Here we can take advantage of 2.0's refactoring of strategy_options to identify the "chop_path" concept can be simplified to work with "natural" paths alone. In addition, identified existing logic in PropRegistry that works fine, but needed the "is_unnatural" attribute to be more accurate for a given path, so we set that up front to True if the ancestor is_unnatural. Fixes: #9715 Change-Id: Ie6b3f55b6a23d0d32628afd22437094263745114
* | do not allow non-cache-key entity objects in annotationsMike Bayer2023-04-301-6/+20
|/ | | | | | | | | | | Fixed critical caching issue where combination of :func:`_orm.aliased()` :func:`_sql.case` and :func:`_hybrid.hybrid_property` expressions would cause a cache key mismatch, leading to cache keys that held onto the actual :func:`_orm.aliased` object while also not matching each other, filling up the cache. Fixes: #9728 Change-Id: I700645b5629a81a0104cf923db72a7421fa43ff4
* support parameters in all ORM insert modesMike Bayer2023-04-262-23/+85
| | | | | | | | | | | | | | | | | | | | Fixed 2.0 regression where use of :func:`_sql.bindparam()` inside of :meth:`_dml.Insert.values` would fail to be interpreted correctly when executing the :class:`_dml.Insert` statement using the ORM :class:`_orm.Session`, due to the new ORM-enabled insert feature not implementing this use case. In addition, the bulk INSERT and UPDATE features now add these capabilities: * The requirement that extra parameters aren't passed when using ORM INSERT using the "orm" dml_strategy setting is lifted. * The requirement that additional WHERE criteria is not passed when using ORM UPDATE using the "bulk" dml_strategy setting is lifted. Note that in this case, the check for expected row count is turned off. Fixes: #9583 Change-Id: I539c18893b697caeab5a5f0195a27d4f0487e728
* Merge "add deterministic imv returning ordering using sentinel columns" into ↵mike bayer2023-04-218-32/+129
|\ | | | | | | main
| * add deterministic imv returning ordering using sentinel columnsMike Bayer2023-04-218-32/+129
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Repaired a major shortcoming which was identified in the :ref:`engine_insertmanyvalues` performance optimization feature first introduced in the 2.0 series. This was a continuation of the change in 2.0.9 which disabled the SQL Server version of the feature due to a reliance in the ORM on apparent row ordering that is not guaranteed to take place. The fix applies new logic to all "insertmanyvalues" operations, which takes effect when a new parameter :paramref:`_dml.Insert.returning.sort_by_parameter_order` on the :meth:`_dml.Insert.returning` or :meth:`_dml.UpdateBase.return_defaults` methods, that through a combination of alternate SQL forms, direct correspondence of client side parameters, and in some cases downgrading to running row-at-a-time, will apply sorting to each batch of returned rows using correspondence to primary key or other unique values in each row which can be correlated to the input data. Performance impact is expected to be minimal as nearly all common primary key scenarios are suitable for parameter-ordered batching to be achieved for all backends other than SQLite, while "row-at-a-time" mode operates with a bare minimum of Python overhead compared to the very heavyweight approaches used in the 1.x series. For SQLite, there is no difference in performance when "row-at-a-time" mode is used. It's anticipated that with an efficient "row-at-a-time" INSERT with RETURNING batching capability, the "insertmanyvalues" feature can be later be more easily generalized to third party backends that include RETURNING support but not necessarily easy ways to guarantee a correspondence with parameter order. Fixes: #9618 References: #9603 Change-Id: I1d79353f5f19638f752936ba1c35e4dc235a8b7c
* | Merge "try to omit unnecessary cols for ORM bulk insert + returning" into mainmike bayer2023-04-212-9/+27
|\ \ | |/
| * try to omit unnecessary cols for ORM bulk insert + returningMike Bayer2023-04-212-9/+27
| | | | | | | | | | | | | | | | | | Fixed bug in ORM bulk insert feature where additional unnecessary columns would be rendered in the INSERT statement if RETURNING of individual columns were requested. Fixes: #9685 Change-Id: Ibf5f06ab017215c7c9bd8850c3a006f73fe78c68
* | improve return type for QueryableAttribute.and_()Mike Bayer2023-04-181-1/+1
|/ | | | | | | | | Fixed typing issue where :meth:`_orm.PropComparator.and_` expressions would not be correctly typed inside of loader options such as :func:`_orm.selectinload`. Fixes: #9669 Change-Id: I874cb22c004e0a24f2b7f530fda542de2c4c6d3b
* Merge "apply criteria options from top-level core-only statement" into mainmike bayer2023-04-172-35/+69
|\
| * apply criteria options from top-level core-only statementMike Bayer2023-04-172-35/+69
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Made an improvement to the :func:`_orm.with_loader_criteria` loader option to allow it to be indicated in the :meth:`.Executable.options` method of a top-level statement that is not itself an ORM statement. Examples include :func:`_sql.select` that's embedded in compound statements such as :func:`_sql.union`, within an :meth:`_dml.Insert.from_select` construct, as well as within CTE expressions that are not ORM related at the top level. Improved propagation of :func:`_orm.with_loader_criteria` within ORM enabled UPDATE and DELETE statements as well. Fixes: #9635 Change-Id: I088ad91929dc797c06f292f5dc547d48ffb30430
* | dont assume _compile_options are presentMike Bayer2023-04-171-1/+5
|/ | | | | | | | | | | | Fixed bug where various ORM-specific getters such as :attr:`.ORMExecuteState.is_column_load`, :attr:`.ORMExecuteState.is_relationship_load`, :attr:`.ORMExecuteState.loader_strategy_path` etc. would throw an ``AttributeError`` if the SQL statement itself were a "compound select" such as a UNION. Fixes: #9634 Change-Id: Ia37df5d6f89d6534d69237dcab294bd849ece28b
* Merge "Remove old versionadded and versionchanged" into mainmike bayer2023-04-1210-130/+2
|\
| * Remove old versionadded and versionchangedFederico Caselli2023-04-1210-130/+2
| | | | | | | | | | | | | | Removed versionadded and versionchanged for version prior to 1.2 since they are no longer useful. Change-Id: I5c53d1188bc5fec3ab4be39ef761650ed8fa6d3e
* | establish column_property and query_expression as readonly from a dc perspectiveMike Bayer2023-04-124-54/+112
|/ | | | | | | | | | | | | | | | | | | | | | | | | | Fixed bug in ORM Declarative Dataclasses where the :func:`_orm.queryable_attribute` and :func:`_orm.column_property` constructs, which are documented as read-only constructs in the context of a Declarative mapping, could not be used with a :class:`_orm.MappedAsDataclass` class without adding ``init=False``, which in the case of :func:`_orm.queryable_attribute` was not possible as no ``init`` parameter was included. These constructs have been modified from a dataclass perspective to be assumed to be "read only", setting ``init=False`` by default and no longer including them in the pep-681 constructor. The dataclass parameters for :func:`_orm.column_property` ``init``, ``default``, ``default_factory``, ``kw_only`` are now deprecated; these fields don't apply to :func:`_orm.column_property` as used in a Declarative dataclasses configuration where the construct would be read-only. Also added read-specific parameter :paramref:`_orm.queryable_attribute.compare` to :func:`_orm.queryable_attribute`; :paramref:`_orm.queryable_attribute.repr` was already present. Added missing :paramref:`_orm.mapped_column.active_history` parameter to :func:`_orm.mapped_column` construct. Fixes: #9628 Change-Id: I2ab44d6b763b20410bd1ebb5ac949a6d223f1ce2
* include declared_directive as a declared_attrMike Bayer2023-04-101-2/+2
| | | | | | | | | | | Fixed issue where the :meth:`_orm.declared_attr.directive` modifier was not correctly honored for subclasses when applied to the ``__mapper_args__`` special method name, as opposed to direct use of :class:`_orm.declared_attr`. The two constructs should have identical runtime behaviors. Fixes: #9625 Change-Id: I0dfe9e73bb45f70dbebc8e94ce280ad3b52e867f
* Fix typo in the doc for `mapped_column` (#9612)Harshit Doshi2023-04-081-1/+1
| | | | | I found an extremely and probably insignificant typo in the doc for `mapped_column` while actually trying to use and understand it for my project and have fixed it here. The typo was for a "for" which was instead mentioned as "or".
* consider aliased mappers in cycles alsoMike Bayer2023-04-021-1/+1
| | | | | | | | | | | Fixed endless loop which could occur when using "relationship to aliased class" feature and also indicating a recursive eager loader such as ``lazy="selectinload"`` in the loader, in combination with another eager loader on the opposite side. The check for cycles has been fixed to include aliased class relationships. Fixes: #9590 Change-Id: I8d340882f040ff9289c209bedd8fbdfd7186f944
* Wrap dataclass exceptions clarifying originFederico Caselli2023-03-311-0/+8
| | | | | | | | | | | | | Exceptions such as ``TypeError`` and ``ValueError`` raised by Python dataclasses when making use of the :class:`_orm.MappedAsDataclass` mixin class or :meth:`_orm.registry.mapped_as_dataclass` decorator are now wrapped within an :class:`.InvalidRequestError` wrapper along with informative context about the error message, referring to the Python dataclasses documentation as the authoritative source of background information on the cause of the exception. Fixes: #9563 Change-Id: I25652485b91c4ee8cf112b91aae8f9041061a8bd
* Merge "skip anno-only mixin columns that are overridden on subclasses" into mainmike bayer2023-03-311-2/+16
|\
| * skip anno-only mixin columns that are overridden on subclassesMike Bayer2023-03-301-2/+16
| | | | | | | | | | | | | | | | | | | | | | Fixed issue where an annotation-only :class:`_orm.Mapped` directive could not be used in a Declarative mixin class, without that attribute attempting to take effect for single- or joined-inheritance subclasses of mapped classes that had already mapped that attribute on a superclass, producing conflicting column errors and/or warnings. Fixes: #9564 Change-Id: I0f92be2ae98a8c45afce3e06d0a7cc61c19a96f4
* | Merge "copy deferred attributes for mapped_column" into mainmike bayer2023-03-311-0/+2
|\ \
| * | copy deferred attributes for mapped_columnMike Bayer2023-03-301-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | Fixed issue where the :func:`_orm.mapped_column` construct would raise an internal error if used on a Declarative mixin and included the :paramref:`_orm.mapped_column.deferred` parameter. Fixes: #9550 Change-Id: I0cc5ec4910656abc9a3fb7b7b60880256cd7c61e
* | | Merge "warn for DC mixin / abstract fields that are not on a dataclass" into ↵mike bayer2023-03-311-11/+52
|\ \ \ | | | | | | | | | | | | main
| * | | warn for DC mixin / abstract fields that are not on a dataclassMike Bayer2023-03-301-11/+52
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fields that are declared on Declarative Mixins and then combined with classes that make use of :class:`_orm.MappedAsDataclass`, where those mixin fields are not themselves part of a dataclass, now emit a deprecation warning as these fields will be ignored in a future release, as Python dataclasses behavior is to ignore these fields. Type checkers will not see these fields under pep-681. Fixes: #9350 Change-Id: Ie0a0ac0f0bb58d1c2aae13b4a8dcd0439a4f5477
* | | assign correct typing to deferred(), query_property()Mike Bayer2023-03-301-5/+4
|/ / | | | | | | | | | | | | | | Fixed typing for :func:`_orm.deferred` and :func:`_orm.query_expression` to work correctly with 2.0 style mappings. Fixes: #9536 Change-Id: Idb88c0a11b7896093234b3cc31595f64b552cba0
* | Merge "warn for all unmapped expressions" into mainmike bayer2023-03-301-3/+6
|\ \ | |/ |/|
| * warn for all unmapped expressionsMike Bayer2023-03-241-3/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Expanded the warning emitted when a plain :func:`_sql.column` object is present in a Declarative mapping to include any arbitrary SQL expression that is not declared within an appropriate property type such as :func:`_orm.column_property`, :func:`_orm.deferred`, etc. These attributes are otherwise not mapped at all and remain unchanged within the class dictionary. As it seems likely that such an expression is usually not what's intended, this case now warns for all such otherwise ignored expressions, rather than just the :func:`_sql.column` case. Fixes: #9537 Change-Id: Ic4ca7a071a28adf4ea8680690025d927522a0805
* | Merge "disable raise sql for the delete cascade" into mainmike bayer2023-03-291-1/+1
|\ \
| * | disable raise sql for the delete cascadeMike Bayer2023-03-281-1/+1
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Towards maintaining consistency with unit-of-work changes made for :ticket:`5984` and :ticket:`8862`, both of which disable "lazy='raise'" handling within :class:`_orm.Session` processes that aren't triggered by attribute access, the :meth:`_orm.Session.delete` method will now also disable "lazy='raise'" handling when it traverses relationship paths in order to process the "delete" and "delete-orphan" cascade rules. Previously, there was no easy way to generically call :meth:`_orm.Session.delete` on an object that had "lazy='raise'" set up such that only the necessary relationships would be loaded. As "lazy='raise'" is primarily intended to catch SQL loading that emits on attribute access, :meth:`_orm.Session.delete` is now made to behave like other :class:`_orm.Session` methods including :meth:`_orm.Session.merge` as well as :meth:`_orm.Session.flush` along with autoflush. Fixes: #9549 Change-Id: Ie049e66ce2bd35900eae4af0e9b795633303ca63
* | return None for no parententity on ProxyMike Bayer2023-03-211-2/+2
|/ | | | | | | | | | | | Fixed regression where accessing the expression value of a hybrid property on a class that was either unmapped or not-yet-mapped (such as calling upon it within a :func:`_orm.declared_attr` method) would raise an internal error, as an internal fetch for the parent class' mapper would fail and an instruction for this failure to be ignored were inadvertently removed in 2.0. Fixes: #9519 Change-Id: If195d26a5ddd2312a373004eb7a1403e1d11e7a4
* add explicit overload for composite -> callableMike Bayer2023-03-171-0/+22
| | | | | | | | Fixed typing issue where :func:`_orm.composite` would not allow an arbitrary callable as the source of the composite class. Fixes: #9502 Change-Id: I5b098b70b2fb7b48f54eaccbb7d5d3d9bdebc781
* implement active_history for compositesMike Bayer2023-03-091-2/+18
| | | | | | | | | | | | | | | | | | Fixed bug where the "active history" feature was not fully implemented for composite attributes, making it impossible to receive events that included the "old" value. This seems to have been the case with older SQLAlchemy versions as well, where "active_history" would be propagated to the underlying column-based attributes, but an event handler listening to the composite attribute itself would not be given the "old" value being replaced, even if the composite() were set up with active_history=True. Additionally, fixed a regression that's local to 2.0 which disallowed active_history on composite from being assigned to the impl with ``attr.impl.active_history=True``. Fixes: #9460 Change-Id: I6d7752a01c8d3fd78de7a90de10e8c52f9b3cd4e
* update for mypy 1.1.1Mike Bayer2023-03-081-10/+10
| | | | | | | looks like mypy 1.1.1 slipped in after the last jenkins pep484 build for SQLAlchemy. update for its new issues. Change-Id: Ifca967a75d5e592c04b0138aeda9b646067fe59b
* Merge "KeyFuncDict regression fixes and dataclass fixes" into mainmike bayer2023-03-061-35/+98
|\
| * KeyFuncDict regression fixes and dataclass fixesMike Bayer2023-03-051-35/+98
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | adapt None-key warning for non-mapped attributes Fixed multiple regressions due to :ticket:`8372`, involving :func:`_orm.attribute_mapped_collection` (now called :func:`_orm.attribute_keyed_dict`). First, the collection was no longer usable with "key" attributes that were not themselves ordinary mapped attributes; attributes linked to descriptors and/or association proxy attributes have been fixed. Second, if an event or other operation needed access to the "key" in order to populate the dictionary from an mapped attribute that was not loaded, this also would raise an error inappropriately, rather than trying to load the attribute as was the behavior in 1.4. This is also fixed. For both cases, the behavior of :ticket:`8372` has been expanded. :ticket:`8372` introduced an error that raises when the derived key that would be used as a mapped dictionary key is effectively unassigned. In this change, a warning only is emitted if the effective value of the ".key" attribute is ``None``, where it cannot be unambiguously determined if this ``None`` was intentional or not. ``None`` will be not supported as mapped collection dictionary keys going forward (as it typically refers to NULL which means "unknown"). Setting :paramref:`_orm.attribute_keyed_dict.ignore_unpopulated_attribute` will now cause such ``None`` keys to be ignored as well. Add value constructors to dictionary collections Added constructor arguments to the built-in mapping collection types including :class:`.KeyFuncDict`, :func:`_orm.attribute_keyed_dict`, :func:`_orm.column_keyed_dict` so that these dictionary types may be constructed in place given the data up front; this provides further compatibility with tools such as Python dataclasses ``.asdict()`` which relies upon invoking these classes directly as ordinary dictionary classes. Fixes: #9418 Fixes: #9424 Change-Id: Ib16c4e690b7ac3fcc34df2f139cad61c6c4b2b19
* | audition pymssql once more; retire sane_rowcount_returningMike Bayer2023-03-041-13/+2
|/ | | | | | | | | | | | | | | | | | | | | | | | | | pymssql seems to be maintained again and seems to be working completely, so let's try re-enabling it. Fixed issue in the new :class:`.Uuid` datatype which prevented it from working with the pymssql driver. As pymssql seems to be maintained again, restored testing support for pymssql. Tweaked the pymssql dialect to take better advantage of RETURNING for INSERT statements in order to retrieve last inserted primary key values, in the same way as occurs for the mssql+pyodbc dialect right now. Identified that the ``sqlite`` and ``mssql+pyodbc`` dialects are now compatible with the SQLAlchemy ORM's "versioned rows" feature, since SQLAlchemy now computes rowcount for a RETURNING statement in this specific case by counting the rows returned, rather than relying upon ``cursor.rowcount``. In particular, the ORM versioned rows use case (documented at :ref:`mapper_version_counter`) should now be fully supported with the SQL Server pyodbc dialect. Change-Id: I38a0666587212327aecf8f98e86031ab25d1f14d References: #5321 Fixes: #9414