| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
fix a handful of warnings that were emitting but not raising,
usually because they were inside an "expect_warnings" block.
modify "expect_warnings" to always use "raise_on_any_unexpected"
behavior; remove this parameter.
Fixed issue in semi-private ``await_only()`` and ``await_fallback()``
concurrency functions where the given awaitable would remain un-awaited if
the function threw a ``GreenletError``, which could cause "was not awaited"
warnings later on if the program continued. In this case, the given
awaitable is now cancelled before the exception is thrown.
Change-Id: I33668c5e8c670454a3d879e559096fb873b57244
|
| |/
|
|
|
|
|
|
|
|
|
| |
Fixed issue in :func:`_orm.mapped_column` construct where the correct
warning for "column X named directly multiple times" would not be emitted
when ORM mapped attributes referred to the same :class:`_schema.Column`, if
the :func:`_orm.mapped_column` construct were involved, raising an internal
assertion instead.
Fixes: #9630
Change-Id: I5d9dfaaa225aefb487c9cd981ba3ad78507bb577
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Repaired a major shortcoming which was identified in the
:ref:`engine_insertmanyvalues` performance optimization feature first
introduced in the 2.0 series. This was a continuation of the change in
2.0.9 which disabled the SQL Server version of the feature due to a
reliance in the ORM on apparent row ordering that is not guaranteed to take
place. The fix applies new logic to all "insertmanyvalues" operations,
which takes effect when a new parameter
:paramref:`_dml.Insert.returning.sort_by_parameter_order` on the
:meth:`_dml.Insert.returning` or :meth:`_dml.UpdateBase.return_defaults`
methods, that through a combination of alternate SQL forms, direct
correspondence of client side parameters, and in some cases downgrading to
running row-at-a-time, will apply sorting to each batch of returned rows
using correspondence to primary key or other unique values in each row
which can be correlated to the input data.
Performance impact is expected to be minimal as nearly all common primary
key scenarios are suitable for parameter-ordered batching to be
achieved for all backends other than SQLite, while "row-at-a-time"
mode operates with a bare minimum of Python overhead compared to the very
heavyweight approaches used in the 1.x series. For SQLite, there is no
difference in performance when "row-at-a-time" mode is used.
It's anticipated that with an efficient "row-at-a-time" INSERT with
RETURNING batching capability, the "insertmanyvalues" feature can be later
be more easily generalized to third party backends that include RETURNING
support but not necessarily easy ways to guarantee a correspondence
with parameter order.
Fixes: #9618
References: #9603
Change-Id: I1d79353f5f19638f752936ba1c35e4dc235a8b7c
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixed bug in ORM Declarative Dataclasses where the
:func:`_orm.queryable_attribute` and :func:`_orm.column_property`
constructs, which are documented as read-only constructs in the context of
a Declarative mapping, could not be used with a
:class:`_orm.MappedAsDataclass` class without adding ``init=False``, which
in the case of :func:`_orm.queryable_attribute` was not possible as no
``init`` parameter was included. These constructs have been modified from a
dataclass perspective to be assumed to be "read only", setting
``init=False`` by default and no longer including them in the pep-681
constructor. The dataclass parameters for :func:`_orm.column_property`
``init``, ``default``, ``default_factory``, ``kw_only`` are now deprecated;
these fields don't apply to :func:`_orm.column_property` as used in a
Declarative dataclasses configuration where the construct would be
read-only. Also added read-specific parameter
:paramref:`_orm.queryable_attribute.compare` to
:func:`_orm.queryable_attribute`; :paramref:`_orm.queryable_attribute.repr`
was already present.
Added missing :paramref:`_orm.mapped_column.active_history` parameter
to :func:`_orm.mapped_column` construct.
Fixes: #9628
Change-Id: I2ab44d6b763b20410bd1ebb5ac949a6d223f1ce2
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Expanded the warning emitted when a plain :func:`_sql.column` object is
present in a Declarative mapping to include any arbitrary SQL expression
that is not declared within an appropriate property type such as
:func:`_orm.column_property`, :func:`_orm.deferred`, etc. These attributes
are otherwise not mapped at all and remain unchanged within the class
dictionary. As it seems likely that such an expression is usually not
what's intended, this case now warns for all such otherwise ignored
expressions, rather than just the :func:`_sql.column` case.
Fixes: #9537
Change-Id: Ic4ca7a071a28adf4ea8680690025d927522a0805
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds the very small plugin flake8-import-single which
will prevent us from having an import with more than one symbol
on a line.
Flake8 by itself prevents this pattern with E401:
import collections, os, sys
However does not do anything with this:
from sqlalchemy import Column, text
Both statements have the same issues generating merge artifacts
as well as presenting a manual decision to be made. While
zimports generally cleans up such imports at the top level, we
don't enforce zimports / pre-commit use.
the plugin finds the same issue for imports that are inside of
test methods. We shouldn't usually have imports in test methods
so most of them here are moved to be top level.
The version is pinned at 0.1.5; the project seems to have no
activity since 2019, however there are three 0.1.6dev releases
on pypi which stopped in September 2019, they seem to be
experiments with packaging. The source for 0.1.5
is extremely simple and only reveals one method to flake8
(the run() method).
Change-Id: Icea894e43bad9c0b5d4feb5f49c6c666d6ea6aa1
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
To accommodate a change in column ordering used by ORM Declarative in
SQLAlchemy 2.0, a new parameter :paramref:`_orm.mapped_column.sort_order`
has been added that can be used to control the order of the columns defined
in the table by the ORM, for common use cases such as mixins with primary
key columns that should appear first in tables. The change notes at
:ref:`change_9297` illustrate the default change in ordering behavior
(which is part of all SQLAlchemy 2.0 releases) as well as use of the
:paramref:`_orm.mapped_column.sort_order` to control column ordering when
using mixins and multiple classes (new in 2.0.4).
Fixes: #9297
Change-Id: Ic7163d64efdc0eccb53d6ae0dd89ec83427fb675
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixed regression caused by the fix for :ticket:`9171`, which itself was
fixing a regression, involving the mechanics of ``__init__()`` on classes
that extend from :class:`_orm.DeclarativeBase`. The change made it such
that ``__init__()`` was applied to the user-defined base if there were no
``__init__()`` method directly on the class. This has been adjusted so that
``__init__()`` is applied only if no other class in the hierarchy of the
user-defined base has an ``__init__()`` method. This again allows
user-defined base classes based on :class:`_orm.DeclarativeBase` to include
mixins that themselves include a custom ``__init__()`` method.
Fixes: #9249
Change-Id: I78f32590ce9ffe245eccb4bd5bd7c884d4e015d5
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Repaired ORM Declarative mappings to allow for the
:paramref:`_orm.Mapper.primary_key` parameter to be specified within
``__mapper_args__`` when using :func:`_orm.mapped_column`. Despite this
usage being directly in the 2.0 documentation, the :class:`_orm.Mapper` was
not accepting the :func:`_orm.mapped_column` construct in this context. Ths
feature was already working for the :paramref:`_orm.Mapper.version_id_col`
and :paramref:`_orm.Mapper.polymorphic_on` parameters.
As part of this change, the ``__mapper_args__`` attribute may be specified
without using :func:`_orm.declared_attr` on a non-mapped mixin class,
including a ``"primary_key"`` entry that refers to :class:`_schema.Column`
or :func:`_orm.mapped_column` objects locally present on the mixin;
Declarative will also translate these columns into the correct ones for a
particular mapped class. This again was working already for the
:paramref:`_orm.Mapper.version_id_col` and
:paramref:`_orm.Mapper.polymorphic_on` parameters. Additionally,
elements within ``"primary_key"`` may be indicated as string names of
existing mapped properties.
Fixes: #9240
Change-Id: Ie2000273289fa23e0af21ef9c6feb3962a8b848c
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixed regression in :class:`.DeclarativeBase` class where the registry's
default constructor would not be applied to the base itself, which is
different from how the previous :func:`_orm.declarative_base` construct
works. This would prevent a mapped class with its own ``__init__()`` method
from calling ``super().__init__()`` in order to access the registry's
default constructor and automatically populate attributes, instead hitting
``object.__init__()`` which would raise a ``TypeError`` on any arguments.
This is a very simple change in code, however explaining it is
very complicated.
Fixes: #9171
Change-Id: I4baecdf671861a8198d835e286fe19a51ecda126
|
| |
|
|
|
| |
Fixes: #8624
Change-Id: Ia7a66ae9ba534ed7152f95dfd0f7d05b9d00165a
|
| |
|
|
|
|
|
| |
this is a bit of a goofy test which can occasionally fail,
so add a set to prevent names from being duplicated.
Change-Id: Ie7ac605f517ce31f2c5d092a692d93f733180716
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixed issue where table reflection using :paramref:`.Table.extend_existing`
would fail to deduplicate a same-named column if the existing
:class:`.Table` used a separate key. The
:paramref:`.Table.autoload_replace` parameter would allow the column to be
skipped but under no circumstances should a :class:`.Table` ever have the
same-named column twice.
Additionally, changed deprecation warnings to exceptions
as were implemented in I1d58c8ebe081079cb669e7ead60886ffc1b1a7f5 .
Fixes: #8925
Change-Id: I83d0f8658177a7ffbb06e01dbca91377d1a98d49
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixed issue where use of an unknown datatype within a :class:`.Mapped`
annotation for a column-based attribute would silently fail to map the
attribute, rather than reporting an exception; an informative exception
message is now raised.
tighten up iteration of names on mapped classes to more fully
exclude a large number of underscored names, so that we can avoid trying
to look at annotations for them or anything else. centralize the
"list of names we care about" more fully within _cls_attr_resolver
and base it on underscore conventions we should usually ignore,
with the exception of the few underscore names we want to see.
Fixes: #8888
Change-Id: I3c0a1666579fe67b3c40cc74fa443b6f1de354ce
|
| |
|
|
|
|
|
|
| |
command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>"
pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not
exists in sqlalchemy fixtures
Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Changed a fundamental configuration behavior of :class:`.Mapper`, where
:class:`_schema.Column` objects that are explicitly present in the
:paramref:`_orm.Mapper.properties` dictionary, either directly or enclosed
within a mapper property object, will now be mapped within the order of how
they appear within the mapped :class:`.Table` (or other selectable) itself
(assuming they are in fact part of that table's list of columns), thereby
maintaining the same order of columns in the mapped selectable as is
instrumented on the mapped class, as well as what renders in an ORM SELECT
statement for that mapper. Previously (where "previously" means since
version 0.0.1), :class:`.Column` objects in the
:paramref:`_orm.Mapper.properties` dictionary would always be mapped first,
ahead of when the other columns in the mapped :class:`.Table` would be
mapped, causing a discrepancy in the order in which the mapper would
assign attributes to the mapped class as well as the order in which they
would render in statements.
The change most prominently takes place in the way that Declarative
assigns declared columns to the :class:`.Mapper`, specifically how
:class:`.Column` (or :func:`_orm.mapped_column`) objects are handled
when they have a DDL name that is explicitly different from the mapped
attribute name, as well as when constructs such as :func:`_orm.deferred`
etc. are used. The new behavior will see the column ordering within
the mapped :class:`.Table` being the same order in which the attributes
are mapped onto the class, assigned within the :class:`.Mapper` itself,
and rendered in ORM statements such as SELECT statements, independent
of how the :class:`_schema.Column` was configured against the
:class:`.Mapper`.
Fixes: #8705
Change-Id: I95cc05061a97fe6b1654bab70e2f6da30f8f3bd3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We made all the MapperProperty classes a subclass
of Mapped[] to allow declarative mappings to name
Mapped[] on the left side. this was cheating a bit because
MapperProperty is not actually a descriptor, and the mapping
process replaces the object with InstrumentedAttribute at
mapping time, which is the actual Mapped[] descriptor.
But now in I6929f3da6e441cad92285e7309030a9bac4e429d we
are considering making the "cheating" a little more extensive
by putting DynamicMapped / WriteOnlyMapped in Relationship's
hierarchy, which need a flat out "type: ignore" to work.
Instead of pushing more cheats into the core classes, move
out the "Declarative"-facing versions of these classes to be
typing only: Relationship, Composite, Synonym, and MappedSQLExpression
added for ColumnProperty. Keep the internals expressed on the
old names, RelationshipProperty, CompositeProperty, SynonymProperty,
ColumnProprerty, which will remain "pure" with fully correct typing.
then have the typing only endpoints be where the "cheating"
and "type: ignores" have to happen, so that these are more or less
slightly better forms of "Any".
Change-Id: Ied7cc11196c9204da6851f49593d1b1fd2ef8ad8
|
| |
|
|
|
| |
Fixes: #8248
Change-Id: I4f4c690dd8659eaf74e9c757d681e9edc7d33eee
|
| |
|
|
|
|
| |
trying to get remaining must-haves for ORM
Change-Id: I66a3ecbbb8e5ba37c818c8a92737b576ecf012f7
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
large patch to get ORM / typing efforts started.
this is to support adding new test cases to mypy,
support dropping sqlalchemy2-stubs entirely from the
test suite, validate major ORM typing reorganization
to eliminate the need for the mypy plugin.
* New declarative approach which uses annotation
introspection, fixes: #7535
* Mapped[] is now at the base of all ORM constructs
that find themselves in classes, to support direct
typing without plugins
* Mypy plugin updated for new typing structures
* Mypy test suite broken out into "plugin" tests vs.
"plain" tests, and enhanced to better support test
structures where we assert that various objects are
introspected by the type checker as we expect.
as we go forward with typing, we will
add new use cases to "plain" where we can assert that
types are introspected as we expect.
* For typing support, users will be much more exposed to the
class names of things. Add these all to "sqlalchemy" import
space.
* Column(ForeignKey()) no longer needs to be `@declared_attr`
if the FK refers to a remote table
* composite() attributes mapped to a dataclass no longer
need to implement a `__composite_values__()` method
* with_variant() accepts multiple dialect names
Change-Id: I22797c0be73a8fbbd2d6f5e0c0b7258b17fe145d
Fixes: #7535
Fixes: #7551
References: #6810
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Improvements to the test suite's integration with pytest such that the
"warnings" plugin, if manually enabled, will not interfere with the test
suite, such that third parties can enable the warnings plugin or make use
of the ``-W`` parameter and SQLAlchemy's test suite will continue to pass.
Additionally, modernized the detection of the "pytest-xdist" plugin so that
plugins can be globally disabled using PYTEST_DISABLE_PLUGIN_AUTOLOAD=1
without breaking the test suite if xdist were still installed. Warning
filters that promote deprecation warnings to errors are now localized to
SQLAlchemy-specific warnings, or within SQLAlchemy-specific sources for
general Python deprecation warnings, so that non-SQLAlchemy deprecation
warnings emitted from pytest plugins should also not impact the test suite.
Fixes: #7599
Change-Id: Ibcf09af25228d39ee5a943fda82d8a9302433726
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
introduces:
1. new mapped_column() helper
2. DeclarativeBase helper
3. declared_attr has been re-typed
4. rework of Mapped[] to return InstrumentedAtribute for
class get, so works without Mapped itself having expression
methods
5. ORM constructs now generic on [_T]
also includes some early typing work, most of which will
be in later commits:
1. URL and History become typing.NamedTuple
2. come up with type-checking friendly way of type
checking cy extensions, where type checking will be applied
to the py versions, just needed to come up with a succinct
conditional pattern for the imports
References: #6810
References: #7535
References: #7562
Change-Id: Ie5d9a44631626c021d130ca4ce395aba623c71fb
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Finalize all remaining removed-in-2.0 changes so that we
can begin doing pep-484 typing without old things
getting in the way (we will also have to do public_factory).
note there are a few "moved_in_20()" and "became_legacy_in_20()"
warnings still in place. The SQLALCHEMY_WARN_20 variable
is now removed.
Also removed here are the legacy "in place mutators" for Select
statements, and some keyword-only argument signatures in Core
have been added.
Also in the big change department, the ORM mapper() function
is removed entirely; the Mapper class is otherwise unchanged,
just the public-facing API function. Mappers are now always
given a registry in which to participate, however the
argument signature of Mapper is not changed. ideally "registry"
would be the first positional argument.
Fixes: #7257
Change-Id: Ic70c57b9f1cf7eb996338af5183b11bdeb3e1623
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The architecture of Load is mostly rewritten here.
The change includes removal of the "pluggable" aspect
of the loader options, which would patch new methods onto
Load. This has been replaced by normal methods that
respond normally to typing annotations. As part of this
change, the bake_loaders() and unbake_loaders() options,
which have no effect since 1.4 and were unlikely to be
in any common use, have been removed.
Additionally, to support annotations for methods that
make use of @decorator, @generative etc., modified
format_argspec_plus to no longer return "args", instead
returns "grouped_args" which is always grouped and
allows return annotations to format correctly.
Fixes: #6986
Change-Id: I6117c642345cdde65a64389bba6057ddd5374427
|
| |
|
|
| |
Change-Id: I8172fdcc3103ff92aa049827728484c8779af6b7
|
| |
|
|
|
| |
References: #4600
Change-Id: I2a62ddfe00bc562720f0eae700a497495d7a987a
|
| |
|
|
|
| |
References: #4600
Change-Id: I61e35bc93fe95610ae75b31c18a3282558cd4ffe
|
| |
|
|
|
|
|
| |
There are still some SQLite / MySQL specific occurrences
of "master" but this is most of it.
Change-Id: I0144c992e2f0207777e20e058b63a11c031986b9
|
| |
|
|
|
|
|
| |
* sqlalchemy.ext.declarative names
* declarative_base(bind)
Change-Id: I0ca26894b224458b58e46504c5ff7b5d3031a829
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An extra layer of warning messages has been added to the functionality
of :meth:`_orm.Query.join` and the ORM version of
:meth:`_sql.Select.join`, where a few places where "automatic aliasing"
continues to occur will now be called out as a pattern to avoid, mostly
specific to the area of joined table inheritance where classes that share
common base tables are being joined together without using explicit aliases.
One case emits a legacy warning for a pattern that's not recommended,
the other case is fully deprecated.
The automatic aliasing within ORM join() which occurs for overlapping
mapped tables does not work consistently with all APIs such as
``contains_eager()``, and rather than continue to try to make these use
cases work everywhere, replacing with a more user-explicit pattern
is clearer, less prone to bugs and simplifies SQLAlchemy's internals
further.
The warnings include links to the errors.rst page where each pattern is
demonstrated along with the recommended pattern to fix.
* Improved the exception message generated when configuring a mapping with
joined table inheritance where the two tables either have no foreign key
relationships set up, or where they have multiple foreign key relationships
set up. The message is now ORM specific and includes context that the
:paramref:`_orm.Mapper.inherit_condition` parameter may be needed
particularly for the ambiguous foreign keys case.
* Add explicit support in the _expect_warnings() assertion for nested
_expect_warnings calls
* generalize the NoCache fixture, which we also need to catch warnings
during compilation consistently
* generalize the __str__() method for the HasCode mixin so all warnings
and errors include the code link in their string
Fixes: #6974
Change-Id: I84ed79ba2112c39eaab7973b6d6f46de7fa80842
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixed regression where recent changes to support Python dataclasses had the
inadvertent effect that an ORM mapped class could not successfully override
the ``__new__()`` method.
In this case the "__new__" method comes out as staticmethod in
cls.__dict__ vs. a function in the metaclass dict_, so comparing
using identity fails. I was hoping not to have too much
"dataclass" hardcoded, the logic here if it were generalized
to other attribute declaration systems there
would still have a flag that indicates an attribute is part of
the "special declaration system".
Fixes: #6331
Change-Id: Ia28a44fb57c668fa2fc5cd1ff38fd511f2c747e6
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Fixed bug where user-mapped classes that contained an attribute named
"registry" would cause conflicts with the new registry-based mapping system
when using :class:`.DeclarativeMeta`. While the attribute remains
something that can be set explicitly on a declarative base to be
consumed by the metaclass, once located it is placed under a private
class variable so it does not conflict with future subclasses that use
the same name for other purposes.
Fixes: #6054
Change-Id: I1f2e04b0d74c493e7e90eadead4e861d8960a794
|
| |
|
|
|
|
|
|
|
| |
officially document how to make declarative base
non-dynamically such that mypy and similar tools
can process it without plugins
Change-Id: I884f9a7c06c4a8b8111948a2dd64e308e7dce4fc
References: #4609
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Mapper "configuration", which occurs within the
:func:`_orm.configure_mappers` function, is now organized to be on a
per-registry basis. This allows for example the mappers within a certain
declarative base to be configured, but not those of another base that is
also present in memory. The goal is to provide a means of reducing
application startup time by only running the "configure" process for sets
of mappers that are needed. This also adds the
:meth:`_orm.registry.configure` method that will run configure for the
mappers local in a particular registry only.
Fixes: #5897
Change-Id: I14bd96982d6d46e241bd6baa2cf97471d21e7caa
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
To allow the "connection" pytest fixture and others work
correctly in conjunction with setup/teardown that expects
to be external to the transaction, remove and prevent any usage
of "xdist" style names that are hardcoded by pytest to run
inside of fixtures, even function level ones. Instead use
pytest autouse fixtures to implement our own
r"setup|teardown_test(?:_class)?" methods so that we can ensure
function-scoped fixtures are run within them. A new more
explicit flow is set up within plugin_base and pytestplugin
such that the order of setup/teardown steps, which there are now
many, is fully documented and controllable. New granularity
has been added to the test teardown phase to distinguish
between "end of the test" when lock-holding structures on
connections should be released to allow for table drops,
vs. "end of the test plus its teardown steps" when we can
perform final cleanup on connections and run assertions
that everything is closed out.
From there we can remove most of the defensive "tear down everything"
logic inside of engines which for many years would frequently dispose
of pools over and over again, creating for a broken and expensive
connection flow. A quick test shows that running test/sql/ against
a single Postgresql engine with the new approach uses 75% fewer new
connections, creating 42 new connections total, vs. 164 new
connections total with the previous system.
As part of this, the new fixtures metadata/connection/future_connection
have been integrated such that they can be combined together
effectively. The fixture_session(), provide_metadata() fixtures
have been improved, including that fixture_session() now strongly
references sessions which are explicitly torn down before
table drops occur afer a test.
Major changes have been made to the
ConnectionKiller such that it now features different "scopes" for
testing engines and will limit its cleanup to those testing
engines corresponding to end of test, end of test class, or
end of test session. The system by which it tracks DBAPI
connections has been reworked, is ultimately somewhat similar to
how it worked before but is organized more clearly along
with the proxy-tracking logic. A "testing_engine" fixture
is also added that works as a pytest fixture rather than a
standalone function. The connection cleanup logic should
now be very robust, as we now can use the same global
connection pools for the whole suite without ever disposing
them, while also running a query for PostgreSQL
locks remaining after every test and assert there are no open
transactions leaking between tests at all. Additional steps
are added that also accommodate for asyncio connections not
explicitly closed, as is the case for legacy sync-style
tests as well as the async tests themselves.
As always, hundreds of tests are further refined to use the
new fixtures where problems with loose connections were identified,
largely as a result of the new PostgreSQL assertions,
many more tests have moved from legacy patterns into the newest.
An unfortunate discovery during the creation of this system is that
autouse fixtures (as well as if they are set up by
@pytest.mark.usefixtures) are not usable at our current scale with pytest
4.6.11 running under Python 2. It's unclear if this is due
to the older version of pytest or how it implements itself for
Python 2, as well as if the issue is CPU slowness or just large
memory use, but collecting the full span of tests takes over
a minute for a single process when any autouse fixtures are in
place and on CI the jobs just time out after ten minutes.
So at the moment this patch also reinvents a small version of
"autouse" fixtures when py2k is running, which skips generating
the real fixture and instead uses two global pytest fixtures
(which don't seem to impact performance) to invoke the
"autouse" fixtures ourselves outside of pytest.
This will limit our ability to do more with fixtures
until we can remove py2k support.
py.test is still observed to be much slower in collection in the
4.6.11 version compared to modern 6.2 versions, so add support for new
TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that
will run the suite for fewer backends under Python 2. For Python 3
pin pytest to modern 6.2 versions where performance for collection
has been improved greatly.
Includes the following improvements:
Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would
be raised rather than :class:`.exc.TimeoutError`. Also repaired the
:paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using
the async engine, which previously would ignore the timeout and block
rather than timing out immediately as is the behavior with regular
:class:`.QueuePool`.
For asyncio the connection pool will now also not interact
at all with an asyncio connection whose ConnectionFairy is
being garbage collected; a warning that the connection was
not properly closed is emitted and the connection is discarded.
Within the test suite the ConnectionKiller is now maintaining
strong references to all DBAPI connections and ensuring they
are released when tests end, including those whose ConnectionFairy
proxies are GCed.
Identified cx_Oracle.stmtcachesize as a major factor in Oracle
test scalability issues, this can be reset on a per-test basis
rather than setting it to zero across the board. the addition
of this flag has resolved the long-standing oracle "two task"
error problem.
For SQL Server, changed the temp table style used by the
"suite" tests to be the double-pound-sign, i.e. global,
variety, which is much easier to test generically. There
are already reflection tests that are more finely tuned
to both styles of temp table within the mssql test
suite. Additionally, added an extra step to the
"dropfirst" mechanism for SQL Server that will remove
all foreign key constraints first as some issues were
observed when using this flag when multiple schemas
had not been torn down.
Identified and fixed two subtle failure modes in the
engine, when commit/rollback fails in a begin()
context manager, the connection is explicitly closed,
and when "initialize()" fails on the first new connection
of a dialect, the transactional state on that connection
is still rolled back.
Fixes: #5826
Fixes: #5827
Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
in Iae6ab95938a7e92b6d42086aec534af27b5577d3 I missed
that the "bind" was being stuck onto the MetaData in
TablesTest, which led thousands of ORM tests to still use
bound metadata. Keep looking for bound metadata.
standardize all ORM tests on a single means of getting a
Session when the Session API isn't the thing we are directly
testing, using a new function fixture_session() that replaces
create_session() and uses modern defaults.
Change-Id: Iaf71206e9ee568151496d8bc213a069504bf65ef
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
importantly this means we can remove bound metadata from
the fixtures that are used by Alembic's test suite.
hopefully this is the last one that has to happen to allow
Alembic to be fully 1.4/2.0.
Start moving from @testing.provide_metadata to a pytest
metadata fixture. This does not seem to have any negative
effects even though TablesTest uses a "self.metadata" attribute.
Change-Id: Iae6ab95938a7e92b6d42086aec534af27b5577d3
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Added a comprehensive check and an informative error message for the case
where a mapped class, or a string mapped class name, is passed to
:paramref:`_orm.relationship.secondary`. This is an extremely common error
which warrants a clear message.
Additionally, added a new rule to the class registry resolution such that
with regards to the :paramref:`_orm.relationship.secondary` parameter, if a
mapped class and its table are of the identical string name, the
:class:`.Table` will be favored when resolving this parameter. In all
other cases, the class continues to be favored if a class and table
share the identical name.
Fixes: #5774
Change-Id: I65069d79c1c3897fbd1081a8e1edf3b63b497223
|
| |
|
|
|
|
|
|
|
| |
Fixed bug where the now-deprecated ``autoload`` parameter was being called
internally within the reflection routines when a related table were
reflected.
Fixes: #5684
Change-Id: I6ab439a2f49ff1ae2d3c7a15b531cbafbc3cf594
|
| |
|
|
|
|
|
|
| |
Fixes gc collection issues later on that occur in test_mixin.py
HUGE thanks to Federico Caselli for finding the issue!
Change-Id: I6444e868ab3d6ff62fb644ebe2fbded7df139c9c
|
|
|
The ORM Declarative system is now unified into the ORM itself, with new
import spaces under ``sqlalchemy.orm`` and new kinds of mappings. Support
for decorator-based mappings without using a base class, support for
classical style-mapper() calls that have access to the declarative class
registry for relationships, and full integration of Declarative with 3rd
party class attribute systems like ``dataclasses`` and ``attrs`` is now
supported.
Fixes: #5508
Change-Id: I130b2b6edff6450bfe8a3e6baa099ff04b5471ff
|
