| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
| |
Change-Id: I625af65b3fb1815b1af17dc2ef47dd697fdc3fb1
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
commit two of two. this reorganizes ColumnCollection
to build a new index up front that's used to optimize
the corresponding_column() method.
Additional performance enhancements within ORM-enabled SQL statements,
specifically targeting callcounts within the construction of ORM
statements, using combinations of :func:`_orm.aliased` with
:func:`_sql.union` and similar "compound" constructs, in addition to direct
performance improvements to the ``corresponding_column()`` internal method
that is used heavily by the ORM by constructs like :func:`_orm.aliased` and
similar.
Fixes: #8796
Change-Id: I4a76788007d5a802b9a4081e6a0f6e4b52497b50
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We can cache the annotated cache key for Table, but
for selectables it's not safe, as it fails to pass the
anon_map along and creates many redudant structures in
observed test scenario. It is likely safe for a
Column that's mapped to a Table also, however this is
not implemented here. Will have to see if that part
needs adjusting.
Fixed critical memory issue identified in cache key generation, where for
very large and complex ORM statements that make use of lots of ORM aliases
with subqueries, cache key generation could produce excessively large keys
that were orders of magnitude bigger than the statement itself. Much thanks
to Rollo Konig Brock for their very patient, long term help in finally
identifying this issue.
Also within TypeEngine objects, when we generate elements
for instance variables, skip the None elements at least.
this also saves on tuple complexity.
Fixes: #8790
Change-Id: I448ddbfb45ae0a648815be8dad4faad7d1977427
|
| |
|
|
|
|
|
|
|
|
|
| |
things that were passing with 0.961 need adjustment.
it seems mypy has become very pedantic about the difference
between importing from a module vs. accessing members of that
module as instance variables, so adjust the preloaded
typing block to be explicitly instance variables, since that's
how the accessor works in any case.
Change-Id: I746a3c9102530b7cf9b123aec7be6376657c1169
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rearchitected the schema reflection API to allow some dialects to make use
of high performing batch queries to reflect the schemas of many tables at
once using much fewer queries. The new performance features are targeted
first at the PostgreSQL and Oracle backends, and may be applied to any
dialect that makes use of SELECT queries against system catalog tables to
reflect tables (currently this omits the MySQL and SQLite dialects which
instead make use of parsing the "CREATE TABLE" statement, however these
dialects do not have a pre-existing performance issue with reflection. MS
SQL Server is still a TODO).
The new API is backwards compatible with the previous system, and should
require no changes to third party dialects to retain compatibility;
third party dialects can also opt into the new system by implementing
batched queries for schema reflection.
Along with this change is an updated reflection API that is fully
:pep:`484` typed, features many new methods and some changes.
Fixes: #4379
Change-Id: I897ec09843543aa7012bcdce758792ed3d415d08
|
| |
|
|
|
|
|
| |
also implements __slots__ for QueryableAttribute,
InstrumentedAttribute, Relationship.Comparator.
Change-Id: I47e823160706fc35a616f1179a06c7864089e5b5
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
implement strict typing for schema.py
this module has lots of public API, lots of old decisions
and very hard to follow construction sequences in many
cases, and is also where we get a lot of new feature requests,
so strict typing should help keep things clean.
among improvements here, fixed the pool .info getters
and also figured out how to get ColumnCollection and
related to be covariant so that we may set them up
as returning Column or ColumnClause without any conflicts.
DDL was affected, noting that superclasses of DDLElement
(_DDLCompiles, added recently) can now be passed into
"ddl_if" callables; reorganized ddl into ExecutableDDLElement
as a new name for DDLElement and _DDLCompiles renamed to
BaseDDLElement.
setting up strict also located an API use case that
is completely broken, which is connection.execute(some_default)
returns a scalar value. This case has been deprecated
and new paths have been set up so that connection.scalar()
may be used. This likely wasn't possible in previous
versions because scalar() would assume a CursorResult.
The scalar() change also impacts Session as we have explicit
support (since someone had reported it as a regression)
for session.execute(Sequence()) to work. They will get the
same deprecation message (which omits the word "Connection",
just uses ".execute()" and ".scalar()") and they can then
use Session.scalar() as well. Getting this to type
correctly while still supporting ORM use cases required
some refactoring, and I also set up a keyword only delimeter
for Session.execute() and related as execution_options /
bind_arguments should always be keyword only, applied these
changes to AsyncSession as well.
Additionally simpify Table __init__ now that we are Python
3 only, we can have positional plus explicit kwargs finally.
Simplify Column.__init__ as well again taking advantage
of kw only arguments.
Fill in most/all __init__ methods in sqltypes.py as
the constructor for types is most of the API. should
likely do this for dialect-specific types as well.
Apply _InfoType for all info attributes as should have been
done originally and update descriptor decorators.
Change-Id: I3f9f8ff3f1c8858471ff4545ac83d68c88107527
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
the pep484 task becomes more intense as there is mounting
pressure to come up with a consistency in how data moves
from end-user to instance variable.
current thinking is coming into:
1. there are _typing._XYZArgument objects that represent "what the
user sent"
2. there's the roles, which represent a kind of "filter" for different
kinds of objects. These are mostly important as the argument
we pass to coerce().
3. there's the thing that coerce() returns, which should be what the
construct uses as its internal representation of the thing.
This is _typing._XYZElement.
but there's some controversy over whether or
not we should pass actual ClauseElements around by their role
or not. I think we shouldn't at the moment, but this makes the
"role-ness" of something a little less portable. Like, we have
to set DMLTableRole for TableClause, Join, and Alias, but then
also we have to repeat those three types in order to set up
_DMLTableElement.
Other change introduced here, there was a deannotate=True
for the left/right of a sql.join(). All tests pass without that.
I'd rather not have that there as if we have a join(A, B) where
A, B are mapped classes, we want them inside of the _annotations.
The rationale seems to be performance, but this performance can
be illustrated to be on the compile side which we hope is cached
in the normal case.
CTEs now accommodate for text selects including recursive.
Get typing to accommodate "util.preloaded" cleanly; add "preloaded"
as a real module. This seemed like we would have needed
pep562 `__getattr__()` but we don't, just set names in
globals() as we import them.
References: #6810
Change-Id: I34d17f617de2fe2c086fc556bd55748dc782faf0
|
| |
|
|
|
|
|
|
|
| |
note we are taking out the
ColumnOperartors[SQLCoreOperations] thing; not really clear
why that was needed and at the moment it seems I was likely
confused.
Change-Id: I834b75f9b44f91b97e29f2e1a7b1029bd910e0a1
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
sqlalchemy.sql will require many passes to get all
modules even gradually typed. Will have to pick and
choose what modules can be strictly typed vs. which
can be gradual.
in this patch, emphasis is on visitors.py, cache_key.py,
annotations.py for strict typing, compiler.py is on gradual
typing but has much more structure, in particular where it
connects with the outside world.
The work within compiler.py also reached back out to
engine/cursor.py , default.py quite a bit.
References: #6810
Change-Id: I6e8a29f6013fd216e43d45091bc193f8be0368fd
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
All modules in sqlalchemy.engine are strictly
typed with the exception of cursor, default, and
reflection. cursor and default pass with non-strict
typing, reflection is waiting on the multi-reflection
refactor.
Behavioral changes:
* create_connect_args() methods return a tuple of list,
dict, rather than a list of list, dict
* removed allow_chars parameter from
pyodbc connector ._get_server_version_info()
method
* the parameter list passed to do_executemany is now
a list in all cases. previously, this was being run
through dialect.execute_sequence_format, which
defaults to tuple and was only intended for individual
tuple params.
* broke up dialect.dbapi into dialect.import_dbapi
class method and dialect.dbapi module object. added
a deprecation path for legacy dialects. it's not
really feasible to type a single attr as a classmethod
vs. module type. The "type_compiler" attribute also
has this problem with greater ability to work around,
left that one for now.
* lots of constants changing to be Enum, so that we can
type them. for fixed tuple-position constants in
cursor.py / compiler.py (which are used to avoid the
speed overhead of namedtuple), using Literal[value]
which seems to work well
* some tightening up in Row regarding __getitem__, which
we can do since we are on full 2.0 style result use
* altered the set_connection_execution_options and
set_engine_execution_options event flows so that the
dictionary of options may be mutated within the event
hook, where it will then take effect as the actual
options used. Previously, changing the dict would
be silently ignored which seems counter-intuitive
and not very useful.
* A lot of DefaultDialect/DefaultExecutionContext
methods and attributes, including underscored ones, move
to interfaces. This is not fully ideal as it means
the Dialect/ExecutionContext interfaces aren't publicly
subclassable directly, but their current purpose
is more of documentation for dialect authors who should
(and certainly are) still be subclassing the DefaultXYZ
versions in all cases
Overall, Result was the most extremely difficult class
hierarchy to type here as this hierarchy passes through
largely amorphous "row" datatypes throughout, which
can in fact by all kinds of different things, like
raw DBAPI rows, or Row objects, or "scalar"/Any, but
at the same time these types have meaning so I tried still
maintaining some level of semantic markings for these,
it highlights how complex Result is now, as it's trying
to be extremely efficient and inlined while also being
very open-ended and extensible.
Change-Id: I98b75c0c09eab5355fc7a33ba41dd9874274f12a
|
| |
|
|
|
|
|
|
|
|
| |
__future__.annotations mode allows us to use non-string
annotations for argument and return types in most cases,
but more importantly it removes a large amount of runtime
overhead that would be spent in evaluating the annotations.
Change-Id: I2f5b6126fe0019713fc50001be3627b664019ede
References: #6810
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Changed AliasedInsp to use __slots__
* Migrated all of strategy_options to use __slots__ for objects.
Adds new infrastructure to traversals to support shallow
copy, to dict and from dict based on internal traversal
attributes. Load / _LoadElement then leverage this to
provide clone / generative / getstate without the need
for __dict__ or explicit attribute lists.
Doing this change revealed that there are lots of things that
trigger off of whether or not a class has a __visit_name__ attribute.
so to suit that we've gone back to having Visitable, which is
a better name than Traversible at this point (I think
Traversible is mis-spelled too).
Change-Id: I13d04e494339fac9dbda0b8e78153418abebaf72
References: #7527
|
|
|
All but one metaclass used internally can now
be replaced using __init_subclass__(). Within this
patch we remove:
* events._EventMeta
* sql.visitors.TraversibleType
* sql.visitors.InternalTraversibleType
* testing.fixtures.FindFixture
* testing.fixtures.FindFixtureDeclarative
* langhelpers.EnsureKWArgType
* sql.functions._GenericMeta
* sql.type_api.VisitableCheckKWArg (was a mixture of TraversibleType
and EnsureKWArgType)
The remaining internal class is MetaOptions used by the
sql.Options object which is in turn currently mostly for
ORM internal use, as this type implements class level overrides
for the ``+`` operator.
For declarative, removing DeclarativeMeta in place of
an `__init_subclass__()` class would not be fully feasible as
it would break backwards compatibility with applications that
refer to this class explicitly, but also DeclarativeMeta intercepts
class-level attribute set and delete operations which is a widely
used pattern. An option for declarative base to use
`__init_subclass__()` should be provided but this is out of
scope for this particular change.
Change-Id: I8aa898c7ab59d887739037d34b1cbab36521ab78
References: #6810
|
