summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/sql
Commit message (Collapse)AuthorAgeFilesLines
...
* | Initial ORM typing layoutMike Bayer2022-01-147-381/+331
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | introduces: 1. new mapped_column() helper 2. DeclarativeBase helper 3. declared_attr has been re-typed 4. rework of Mapped[] to return InstrumentedAtribute for class get, so works without Mapped itself having expression methods 5. ORM constructs now generic on [_T] also includes some early typing work, most of which will be in later commits: 1. URL and History become typing.NamedTuple 2. come up with type-checking friendly way of type checking cy extensions, where type checking will be applied to the py versions, just needed to come up with a succinct conditional pattern for the imports References: #6810 References: #7535 References: #7562 Change-Id: Ie5d9a44631626c021d130ca4ce395aba623c71fb
* | initial reorganize for static typingMike Bayer2022-01-1217-2756/+3889
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | start applying foundational annotations to key elements. two main elements addressed here: 1. removal of public_factory() and replacement with explicit functions. this just works much better with typing. 2. typing support for column expressions and operators. The biggest part of this involves stubbing out all the ColumnOperators methods under ColumnElement in a TYPE_CHECKING section. Took me a while to see this method vs. much more complicated things I thought I needed. Also for this version implementing #7519, ColumnElement types against the Python type and not TypeEngine. it is hoped this leads to easier transferrence between ORM/Core as well as eventual support for result set typing. Not clear yet how well this approach will work and what new issues it may introduce. given the current approach we now get full, rich typing for scenarios like this: from sqlalchemy import column, Integer, String, Boolean c1 = column('a', String) c2 = column('a', Integer) expr1 = c2.in_([1, 2, 3]) expr2 = c2 / 5 expr3 = -c2 expr4_a = ~(c2 == 5) expr4_b = ~column('q', Boolean) expr5 = c1 + 'x' expr6 = c2 + 10 Fixes: #7519 Fixes: #6810 Change-Id: I078d9f57955549f6f7868314287175f6c61c44cb
* Merge "remove internal use of metaclasses" into mainmike bayer2022-01-1114-940/+988
|\
| * remove internal use of metaclassesMike Bayer2022-01-1114-940/+988
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | All but one metaclass used internally can now be replaced using __init_subclass__(). Within this patch we remove: * events._EventMeta * sql.visitors.TraversibleType * sql.visitors.InternalTraversibleType * testing.fixtures.FindFixture * testing.fixtures.FindFixtureDeclarative * langhelpers.EnsureKWArgType * sql.functions._GenericMeta * sql.type_api.VisitableCheckKWArg (was a mixture of TraversibleType and EnsureKWArgType) The remaining internal class is MetaOptions used by the sql.Options object which is in turn currently mostly for ORM internal use, as this type implements class level overrides for the ``+`` operator. For declarative, removing DeclarativeMeta in place of an `__init_subclass__()` class would not be fully feasible as it would break backwards compatibility with applications that refer to this class explicitly, but also DeclarativeMeta intercepts class-level attribute set and delete operations which is a widely used pattern. An option for declarative base to use `__init_subclass__()` should be provided but this is out of scope for this particular change. Change-Id: I8aa898c7ab59d887739037d34b1cbab36521ab78 References: #6810
* | Merge "implement second-level type resolution for literals" into mainmike bayer2022-01-112-1/+28
|\ \ | |/ |/|
| * implement second-level type resolution for literalsMike Bayer2022-01-112-1/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added additional rule to the system that determines ``TypeEngine`` implementations from Python literals to apply a second level of adjustment to the type, so that a Python datetime with or without tzinfo can set the ``timezone=True`` parameter on the returned :class:`.DateTime` object, as well as :class:`.Time`. This helps with some round-trip scenarios on type-sensitive PostgreSQL dialects such as asyncpg, psycopg3 (2.0 only). For 1.4 specifically, the backport improves support for asyncpg handling of TIME WITH TIMEZONE, which was not fully implemented. 2.0's reworked PostgreSQL architecture had this handled already. Fixes: #7537 Change-Id: Icdb07db85af5f7f39f1c1ef855fe27609770094b
* | Merge "Fix various source comment/doc typos" into mainmike bayer2022-01-071-1/+1
|\ \
| * | Fix various source comment/doc typosluz paz2021-12-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ### Description Found via `codespell -q 3 -L ba,crate,datas,froms,gord,hist,inh,nd,selectin,strat,ue` Also added codespell to the pep8 tox env ### Checklist This pull request is: - [x] A documentation / typographical error fix - Good to go, no issue or tests are needed Closes: #7338 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7338 Pull-request-sha: 0deac2219396bc0eba7da53eb3a80932edbf2dd7 Change-Id: Icd61db31c8dc655d4a39d8a304194804d08555fe
* | | happy new year 2022Mike Bayer2022-01-0624-24/+24
| | | | | | | | | | | | Change-Id: I49abf2607e0eb0623650efdf0091b1fb3db737ea
* | | Merge "Remove all remaining removed_in_20 warnings slated for removal" into mainmike bayer2022-01-066-603/+45
|\ \ \
| * | | Remove all remaining removed_in_20 warnings slated for removalMike Bayer2022-01-056-603/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Finalize all remaining removed-in-2.0 changes so that we can begin doing pep-484 typing without old things getting in the way (we will also have to do public_factory). note there are a few "moved_in_20()" and "became_legacy_in_20()" warnings still in place. The SQLALCHEMY_WARN_20 variable is now removed. Also removed here are the legacy "in place mutators" for Select statements, and some keyword-only argument signatures in Core have been added. Also in the big change department, the ORM mapper() function is removed entirely; the Mapper class is otherwise unchanged, just the public-facing API function. Mappers are now always given a registry in which to participate, however the argument signature of Mapper is not changed. ideally "registry" would be the first positional argument. Fixes: #7257 Change-Id: Ic70c57b9f1cf7eb996338af5183b11bdeb3e1623
* | | | Remove redundant code for EOL Python <= 3.6Hugo van Kemenade2022-01-061-4/+2
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | <!-- Provide a general summary of your proposed changes in the Title field above --> ### Description <!-- Describe your changes in detail --> There's a few bits and pieces of code to support Python <= 3.6 which are no longer needed and can be removed, to slightly simplify the codebase. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [x] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #7544 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7544 Pull-request-sha: 282b4a91282902a57807aa2541b75b272b547127 Change-Id: I9ddf15fcf72551d52e3f027f337c7fee4aa9083b
* | | Update Black's target-version to py37Hugo van Kemenade2022-01-0511-68/+68
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | <!-- Provide a general summary of your proposed changes in the Title field above --> ### Description <!-- Describe your changes in detail --> Black's `target-version` was still set to `['py27', 'py36']`. Set it to `[py37]` instead. Also update Black and other pre-commit hooks and re-format with Black. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [ ] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #7536 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7536 Pull-request-sha: b3aedf5570d7e0ba6c354e5989835260d0591b08 Change-Id: I8be85636fd2c9449b07a8626050c8bd35bd119d5
* | | implement python_impl to custom_op for basic ORM evaluator extensibilityMike Bayer2022-01-041-5/+65
| |/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | Added new parameter :paramref:`_sql.Operators.op.python_impl`, available from :meth:`_sql.Operators.op` and also when using the :class:`_sql.Operators.custom_op` constructor directly, which allows an in-Python evaluation function to be provided along with the custom SQL operator. This evaluation function becomes the implementation used when the operator object is used given plain Python objects as operands on both sides, and in particular is compatible with the ``synchronize_session='evaluate'`` option used with :ref:`orm_expression_update_delete`. Fixes: #3162 Change-Id: If46ba6a0e303e2180a177ba418a8cafe9b42608e
* | remove 2.0-removed Query elementsMike Bayer2022-01-013-31/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | * :meth:`_orm.Query.join` no longer accepts the "aliased" and "from_joinpoint" arguments * :meth:`_orm.Query.join` no longer accepts chains of multiple join targets in one method call. * ``Query.from_self()`` and ``Query.with_polymorphic()`` are removed. Change-Id: I534d04b53a538a4fc374966eb2bc8eb98a16497d References: #7257
* | Properly type _generative, decorator, public_factoryFederico Caselli2021-12-306-63/+218
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Good new is that pylance likes it and copies over the singature and everything. Bad news is that mypy does not support this yet https://github.com/python/mypy/issues/8645 Other minor bad news is that non_generative is not typed. I've tried using a protocol like the one in the comment but the signature is not ported over by pylance, so it's probably best to just live without it to have the correct signature. notes from mike: these three decorators are at the core of getting the library to be typed, more good news is that pylance will do all the things we like re: public_factory, see https://github.com/microsoft/pyright/issues/2758#issuecomment-1002788656 . For @_generative, we will likely move to using pep 673 once mypy supports it which may be soon. but overall having the explicit "return self" in the methods, while a little inconvenient, makes the typing more straightforward and locally present in the files rather than being decided at a distance. having "return self" present, or not, both have problems, so maybe we will be able to change it again if things change as far as decorator support. As it is, I feel like we are barely squeaking by with our decorators, the typing is already pretty out there. Change-Id: Ic77e13fc861def76a5925331df85c0aa48d77807 References: #6810
* | Merge "remove legacy select patterns" into mainmike bayer2021-12-295-451/+32
|\ \
| * | remove legacy select patternsMike Bayer2021-12-295-451/+32
| | | | | | | | | | | | | | | Change-Id: If6e521a1eb461e08748a0432943b938528a2619e References: #7257
* | | replace Variant with direct feature inside of TypeEngineMike Bayer2021-12-294-109/+103
|/ / | | | | | | | | | | | | | | | | | | | | | | | | The :meth:`_sqltypes.TypeEngine.with_variant` method now returns a copy of the original :class:`_sqltypes.TypeEngine` object, rather than wrapping it inside the ``Variant`` class, which is effectively removed (the import symbol remains for backwards compatibility with code that may be testing for this symbol). While the previous approach maintained in-Python behaviors, maintaining the original type allows for clearer type checking and debugging. Fixes: #6980 Change-Id: I158c7e56306b886b5b82b040205c428a5c4a242c
* | Replace raise_ with raise fromFederico Caselli2021-12-279-207/+130
|/ | | | | Change-Id: I7aaeb5bc130271624335b79cf586581d6c6c34c7 References: #4600
* Merge "factor out UnboundLoad and rearchitect strategy_options.py" into mainmike bayer2021-12-271-2/+4
|\
| * factor out UnboundLoad and rearchitect strategy_options.pyMike Bayer2021-12-271-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The architecture of Load is mostly rewritten here. The change includes removal of the "pluggable" aspect of the loader options, which would patch new methods onto Load. This has been replaced by normal methods that respond normally to typing annotations. As part of this change, the bake_loaders() and unbake_loaders() options, which have no effect since 1.4 and were unlikely to be in any common use, have been removed. Additionally, to support annotations for methods that make use of @decorator, @generative etc., modified format_argspec_plus to no longer return "args", instead returns "grouped_args" which is always grouped and allows return annotations to format correctly. Fixes: #6986 Change-Id: I6117c642345cdde65a64389bba6057ddd5374427
* | Merge "consider truediv as truediv; support floordiv operator" into mainmike bayer2021-12-274-5/+66
|\ \
| * | consider truediv as truediv; support floordiv operatorMike Bayer2021-12-264-5/+66
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implemented full support for "truediv" and "floordiv" using the "/" and "//" operators. A "truediv" operation between two expressions using :class:`_types.Integer` now considers the result to be :class:`_types.Numeric`, and the dialect-level compilation will cast the right operand to a numeric type on a dialect-specific basis to ensure truediv is achieved. For floordiv, conversion is also added for those databases that don't already do floordiv by default (MySQL, Oracle) and the ``FLOOR()`` function is rendered in this case, as well as for cases where the right operand is not an integer (needed for PostgreSQL, others). The change resolves issues both with inconsistent behavior of the division operator on different backends and also fixes an issue where integer division on Oracle would fail to be able to fetch a result due to inappropriate outputtypehandlers. Fixes: #4926 Change-Id: Id54cc018c1fb7a49dd3ce1216d68d40f43fe2659
* | Merge "implement cython for cache_anon_map, prefix_anon_map" into mainmike bayer2021-12-274-58/+77
|\ \ | |/ |/|
| * implement cython for cache_anon_map, prefix_anon_mapMike Bayer2021-12-214-58/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These are small bits where cache_anon_map in particular is part of the cache key generation scheme which is a key target for cython. changing such a tiny element of the cache key gen is doing basically nothing yet, as the cython impl is mostly the exact same speed as the python one. I guess for cython to be effective we'd need to redo the whole cache key generation and possibly not use the same kinds of structures, which might not be very easy to do. Additionally, some cython runtime import errors are being observed on jenkins, add an upfront check to the test suite to indicate if the expected build succeeded when REQUIRE_SQLALCHEMY_CEXT is set. Running case CacheAnonMap Running python .... Done Running cython .... Done | python | cython | cy / py | test_get_anon_non_present| 0.301266758 | 0.231203834 | 0.767438915 | test_get_anon_present| 0.300919362 | 0.227336695 | 0.755473803 | test_has_key_non_present| 0.152725077 | 0.133191719 | 0.872101171 | test_has_key_present| 0.152689778 | 0.133673095 | 0.875455428 | Running case PrefixAnonMap Running python .. Done Running cython .. Done | python | cython | cy / py | test_apply_non_present| 0.358715744 | 0.335245703 | 0.934572034 | test_apply_present | 0.354434996 | 0.338579782 | 0.955266229 | Change-Id: I0d3f1dd285c044afc234479141d831b2ee0455be
* | add recursion check for with_loader_criteria() optionMike Bayer2021-12-221-3/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | Fixed recursion overflow which could occur within ORM statement compilation when using either the :func:`_orm.with_loader_criteria` feature or the the :meth:`_orm.PropComparator.and_` method within a loader strategy in conjunction with a subquery which referred to the same entity being altered by the criteria option, or loaded by the loader strategy. A check for coming across the same loader criteria option in a recursive fashion has been added to accommodate for this scenario. Fixes: #7491 Change-Id: I8701332717c45a21948ea4788a3058c0fbbf03a7
* | accommodate for "clone" of ColumnClauseMike Bayer2021-12-213-3/+18
|/ | | | | | | | | | | | | | | | | | | | | | | | for use with the ClauseElement.params() method, altered ColumnClause._clone() so that while the element stays immutable, if the column is associated with a subquery, it returns a new version of itself as corresponding to a clone of the subquery. this allows processing functions to access the parameters in the subquery and produce a copy of it. The use case here is the expanded use of .params() within loader strategies that use HasCacheKey._apply_params_to_element(). Fixed issue in new "loader criteria" method :meth:`_orm.PropComparator.and_` where usage with a loader strategy like :func:`_orm.selectinload` against a column that was a member of the ``.c.`` collection of a subquery object, where the subquery would be dynamically added to the FROM clause of the statement, would be subject to stale parameter values within the subquery in the SQL statement cache, as the process used by the loader strategy to replace the parameters at execution time would fail to accommodate the subquery when received in this form. Fixes: #7489 Change-Id: Ibb3b6af140b8a62a2c8d05b2ac92e86ca3013c46
* Replace c extension with cython versions.workflow_test_cythonFederico Caselli2021-12-171-1/+1
| | | | | | | | | | | | | | | Re-implement c version immutabledict / processors / resultproxy / utils with cython. Performance is in general in par or better than the c version Added a collection module that has cython version of OrderedSet and IdentitySet Added a new test/perf file to compare the implementations. Run ``python test/perf/compiled_extensions.py all`` to execute the comparison test. See results here: https://docs.google.com/document/d/1nOcDGojHRtXEkuy4vNXcW_XOJd9gqKhSeALGG3kYr6A/edit?usp=sharing Fixes: #7256 Change-Id: I2930ef1894b5048210384728118e586e813f6a76 Signed-off-by: Federico Caselli <cfederico87@gmail.com>
* Merge "Removals: strings for join(), loader_options()." into mainmike bayer2021-12-101-25/+2
|\
| * Removals: strings for join(), loader_options().Mike Bayer2021-12-081-25/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * The :meth:`_orm.Query.join` method no longer accepts strings for relationship names; the long-documented approach of using ``Class.attrname`` for join targets is now standard. * Loader options no longer accept strings for attribute names. The long-documented approach of using ``Class.attrname`` for loader option targets is now standard. It is hoped that a subsequent commit can refactor loader options to no longer need "UnboundLoad" for most cases. Change-Id: If4629882c40523dccbf4459256bf540fb468b618 References: #6986
* | Fix typo in docstringLele Gaifax2021-12-091-1/+1
|/
* Merge "Warn when caching is disabled / document" into mainmike bayer2021-12-078-11/+147
|\
| * Warn when caching is disabled / documentMike Bayer2021-12-068-11/+147
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds new warnings for all elements that don't indicate their caching behavior, including user-defined ClauseElement subclasses and third party dialects. it additionally adds new documentation to discuss an apparent performance degradation in 1.4 when caching is disabled as a result in the significant expense incurred by ORM lazy loaders, which in 1.3 used BakedQuery so were actually cached. As a result of adding the warnings, a fair degree of lesser used SQL expression objects identified that they did not define caching behavior so would have been producing ``[no key]``, including PostgreSQL constructs ``hstore`` and ``array``. These have been amended to use inherit cache where appropriate. "on conflict" constructs in PostgreSQL, MySQL, SQLite still explicitly don't generate a cache key at this time. The change also adds a test for all constructs via assert_compile() to assert they will not generate cache warnings. Fixes: #7394 Change-Id: I85958affbb99bfad0f5efa21bc8f2a95e7e46981
* | fix typo in exists documentationFederico Caselli2021-12-051-1/+1
| | | | | | | | Change-Id: I44d8d6ee4816052a8dda64c00905b17b7ad8698e
* | The where method of exists now accepts multiple cluase.Federico Caselli2021-12-041-2/+8
|/ | | | | | | | | Support multiple clause elements in the :meth:`_sql.Exists.where` method, unifying the api with the on presented by a normal :func:`_sql.select` construct. Fixes: #7386 Change-Id: I5df20478008cd5167053d357cbfad8a641c62b44
* Removals: MetaData.bind, Table.bind, all other .bindMike Bayer2021-12-028-595/+52
| | | | | Change-Id: I1ef2eb2018f4b68825fe40a2a8d99084cf217b35 References: #7257
* provide connectionfairy on initializeMike Bayer2021-11-291-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is so that dialect methods that are called within init can assume the same argument structure as when they are called in other places; we can nail down the type of object as well. This change seems to mostly impact the isolation level routines in the dialects, as these are called during initialize() as well as on established connections. these methods can now assume a non-proxied DBAPI connection object in all cases, as it is commonly required that attributes like ".autocommit" are set on the object which don't work well in a proxied situation. Other changes: * adds an interface for the "connectionfairy" concept called PoolProxiedConnection. * Removes ``Connectable`` superclass of Connection. ``Connectable`` was originally meant to provide for the "method which accepts connection or engine" theme. As this pattern is greatly reduced in 2.0 and Engine no longer extends from it, the ``Connectable`` superclass doesnt serve any real purpose. Leading from that, to set this in I also applied pep 484 annotations to the Dialect base, and then in the interests of seeing some of the typing information show up in my IDE did a little bit for Engine, Connection and others. I hope that it's feasible that we can add annotations to specific classes and attributes ahead of when we actually try to mass-populate the whole library. This was the original spirit of pep-484 that we can apply annotations gradually. I do of course want to try to do a mass-populate although i think even in that case we will end up doing a lot of manual work anyway (in particular for the changes here which are distinct from what the stubs have). Fixes: #7122 Change-Id: I5dd7fbff8a7ae520a81c165091af12a6a68826db
* Added support for ``psycopg`` dialect.Federico Caselli2021-11-261-14/+28
| | | | | | | Both sync and async versions are supported. Fixes: #6842 Change-Id: I57751c5028acebfc6f9c43572562405453a2f2a4
* Merge "propose emulated setinputsizes embedded in the compiler" into mainmike bayer2021-11-252-53/+77
|\
| * propose emulated setinputsizes embedded in the compilerMike Bayer2021-11-232-53/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a new system so that PostgreSQL and other dialects have a reliable way to add casts to bound parameters in SQL statements, replacing previous use of setinputsizes() for PG dialects. rationale: 1. psycopg3 will be using the same SQLAlchemy-side "setinputsizes" as asyncpg, so we will be seeing a lot more of this 2. the full rendering that SQLAlchemy's compilation is performing is in the engine log as well as error messages. Without this, we introduce three levels of SQL rendering, the compiler, the hidden "setinputsizes" in SQLAlchemy, and then whatever the DBAPI driver does. With this new approach, users reporting bugs etc. will be less confused that there are as many as two separate layers of "hidden rendering"; SQLAlchemy's rendering is again fully transparent 3. calling upon a setinputsizes() method for every statement execution is expensive. this way, the work is done behind the caching layer 4. for "fast insertmany()", I also want there to be a fast approach towards setinputsizes. As it was, we were going to be taking a SQL INSERT with thousands of bound parameter placeholders and running a whole second pass on it to apply typecasts. this way, we will at least be able to build the SQL string once without a huge second pass over the whole string 5. psycopg2 can use this same system for its ARRAY casts 6. the general need for PostgreSQL to have lots of type casts is now mostly in the base PostgreSQL dialect and works independently of a DBAPI being present. dependence on DBAPI symbols that aren't complete / consistent / hashable is removed I was originally going to try to build this into bind_expression(), but it was revealed this worked poorly with custom bind_expression() as well as empty sets. the current impl also doesn't need to run a second expression pass over the POSTCOMPILE sections, which came out better than I originally thought it would. Change-Id: I363e6d593d059add7bcc6d1f6c3f91dd2e683c0c
* | Clean up most py3k compatFederico Caselli2021-11-2417-177/+121
|/ | | | Change-Id: I8172fdcc3103ff92aa049827728484c8779af6b7
* Support lightweight compiler column elements w/ slotsMike Bayer2021-11-226-93/+162
| | | | | | | | | | | | | | | | | | | | | | | | | | | | the _CompileLabel class included ``__slots__`` but these weren't used as the superclasses included slots. Create a ``__slots__`` superclass for ``ClauseElement``, creating a new class of compilable SQL elements that don't include heavier features like caching, annotations and cloning, which are meant to be used only in an ad-hoc compiler fashion. Create new ``CompilerColumnElement`` from that which serves in column-oriented contexts, but similarly does not include any expression operator support as it is intended to be used only to generate a string. Apply this to both ``_CompileLabel`` as well as PostgreSQL ``_ColonCast``, which does not actually subclass ``ColumnElement`` as this class has memoized attributes that aren't worth changing, and does not include SQL operator capabilities as these are not needed for these compiler-only objects. this allows us to more inexpensively add new ad-hoc labels / casts etc. at compile time, as we will be seeking to expand out the typecasts that are needed for PostgreSQL dialects in a subsequent patch. Change-Id: I52973ae3295cb6e2eb0d7adc816c678a626643ed
* Remove object in class definitionFederico Caselli2021-11-2218-49/+49
| | | | | References: #4600 Change-Id: I2a62ddfe00bc562720f0eae700a497495d7a987a
* Merge "generalize cache_ok to UserDefinedType" into mainmike bayer2021-11-191-58/+191
|\
| * generalize cache_ok to UserDefinedTypeMike Bayer2021-11-171-58/+191
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Extended the ``cache_ok`` flag and corresponding warning message if this flag is not defined, a behavior first established for :class:`.TypeDecorator` as part of :ticket:`6436`, to also take place for :class:`.UserDefinedType`, by generalizing the flag and associated caching logic to a new common base for these two types, :class:`.ExternalType`. The change means any current :class:`.UserDefinedType` will now cause SQL statement caching to no longer take place for statements which make use of the datatype, along with a warning being emitted, unless the class defines the :attr:`.UserDefinedType.cache_ok` flag as True. If the datatype cannot form a deterministic, hashable cache key derived from its arguments, it may return False which will continue to keep caching disabled but will suppress the warning. In particular, custom datatypes currently used in packages such as SQLAlchemy-utils will need to implement this flag. The issue was observed as a result of a SQLAlchemy-utils datatype that is not currently cacheable. Fixes: #7319 Change-Id: Ie0b5d4587df87bfe66d2fe7cd4585c3882584575
* | Merge "remove "native decimal" warning" into mainmike bayer2021-11-181-10/+0
|\ \
| * | remove "native decimal" warningMike Bayer2021-11-171-10/+0
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Removed the warning that emits from the :class:`_types.Numeric` type about DBAPIs not supporting Decimal values natively. This warning was oriented towards SQLite, which does not have any real way without additional extensions or workarounds of handling precision numeric values more than 15 significant digits as it only uses floating point math to represent numbers. As this is a known and documented limitation in SQLite itself, and not a quirk of the pysqlite driver, there's no need for SQLAlchemy to warn for this. The change does not otherwise modify how precision numerics are handled. Values can continue to be handled as ``Decimal()`` or ``float()`` as configured with the :class:`_types.Numeric`, :class:`_types.Float` , and related datatypes, just without the ability to maintain precision beyond 15 significant digits when using SQLite, unless alternate representations such as strings are used. Fixes: #7299 Change-Id: Ic570f8107177dec3ddbe94c7b43f40057b03276a
* | Merge "Add new sections regarding schemas and reflection" into mainmike bayer2021-11-182-3/+3
|\ \ | |/ |/|
| * Add new sections regarding schemas and reflectionjonathan vanasco2021-11-172-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * add a new section to reflection.rst `Schemas and Reflection`. * this contains some text from the ticket * migrate some text from `Specifying the Schema Name` to new section * migrate some text from PostgreSQL dialect to new section * target text is made more generic * cross-reference the postgres and new sections to one another, to avoid duplication of docs * update some docs 'meta' to 'metadata_obj' Fixes: #4387 Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Change-Id: I2b08672753fb2575d30ada07ead77587468fdade