summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/ext
Commit message (Collapse)AuthorAgeFilesLines
* Build out new declarative systems; deprecate mapper()Mike Bayer2020-09-106-2066/+499
| | | | | | | | | | | | | The ORM Declarative system is now unified into the ORM itself, with new import spaces under ``sqlalchemy.orm`` and new kinds of mappings. Support for decorator-based mappings without using a base class, support for classical style-mapper() calls that have access to the declarative class registry for relationships, and full integration of Declarative with 3rd party class attribute systems like ``dataclasses`` and ``attrs`` is now supported. Fixes: #5508 Change-Id: I130b2b6edff6450bfe8a3e6baa099ff04b5471ff
* Update select usage to use the new 1.4 formatFederico Caselli2020-09-082-5/+5
| | | | | | | | | | | | | | | | This change includes mainly that the bracketed use within select() is moved to positional, and keyword arguments are removed from calls to the select() function. it does not yet fully address other issues such as keyword arguments passed to the table.select(). Additionally, allows False / None to both be considered as "disable" for all of select.correlate(), select.correlate_except(), query.correlate(), which establishes consistency with passing of ``False`` for the legact select(correlate=False) argument. Change-Id: Ie6c6e6abfbd3d75d4c8de504c0cf0159e6999108
* Fix a mis-reference in create_async_engine().Fantix King2020-08-311-1/+1
| | | | | | | | | | | | | | `AsyncMethodRequired` is actually from `sqlalchemy.ext.asyncio.exc`, so here it should be referenced as `async_exc.AsyncMethodRequired`, instead of `exc.AsyncMethodRequired`. Fixes: #5529 Closes: #5545 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5545 Pull-request-sha: d8f885c587dd058f909d4f3bdbec3d0fca176680 Change-Id: I6886558bfd33d3e9e283fbd60c0ec971a1f22c0c
* See if the future is hereMike Bayer2020-08-281-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The docs are going to talk a lot about session.execute(select()) for ORM queries, and additionally it's much easier to help users with queries and such if we can use this new syntax. I'm hoping to see how hard it is to get a unified tutorial started that switches to new syntax. Basically, new syntax is much easier to explain and less buggy. But, if we are starting to present new syntax with the explicit goal of being easier to explain for less experienced programmers, the "future" thing is going to just be an impediment to that. See if we can remove "future" from session.execute(), so that ORM-enabled select() statements return ORM results at that level. This does not change the presence of the "future" flag for the Session's construction and for its transactional behaviors. The only perceptible change of the future flag for session.execute() is that session.execute(select()) where the statement has ORM entities in it now returns ORM new style tuples rather than old style tuples. Like mutating a URL, it's hopefully not very common that people are doing this. Change-Id: I0aa10322bb787d554d32772e3bc60548f1bf6206
* Raise NotImplemenedError for association proxy __clause_element__Mike Bayer2020-08-271-0/+6
| | | | | | | | | It's not possible right now to use an association proxy element as a plain column expression to be SELECTed from or used in a SQL function. An informative error is now raised when this occurs. Fixes: #5542 Change-Id: I334e767ebc0b56c1dccc4a1e5185b0435af77b93
* Make discriminator column used by ConcreteBase configurableJesse Bakker2020-08-171-6/+31
| | | | | | | | | | | | | | | The name of the virtual column used when using the :class:`_declarative.AbstractConcreteBase` and :class:`_declarative.ConcreteBase` classes can now be customized, to allow for models that have a column that is actually named ``type``. Pull request courtesy Jesse-Bakker. Fixes: #5513 Closes: #5514 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5514 Pull-request-sha: 5e7429f3531e2e22fffe996c9760905578d16ef9 Change-Id: I733737844d4f4e1f52dd2475a66c7044ff7292f5
* Implement rudimentary asyncio support w/ asyncpgMike Bayer2020-08-136-0/+1471
| | | | | | | | | | | | | | | | | | | | | | | | | | Using the approach introduced at https://gist.github.com/zzzeek/6287e28054d3baddc07fa21a7227904e We can now create asyncio endpoints that are then handled in "implicit IO" form within the majority of the Core internals. Then coroutines are re-exposed at the point at which we call into asyncpg methods. Patch includes: * asyncpg dialect * asyncio package * engine, result, ORM session classes * new test fixtures, tests * some work with pep-484 and a short plugin for the pyannotate package, which seems to have so-so results Change-Id: Idbcc0eff72c4cad572914acdd6f40ddb1aef1a7d Fixes: #3414
* Implement relationship AND criteria; global loader criteriaMike Bayer2020-08-051-3/+2
| | | | | | | | | | | | | | | | | | | Added the ability to add arbitrary criteria to the ON clause generated by a relationship attribute in a query, which applies to methods such as :meth:`_query.Query.join` as well as loader options like :func:`_orm.joinedload`. Additionally, a "global" version of the option allows limiting criteria to be applied to particular entities in a query globally. Documentation is minimal at this point, new examples will be coming in a subsequent commit. Some adjustments to execution options in how they are represented in the ORMExecuteState as well as well as a few ORM tests that forgot to get merged in a preceding commit. Fixes: #4472 Change-Id: I2b8fc57092dedf35ebd16f6343ad0f0d7d332beb
* Convert lazy loader, selectinload, load_on_ident to lambda statementsMike Bayer2020-08-052-4/+7
| | | | | | | | | Building on newly robust lambdas in I29a513c98917b1d503abfdd61e6b6e8800851aa8, convert key loading off of the "baked" system so that baked is no longer used by the ORM. Change-Id: I3abfb45dd6e50f84f29d39434caa0b550ce27864
* Convert remaining ORM APIs to support 2.0 styleMike Bayer2020-07-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is kind of a mixed bag of all kinds to help get us to 1.4 betas. The documentation stuff is a work in progress. Lots of other relatively small changes to APIs and things. More commits will follow to continue improving the documentation and transitioning to the 1.4/2.0 hybrid documentation. In particular some refinements to Session usage models so that it can match Engine's scoping / transactional patterns, and a decision to start moving away from "subtransactions" completely. * add select().from_statement() to produce FromStatement in an ORM context * begin referring to select() that has "plugins" for the few edge cases where select() will have ORM-only behaviors * convert dynamic.AppenderQuery to its own object that can use select(), though at the moment it uses Query to support legacy join calling forms. * custom query classes for AppenderQuery are replaced by do_orm_execute() hooks for custom actions, a separate gerrit will document this * add Session.get() to replace query.get() * Deprecate session.begin->subtransaction. propose within the test suite a hypothetical recipe for apps that rely on this pattern * introduce Session construction level context manager, sessionmaker context manager, rewrite the whole top of the session_transaction.rst documentation. Establish context manager patterns for Session that are identical to engine * ensure same begin_nested() / commit() behavior as engine * devise all new "join into an external transaction" recipe, add test support for it, add rules into Session so it just works, write new docs. need to ensure this doesn't break anything * vastly reduce the verbosity of lots of session docs as I dont think people read this stuff and it's difficult to keep current in any case * constructs like case(), with_only_columns() really need to move to *columns, add a coercion rule to just change these. * docs need changes everywhere I look. in_() is not in the Core tutorial? how do people even know about it? Remove tons of cruft from Select docs, etc. * build a system for common ORM options like populate_existing and autoflush to populate from execution options. * others? Change-Id: Ia4bea0f804250e54d90b3884cf8aab8b66b82ecf
* Merge "Add future=True to create_engine/Session; unify select()"mike bayer2020-07-081-1/+1
|\
| * Add future=True to create_engine/Session; unify select()Mike Bayer2020-07-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Several weeks of using the future_select() construct has led to the proposal there be just one select() construct again which features the new join() method, and otherwise accepts both the 1.x and 2.x argument styles. This would make migration simpler and reduce confusion. However, confusion may be increased by the fact that select().join() is different Current thinking is we may be better off with a few hard behavioral changes to old and relatively unknown APIs rather than trying to play both sides within two extremely similar but subtly different APIs. At the moment, the .join() thing seems to be the only behavioral change that occurs without the user taking any explicit steps. Session.execute() will still behave the old way as we are adding a future flag. This change also adds the "future" flag to Session() and session.execute(), so that interpretation of the incoming statement, as well as that the new style result is returned, does not occur for existing applications unless they add the use of this flag. The change in general is moving the "removed in 2.0" system further along where we want the test suite to fully pass even if the SQLALCHEMY_WARN_20 flag is set. Get many tests to pass when SQLALCHEMY_WARN_20 is set; this should be ongoing after this patch merges. Improve the RemovedIn20 warning; these are all deprecated "since" 1.4, so ensure that's what the messages read. Make sure the inforamtion link is on all warnings. Add deprecation warnings for parameters present and add warnings to all FromClause.select() types of methods. Fixes: #5379 Fixes: #5284 Change-Id: I765a0b912b3dcd0e995426427d8bb7997cbffd51 References: #5159
* | Add **kw support to DeclarativeMeta.__init__EwenGillies2020-07-061-1/+1
|/ | | | | | | | | | | | | Added a ``**kw`` argument to the :meth:`.DeclarativeMeta.__init__` method. This allows a class to support the :pep:`487` metaclass hook ``__init_subclass__``. Pull request courtesy Ewen Gillies. Fixes: #5357 Closes: #5363 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5363 Pull-request-sha: 0ad05a768316cba03a4d312ab39d3e8fbca7ac54 Change-Id: I1654befe9eb1c8b8e7fc0784bdbe64284614f0ea
* Remove _generate_path_cache_key()Mike Bayer2020-06-291-12/+7
| | | | | | | | | | loader options can now make a deterministic cache key based on the structure they are given, and this accommodates for aliased classes as well so that these cache keys are now "safe". Have baked query call upon the regular cache key method. Change-Id: Iaa2ef4064cfb16146f415ca73080f32003dd830d
* Fix a wide variety of typos and broken linksaplatkouski2020-06-256-19/+19
| | | | | | | | | | | | Note the PR has a few remaining doc linking issues listed in the comment that must be addressed separately. Signed-off-by: aplatkouski <5857672+aplatkouski@users.noreply.github.com> Closes: #5371 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5371 Pull-request-sha: 7e7d233cf3a0c66980c27db0fcdb3c7d93bc2510 Change-Id: I9c36e8d8804483950db4b42c38ee456e384c59e3
* Propose using RETURNING for bulk updates, deletesMike Bayer2020-06-231-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch makes several improvements in the area of bulk updates and deletes as well as the new session mechanics. RETURNING is now used for an UPDATE or DELETE statement emitted for a diaelct that supports "full returning" in order to satisfy the "fetch" strategy; this currently includes PostgreSQL and SQL Server. The Oracle dialect does not support RETURNING for more than one row, so a new dialect capability "full_returning" is added in addition to the existing "implicit_returning", indicating this dialect supports RETURNING for zero or more rows, not just a single identity row. The "fetch" strategy will gracefully degrade to the previous SELECT mechanics for dialects that do not support RETURNING. Additionally, the "fetch" strategy will attempt to use evaluation for the VALUES that were UPDATEd, rather than just expiring the updated attributes. Values should be evalutable in all cases where the value is not a SQL expression. The new approach also incurs some changes in the session.execute mechanics, where do_orm_execute() event handlers can now be chained to each return results; this is in turn used by the handler to detect on a per-bind basis if the fetch strategy needs to do a SELECT or if it can do RETURNING. A test suite is added to test_horizontal_shard that breaks up a single UPDATE or DELETE operation among multiple backends where some are SQLite and don't support RETURNING and others are PostgreSQL and do. The session event mechanics are corrected in terms of the "orm pre execute" hook, which now receives a flag "is_reentrant" so that the two ORM implementations for this can skip on their work if they are being called inside of ORMExecuteState.invoke(), where previously bulk update/delete were calling its SELECT a second time. In order for "fetch" to get the correct identity when called as pre-execute, it also requests the identity_token for each mapped instance which is now added as an optional capability of a SELECT for ORM columns. the identity_token that's placed by horizontal_sharding is now made available within each result row, so that even when fetching a merged result of plain rows we can tell which row belongs to which identity token. The evaluator that takes place within the ORM bulk update and delete for synchronize_session="evaluate" now supports the IN and NOT IN operators. Tuple IN is also supported. Fixes: #1653 Change-Id: I2292b56ae004b997cef0ba4d3fc350ae1dd5efc1
* Turn on caching everywhere, add loggingMike Bayer2020-06-101-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A variety of caching issues found by running all tests with statement caching turned on. The cache system now has a more conservative approach where any subclass of a SQL element will by default invalidate the cache key unless it adds the flag inherit_cache=True at the class level, or if it implements its own caching. Add working caching to a few elements that were omitted previously; fix some caching implementations to suit lesser used edge cases such as json casts and array slices. Refine the way BaseCursorResult and CursorMetaData interact with caching; to suit cases like Alembic modifying table structures, don't cache the cursor metadata if it were created against a cursor.description using non-positional matching, e.g. "select *". if a table re-ordered its columns or added/removed, now that data is obsolete. Additionally we have to adapt the cursor metadata _keymap regardless of if we just processed cursor.description, because if we ran against a cached SQLCompiler we won't have the right columns in _keymap. Other refinements to how and when we do this adaption as some weird cases were exposed in the Postgresql dialect, a text() construct that names just one column that is not actually in the statement. Fixed that also as it looks like a cut-and-paste artifact that doesn't actually affect anything. Various issues with re-use of compiled result maps and cursor metadata in conjunction with tables being changed, such as change in order of columns. mappers can be cleared but the class remains, meaning a mapper has to use itself as the cache key not the class. lots of bound parameter / literal issues, due to Alembic creating a straight subclass of bindparam that renders inline directly. While we can update Alembic to not do this, we have to assume other people might be doing this, so bindparam() implements the inherit_cache=True logic as well that was a bit involved. turn on cache stats in logging. Includes a fix to subqueryloader which moves all setup to the create_row_processor() phase and elminates any storage within the compiled context. This includes some changes to create_row_processor() signature and a revising of the technique used to determine if the loader can participate in polymorphic queries, which is also applied to selectinloading. DML update.values() and ordered_values() now coerces the keys as we have tests that pass an arbitrary class here which only includes __clause_element__(), so the key can't be cached unless it is coerced. this in turn changed how composite attributes support bulk update to use the standard approach of ClauseElement with annotations that are parsed in the ORM context. memory profiling successfully caught that the Session from Query was getting passed into _statement_20() so that was a big win for that test suite. Apparently Compiler had .execute() and .scalar() methods stuck on it, these date back to version 0.4 and there was a single test in the PostgreSQL dialect tests that exercised it for no apparent reason. Removed these methods as well as the concept of a Compiler holding onto a "bind". Fixes: #5386 Change-Id: I990b43aab96b42665af1b2187ad6020bee778784
* Convert bulk update/delete to new execution modelMike Bayer2020-06-062-73/+27
| | | | | | | | | | | | | | | This reorganizes the BulkUD model in sqlalchemy.orm.persistence to be based on the CompileState concept and to allow plain update() / delete() to be passed to session.execute() where the ORM synchronize session logic will take place. Also gets "synchronize_session='fetch'" working with horizontal sharding. Adding a few more result.scalar_one() types of methods as scalar_one() seems like what is normally desired. Fixes: #5160 Change-Id: I8001ebdad089da34119eb459709731ba6c0ba975
* Improve rendering of core statements w/ ORM elementsMike Bayer2020-05-312-6/+5
| | | | | | | | | | | | | | | | | | | | This patch contains a variety of ORM and expression layer tweaks to support ORM constructs in select() statements, without the 1.3.x requiremnt in Query that a full _compile_context() + new select() is needed in order to get a working statement object. Includes such tweaks as the ability to implement aliased class of an aliased class, as we are looking to fully support ACs against subqueries, as well as the ability to access anonymously-labeled ColumnProperty expressions within subqueries by naming the ".key" of the label after the property key. Some tuning to query.join() as well as ORMJoin internals to allow things to work more smoothly. Change-Id: Id810f485c5f7ed971529489b84694e02a3356d6d
* callcount reductions and refinement for cached queriesMike Bayer2020-05-282-52/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit includes that we've removed the "_orm_query" attribute from compile state as well as query context. The attribute created reference cycles and also added method call overhead. As part of this change, the interface for ORMExecuteState changes a bit, as well as the interface for the horizontal sharding extension which now deprecates the "query_chooser" callable in favor of "execute_chooser", which receives the contextual object. This will also work more nicely when we implement the new execution path for bulk updates and deletes. Pre-merge execution options for statement, connection, arguments all up front in Connection. that way they can be passed to the before_execute / after_execute events, and the ExecutionContext doesn't have to merge as second time. Core execute is pretty close to 1.3 now. baked wasn't using the new one()/first()/one_or_none() methods, fixed that. Convert non-buffered cursor strategy to be a stateless singleton. inline all the paths by which the strategy gets chosen, oracle and SQL Server dialects make use of the already-invoked post_exec() hook to establish the alternate strategies, and this is actually much nicer than it was before. Add caching to mapper instance processor for getters. Identified a reference cycle per query that was showing up as a lot of gc cleanup, fixed that. After all that, performance not budging much. Even test_baked_query now runs with significantly fewer function calls than 1.3, still 40% slower. Basically something about the new patterns just makes this slower and while I've walked a whole bunch of them back, it hardly makes a dent. that said, the performance issues are relatively small, in the 20-40% time increase range, and the new caching feature does provide for regular ORM and Core queries that are cached, and they are faster than non-cached. Change-Id: I7b0b0d8ca550c05f79e82f75cd8eff0bbfade053
* Revert 903b18828461bb8cb8dca4acc56809b3df2b14d5Mike Bayer2020-05-251-7/+30
| | | | | | This was accidentally pushed just now. Change-Id: I4da4151c4a81e5cf72146f8dcab3537301ccaae9
* Small callcount reductions and refinement for cached queriesMike Bayer2020-05-251-30/+7
| | | | | | | | | | | | | | | baked wasn't using the new one()/first()/one_or_none() methods, fixed that. loading._instance_processor() can skip setting up the quick populators every time because it can cache the getters. Callcounts have gone below what 1.3 does for the test_baked_query performance suite, however runtime for continued inexplicable reasons has not :(. still suspecting the result tuples but this seems so hard to believe. Change-Id: Ifbca04834d27350e0fa82cb8512e66112abc8729
* Convert execution to move through SessionMike Bayer2020-05-252-185/+141
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch replaces the ORM execution flow with a single pathway through Session.execute() for all queries, including Core and ORM. Currently included is full support for ORM Query, Query.from_statement(), select(), as well as the baked query and horizontal shard systems. Initial changes have also been made to the dogpile caching example, which like baked query makes use of a new ORM-specific execution hook that replaces the use of both QueryEvents.before_compile() as well as Query._execute_and_instances() as the central ORM interception hooks. select() and Query() constructs alike can be passed to Session.execute() where they will return ORM results in a Results object. This API is currently used internally by Query. Full support for Session.execute()->results to behave in a fully 2.0 fashion will be in later changesets. bulk update/delete with ORM support will also be delivered via the update() and delete() constructs, however these have not yet been adapted to the new system and may follow in a subsequent update. Performance is also beginning to lag as of this commit and some previous ones. It is hoped that a few central functions such as the coercions functions can be rewritten in C to re-gain performance. Additionally, query caching is now available and some subsequent patches will attempt to cache more of the per-execution work from the ORM layer, e.g. column getters and adapters. This patch also contains initial "turn on" of the caching system enginewide via the query_cache_size parameter to create_engine(). Still defaulting at zero for "no caching". The caching system still needs adjustments in order to gain adequate performance. Change-Id: I047a7ebb26aa85dc01f6789fac2bff561dcd555d
* Unify Query and select() , move all processing to compile phaseMike Bayer2020-05-243-47/+80
| | | | | | | | | | | | | | | | | | | | | | | | | | Convert Query to do virtually all compile state computation in the _compile_context() phase, and organize it all such that a plain select() construct may also be used as the source of information in order to generate ORM query state. This makes it such that Query is not needed except for its additional methods like from_self() which are all to be deprecated. The construction of ORM state will occur beyond the caching boundary when the new execution model is integrated. future select() gains a working join() and filter_by() method. as we continue to rebase and merge each commit in the steps, callcounts continue to bump around. will have to look at the final result when it's all in. References: #5159 References: #4705 References: #4639 References: #4871 References: #5010 Change-Id: I19e05b3424b07114cce6c439b05198ac47f7ac10
* Baked query needs to spoil fully on uncachable optionMike Bayer2020-05-041-2/+2
| | | | | | | | | | | | Fixed issue in the area of where loader options such as selectinload() interact with the baked query system, such that the caching of a query is not supposed to occur if the loader options themselves have elements such as with_polymorphic() objects in them that currently are not cache-compatible. The baked loader could sometimes not fully invalidate itself in these some of these scenarios leading to missed eager loads. Fixes: #5303 Change-Id: Iecf847204a619694d89297f83b63b613ef9767de
* Integrate new Result into ORM queryMike Bayer2020-05-022-11/+21
| | | | | | | | | | | | | | | | | | | | | | | | The next step in the 2.0 ORM changes is to have the ORM integrate with the new Result object fully. this patch uses Result to represent ORM objects rather than lists. public API to get at this Result is not added yet. dogpile.cache and horizontal sharding recipe/extensions have small adjustments to accommodate this change. Callcounts have fluctuated, some slightly better and some slightly worse. A few have gone up by a bit, however as the codebase is still in flux it is anticipated there will be some performance gains later on as ORM fetching is refined to no longer need to accommodate for extensive aliasing. The addition of caching will then change the entire story. References: #5087 References: #4395 Change-Id: If1a23824ffb77d8d58cf2338cf35dd6b5963b17f
* Documentation updates for ResultProxy -> ResultMike Bayer2020-05-011-3/+3
| | | | | | | | | This is based off of I8091919d45421e3f53029b8660427f844fee0228 and includes all documentation-only changes as a separate merge, once the parent is merged. Change-Id: I711adea23df0f9f0b1fe7c76210bd2de6d31842d
* Set up absolute references for create_engine and relatedMike Bayer2020-04-142-7/+12
| | | | | | | includes more replacements for create_engine(), Connection, disambiguation of Result from future/baked Change-Id: Icb60a79ee7a6c45ea9056c211ffd1be110da3b5e
* Run search and replace of symbolic module namesMike Bayer2020-04-1411-115/+151
| | | | | | | | Replaces a wide array of Sphinx-relative doc references with an abbreviated absolute form now supported by zzzeeksphinx. Change-Id: I94bffcc3f37885ffdde6238767224296339698a2
* Enable zzzeeksphinx module prefixesMike Bayer2020-04-142-5/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | zzzeeksphinx 1.1.2 in git can now convert short prefix names in a configured lookup to fully qualified module names, so that we can have succinct and portable pyrefs that still resolve absolutely. It also includes a formatter that will format all pyrefs in a fully consistent way regardless of the package path, by unconditionally removing all package tokens but always leaving class names in place including for methods, which means we no longer have to deal with tildes in pyrefs. The most immediate goal of the absolute prefixes is that we have lots of "ambiguous" names that appear in muliple places, like select(), ARRAY, ENUM etc. With the incoming future packages there is going to be lots of name overlap so it is necessary that all names eventually use absolute package paths when Sphinx receives them. In multiple stages, pyrefs will be converted using the zzzeeksphinx tools/fix_xrefs.py tool so that doclinks can be made absolute using symbolic prefixes. For this review, the actual search and replace of symbols is not performed, instead some general cleanup to prepare the docs as well as a lookup file used by the tool to do the conversion. this relatively small patch will be backported with appropriate changes to 1.3, 1.2, 1.1 and the tool can then be run on each branch individually. We are shooting for almost no warnings at all for master (still a handful I can't figure out which don't seem to have any impact) , very few for 1.3, and for 1.2 / 1.1 we hope for a significant reduction in warnings. Overall for all versions pyrefs should always point to the correct target, if they are in fact hyperlinked. it's better for a ref to go nowhere and be plain text than go to the wrong thing. Right now, hundreds of API links are pointing to the wrong thing as they are ambiguous names such as refresh(), insert(), update(), select(), join(), JSON etc. and Sphinx sends these all to essesntially random destinations among as many as five or six possible choices per symbol. A shorthand system that allows us to use absolute refs without having to type out a full blown absoulte module is the only way this is going to work, and we should ultimately seek to abandon any use of prefix dot for lookups. Everything should be on an underscore token so at the very least the module spaces can be reorganized without having to search and replace the entire documentation every time. Change-Id: I484a7329034af275fcdb322b62b6255dfeea9151
* Remove code deprecated before version 1.1Federico Caselli2020-04-093-35/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Remove deprecated method ``get_primary_keys` in the :class:`.Dialect` and :class:`.Inspector` classes. - Remove deprecated event ``dbapi_error`` and the method ``ConnectionEvents.dbapi_error`. - Remove support for deprecated engine URLs of the form ``postgres://``. - Remove deprecated dialect ``mysql+gaerdbms``. - Remove deprecated parameter ``quoting`` from :class:`.mysql.ENUM` and :class:`.mysql.SET` in the ``mysql`` dialect. - Remove deprecated function ``comparable_property``. and function ``comparable_using`` in the declarative extension. - Remove deprecated function ``compile_mappers``. - Remove deprecated method ``collection.linker``. - Remove deprecated method ``Session.prune`` and parameter ``Session.weak_identity_map``. This change also removes the class ``StrongInstanceDict``. - Remove deprecated parameter ``mapper.order_by``. - Remove deprecated parameter ``Session._enable_transaction_accounting`. - Remove deprecated parameter ``Session.is_modified.passive``. - Remove deprecated class ``Binary``. Please use :class:`.LargeBinary`. - Remove deprecated methods ``Compiled.compile``, ``ClauseElement.__and__`` and ``ClauseElement.__or__`` and attribute ``Over.func``. - Remove deprecated ``FromClause.count`` method. - Remove deprecated parameter ``Table.useexisting``. - Remove deprecated parameters ``text.bindparams`` and ``text.typemap``. - Remove boolean support for the ``passive`` parameter in ``get_history``. - Remove deprecated ``adapt_operator`` in ``UserDefinedType.Comparator``. Fixes: #4643 Change-Id: Idcd390c77bf7b0e9957907716993bdaa3f1a1763
* Use dot-separated name resolution for relationship targetMike Bayer2020-04-071-13/+43
| | | | | | | | | | | | | | | The string argument accepted as the first positional argument by the :func:`.relationship` function when using the Declarative API is no longer interpreted using the Python ``eval()`` function; instead, the name is dot separated and the names are looked up directly in the name resolution dictionary without treating the value as a Python expression. However, passing a string argument to the other :func:`.relationship` parameters that necessarily must accept Python expressions will still use ``eval()``; the documentation has been clarified to ensure that there is no ambiguity that this is in use. Fixes: #5238 Change-Id: Id802f403190adfab0ca034afe2214ba10fd9cfbb
* Add a third labeling mode for SELECT statementsMike Bayer2020-03-291-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Enhanced the disambiguating labels feature of the :func:`~.sql.expression.select` construct such that when a select statement is used in a subquery, repeated column names from different tables are now automatically labeled with a unique label name, without the need to use the full "apply_labels()" feature that conbines tablename plus column name. The disambigated labels are available as plain string keys in the .c collection of the subquery, and most importantly the feature allows an ORM :func:`.orm.aliased` construct against the combination of an entity and an arbitrary subquery to work correctly, targeting the correct columns despite same-named columns in the source tables, without the need for an "apply labels" warning. The existing labeling style is now called LABEL_STYLE_TABLENAME_PLUS_COL. This labeling style will remain used throughout the ORM as has been the case for over a decade, however, the new disambiguation scheme could theoretically replace this scheme entirely. The new scheme would dramatically alter how SQL looks when rendered from the ORM to be more succinct but arguably harder to read. The tablename_columnname scheme used by Join.c is unaffected here, as that's still hardcoded to that scheme. Fixes: #5221 Change-Id: Ib47d9e0f35046b3afc77bef6e65709b93d0c3026
* Simplified module pre-loading strategy and made it linter friendlyFederico Caselli2020-03-074-7/+7
| | | | | | | | | | | | | | | | | Introduced a modules registry to register modules that should be lazily loaded in the package init. This ensures that they are in the system module cache, avoiding potential thread safety issues as when importing them directly in the function that uses them. The module registry is used to obtain these modules directly, ensuring that the all the lazily loaded modules are resolved at the proper time This replaces dependency_for decorator and the dependencies decorator logic, removing the need to pass the resolved modules as arguments of the decodated functions and removes possible errors caused by linters. Fixes: #4689 Fixes: #4656 Change-Id: I2e291eba4297867fc0ddb5d875b9f7af34751d01
* Ensure all nested exception throws have a causeMike Bayer2020-03-025-31/+55
| | | | | | | | | | | | | | | Applied an explicit "cause" to most if not all internally raised exceptions that are raised from within an internal exception catch, to avoid misleading stacktraces that suggest an error within the handling of an exception. While it would be preferable to suppress the internally caught exception in the way that the ``__suppress_context__`` attribute would, there does not as yet seem to be a way to do this without suppressing an enclosing user constructed context, so for now it exposes the internally caught exception as the cause so that full information about the context of the error is maintained. Fixes: #4849 Change-Id: I55a86b29023675d9e5e49bc7edc5a2dc0bcd4751
* Remove print statement in favor of print() function in docs and examplesAlbert Tugushev2020-02-262-16/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | <!-- Provide a general summary of your proposed changes in the Title field above --> ### Description <!-- Describe your changes in detail --> Remove print statements ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [X] A documentation / typographical error fix - Good to go, no issue or tests are needed - [ ] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #5166 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5166 Pull-request-sha: 04a7394f71298322188f0861b4dfe93e5485839d Change-Id: Ib90a59fac929661a18748c6e44966fb87e3978c6
* Result initial introductionMike Bayer2020-02-211-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This builds on cc718cccc0bf8a01abdf4068c7ea4f3 which moved RowProxy to Row, allowing Row to be more like a named tuple. - KeyedTuple in ORM is replaced with Row - ResultSetMetaData broken out into "simple" and "cursor" versions for ORM and Core, as well as LegacyCursor version. - Row now has _mapping attribute that supplies full mapping behavior. Row and SimpleRow both have named tuple behavior otherwise. LegacyRow has some mapping features on the tuple which emit deprecation warnings (e.g. keys(), values(), etc). the biggest change for mapping->tuple is the behavior of __contains__ which moves from testing of "key in row" to "value in row". - ResultProxy breaks into ResultProxy and FutureResult (interim), the latter has the newer APIs. Made available to dialects using execution options. - internal reflection methods and most tests move off of implicit Row mapping behavior and move to row._mapping, result.mappings() method using future result - a new strategy system for cursor handling replaces the various subclasses of RowProxy - some execution context adjustments. We will leave EC in but refined things like get_result_proxy() and out parameter handling. Dialects for 1.4 will need to adjust from get_result_proxy() to get_result_cursor_strategy(), if they are using this method - out parameter handling now accommodated by get_out_parameter_values() EC method. Oracle changes for this. external dialect for DB2 for example will also need to adjust for this. - deprecate case_insensitive flag for engine / result, this feature is not used mapping-methods on Row are deprecated, and replaced with Row._mapping.<meth>, including: row.keys() -> use row._mapping.keys() row.items() -> use row._mapping.items() row.values() -> use row._mapping.values() key in row -> use key in row._mapping int in row -> use int < len(row) Fixes: #4710 Fixes: #4878 Change-Id: Ieb9085e9bcff564359095b754da9ae0af55679f0
* Add keywords to MutableList.sort()Mike Bayer2020-01-231-2/+2
| | | | | | | | Added keyword arguments to the :meth:`.MutableList.sort` function so that a key function as well as the "reverse" keyword argument can be provided. Fixes: #5114 Change-Id: Iefb29e1ccadfad6ecba558ce575029307001b88e
* Merge "Use context managers for threading.Lock()"mike bayer2020-01-032-8/+2
|\
| * Use context managers for threading.Lock()Heckad2020-01-032-8/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | (zzzeek:) For some reason I thought that threading.Lock() still did not support context managers, at least in Python 2, however this does not seem to be the case. Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Closes: #5069 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5069 Pull-request-sha: efeac06dda5afdbe33abcf9b27c8b5b5725c8444 Change-Id: Ic64fcd99cd587bc70b4ecc5b45d8205b5c76eff2
* | happy new yearMike Bayer2020-01-0115-15/+15
|/ | | | Change-Id: I08440dc25e40ea1ccea1778f6ee9e28a00808235
* Add anonymizing context to cache keys, comparison; convert traversalMike Bayer2019-11-042-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Created new visitor system called "internal traversal" that applies a data driven approach to the concept of a class that defines its own traversal steps, in contrast to the existing style of traversal now known as "external traversal" where the visitor class defines the traversal, i.e. the SQLCompiler. The internal traversal system now implements get_children(), _copy_internals(), compare() and _cache_key() for most Core elements. Core elements with special needs like Select still implement some of these methods directly however most of these methods are no longer explicitly implemented. The data-driven system is also applied to ORM elements that take part in SQL expressions so that these objects, like mappers, aliasedclass, query options, etc. can all participate in the cache key process. Still not considered is that this approach to defining traversibility will be used to create some kind of generic introspection system that works across Core / ORM. It's also not clear if real statement caching using the _cache_key() method is feasible, if it is shown that running _cache_key() is nearly as expensive as compiling in any case. Because it is data driven, it is more straightforward to optimize using inlined code, as is the case now, as well as potentially using C code to speed it up. In addition, the caching sytem now accommodates for anonymous name labels, which is essential so that constructs which have anonymous labels can be cacheable, that is, their position within a statement in relation to other anonymous names causes them to generate an integer counter relative to that construct which will be the same every time. Gathering of bound parameters from any cache key generation is also now required as there is no use case for a cache key that does not extract bound parameter values. Applies-to: #4639 Change-Id: I0660584def8627cad566719ee98d3be045db4b8d
* Don't cache a query that has before_compile modificationsMike Bayer2019-10-261-1/+11
| | | | | | | | | | | | | | | | | The :class:`.BakedQuery` will not cache a query that was modified by a :meth:`.QueryEvents.before_compile` event, so that compilation hooks that may be applying ad-hoc modifications to queries will take effect on each run. In particular this is helpful for events that modify queries used in lazy loading as well as eager loading such as "select in" loading. In order to re-enable caching for a query modified by this event, a new flag ``bake_ok`` is added; see :ref:`baked_with_before_compile` for details. A longer term plan to provide a new form of SQL caching should solve this kind of issue more comprehensively. Fixes: #4947 Change-Id: I5823c4fa00e7b6d46a2e8461b02d8b16605a6ed0
* Add result map targeting for custom compiled, text objectsMike Bayer2019-10-071-1/+20
| | | | | | | | | | | | | | | | In order for text(), custom compiled objects, etc. to be usable by Query(), they are all targeted by object key in the result map. As we no longer want Query to implicitly label these, as well as that text() has no label feature, support adding entries to the result map that have no name, key, or type, only the object itself, and then ensure that the compiler sets up for positional targeting when this condition is detected. Allows for more flexible ORM query usage with custom expressions and text() while having less special logic in query itself. Fixes: #4887 Change-Id: Ie073da127d292d43cb132a2b31bc90af88bfe2fd
* Move identity_lookup to sessionMike Bayer2019-09-271-34/+41
| | | | | | | | This performance critical method is on Query needlessly, just to appease the horizontal sharding API. Have the performance impact of invoking Query only incur if horizontal sharding is actually used. Change-Id: I03db2befe2f5614380258927a62ed389a6ba0fae
* Remove unnecessary util.callable usageSteven Loria2019-09-092-5/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes: #4850 <!-- Provide a general summary of your proposed changes in the Title field above --> ### Description <!-- Describe your changes in detail --> Removes usage of `util.callable`. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [x] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #4851 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4851 Pull-request-sha: a0ccdff2cb74f5e944d8baccc269c382b591c8e2 Change-Id: I79918f44becbc5dbefdc7ff65128695c1cabed1d
* Remove threadlocal strategy docs, remaining contextual_connectMike Bayer2019-08-051-1/+1
| | | | | | | | | | | | in dfb20f07d8, we merged the removal of the threadlocal strategy from the 1.4 branch. We forgot to take the docs out :). Also, there seems to be an uncovered _contextual_connect call in horizontal shard. Cover that and repair. Fixes: #4632 Closes: #4795 Change-Id: Id05cbbebe34a8f547c9c84369a929a2926c7d093
* Squashed commit of the following:Mike Bayer2019-07-051-1/+1
| | | | | | | | | | | | | | | | commit d4f3bedc74568b7ec543988ee2d43e64c5ace28f Author: Carson Ip <carsonip@users.noreply.github.com> Date: Fri Jul 5 11:20:12 2019 +0800 Fix typo in docstring commit a3e4b05744f51ec5d12a2fee1ad6093de904273e Author: Carson Ip <carsonip@users.noreply.github.com> Date: Fri Jul 5 11:14:57 2019 +0800 Fix typo in docstring Change-Id: Ifa2ebff5629bf970e5fac28bba64d501376cfae9
* Consult is_attrbute flag to determine descriptor; enable for assoc proxyMike Bayer2019-06-141-1/+1
| | | | | | | | | | | | Fixed bug where the :attr:`.Mapper.all_orm_descriptors` accessor would return an entry for the :class:`.Mapper` itself under the declarative ``__mapper___`` key, when this is not a descriptor. The ``.is_attribute`` flag that's present on all :class:`.InspectionAttr` objects is now consulted, which has also been modified to be ``True`` for an association proxy, as it was erroneously set to False for this object. Fixes: #4729 Change-Id: Ia02388cc25d004e32d337140b62a587f3e5a0b7b
* Add QueryableAttribute._impl_uses_objects accessor for AssociationProxyMike Bayer2019-05-191-6/+2
| | | | | | | | | | | | | | Fixed regression where new association proxy system was still not proxying hybrid attributes when they made use of the ``@hybrid_property.expression`` decorator to return an alternate SQL expression, or when the hybrid returned an arbitrary :class:`.PropComparator`, at the expression level. This involved futher generalization of the heuristics used to detect the type of object being proxied at the level of :class:`.QueryableAttribute`, to better detect if the descriptor ultimately serves mapped classes or column expressions. Fixes: #4690 Change-Id: I5b5300661291c94a23de53bcf92d747701720aa1
View Lorry Controller Status