summaryrefslogtreecommitdiff
path: root/test/base
Commit message (Collapse)AuthorAgeFilesLines
* introduce deferred lambdasMike Bayer2020-07-031-0/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The coercions system allows us to add in lambdas as arguments to Core and ORM elements without changing them at all. By allowing the lambda to produce a deterministic cache key where we can also cheat and yank out literal parameters means we can move towards having 90% of "baked" functionality in a clearer way right in Core / ORM. As a second step, we can have whole statements inside the lambda, and can then add generation with __add__(), so then we have 100% of "baked" functionality with full support of ad-hoc literal values. Adds some more short_selects tests for the moment for comparison. Other tweaks inside cache key generation as we're trying to approach a certain level of performance such that we can remove the use of "baked" from the loader strategies. As we have not yet closed #4639, however the caching feature has been fully integrated as of b0cfa7379cf8513a821a3dbe3028c4965d9f85bd, we will also add complete caching documentation here and close that issue as well. Closes: #4639 Fixes: #5380 Change-Id: If91f61527236fd4d7ae3cad1f24c38be921c90ba
* Fix a wide variety of typos and broken linksaplatkouski2020-06-251-2/+2
| | | | | | | | | | | | Note the PR has a few remaining doc linking issues listed in the comment that must be addressed separately. Signed-off-by: aplatkouski <5857672+aplatkouski@users.noreply.github.com> Closes: #5371 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5371 Pull-request-sha: 7e7d233cf3a0c66980c27db0fcdb3c7d93bc2510 Change-Id: I9c36e8d8804483950db4b42c38ee456e384c59e3
* Add version token to error URLMike Bayer2020-06-111-9/+28
| | | | | | | | | | | | | the sqlalche.me redirector now supports the numerical version code in the URL, e.g. /13/, /14/, /20/, etc., so that we can redirect to the error codes for the appropriate version of SQLAlchemy in use without going through the catch-all "latest" link. If a particular version of the docs is no longer on the site, the redirect will revert to falling through the "latest" link (which ultimately lands on the current release version, /13/ at the time of this writing). Change-Id: I3bb463fd6fb6c8767c95a57f3699aba715a9a72d
* Convert bulk update/delete to new execution modelMike Bayer2020-06-061-1/+22
| | | | | | | | | | | | | | | This reorganizes the BulkUD model in sqlalchemy.orm.persistence to be based on the CompileState concept and to allow plain update() / delete() to be passed to session.execute() where the ORM synchronize session logic will take place. Also gets "synchronize_session='fetch'" working with horizontal sharding. Adding a few more result.scalar_one() types of methods as scalar_one() seems like what is normally desired. Fixes: #5160 Change-Id: I8001ebdad089da34119eb459709731ba6c0ba975
* callcount reductions and refinement for cached queriesMike Bayer2020-05-281-1/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit includes that we've removed the "_orm_query" attribute from compile state as well as query context. The attribute created reference cycles and also added method call overhead. As part of this change, the interface for ORMExecuteState changes a bit, as well as the interface for the horizontal sharding extension which now deprecates the "query_chooser" callable in favor of "execute_chooser", which receives the contextual object. This will also work more nicely when we implement the new execution path for bulk updates and deletes. Pre-merge execution options for statement, connection, arguments all up front in Connection. that way they can be passed to the before_execute / after_execute events, and the ExecutionContext doesn't have to merge as second time. Core execute is pretty close to 1.3 now. baked wasn't using the new one()/first()/one_or_none() methods, fixed that. Convert non-buffered cursor strategy to be a stateless singleton. inline all the paths by which the strategy gets chosen, oracle and SQL Server dialects make use of the already-invoked post_exec() hook to establish the alternate strategies, and this is actually much nicer than it was before. Add caching to mapper instance processor for getters. Identified a reference cycle per query that was showing up as a lot of gc cleanup, fixed that. After all that, performance not budging much. Even test_baked_query now runs with significantly fewer function calls than 1.3, still 40% slower. Basically something about the new patterns just makes this slower and while I've walked a whole bunch of them back, it hardly makes a dent. that said, the performance issues are relatively small, in the 20-40% time increase range, and the new caching feature does provide for regular ORM and Core queries that are cached, and they are faster than non-cached. Change-Id: I7b0b0d8ca550c05f79e82f75cd8eff0bbfade053
* Convert execution to move through SessionMike Bayer2020-05-251-13/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch replaces the ORM execution flow with a single pathway through Session.execute() for all queries, including Core and ORM. Currently included is full support for ORM Query, Query.from_statement(), select(), as well as the baked query and horizontal shard systems. Initial changes have also been made to the dogpile caching example, which like baked query makes use of a new ORM-specific execution hook that replaces the use of both QueryEvents.before_compile() as well as Query._execute_and_instances() as the central ORM interception hooks. select() and Query() constructs alike can be passed to Session.execute() where they will return ORM results in a Results object. This API is currently used internally by Query. Full support for Session.execute()->results to behave in a fully 2.0 fashion will be in later changesets. bulk update/delete with ORM support will also be delivered via the update() and delete() constructs, however these have not yet been adapted to the new system and may follow in a subsequent update. Performance is also beginning to lag as of this commit and some previous ones. It is hoped that a few central functions such as the coercions functions can be rewritten in C to re-gain performance. Additionally, query caching is now available and some subsequent patches will attempt to cache more of the per-execution work from the ORM layer, e.g. column getters and adapters. This patch also contains initial "turn on" of the caching system enginewide via the query_cache_size parameter to create_engine(). Still defaulting at zero for "no caching". The caching system still needs adjustments in order to gain adequate performance. Change-Id: I047a7ebb26aa85dc01f6789fac2bff561dcd555d
* inline one_or_noneMike Bayer2020-05-241-0/+31
| | | | | | | Remove a bunch of unnecessary functions for this case. add test coverage to ensure uniqueness logic works. Change-Id: I2e6232c5667a3277b0ec8d7e47085a267f23d75f
* Avoid proxy functions in row functionsFederico Caselli2020-05-231-0/+10
| | | | | | | | | | | This streamlines a bit for non-C implementations, however also adds and tests behavioral contracts that mappings should not allow integer or slice access and should behave like a Python mapping in that it raises KeyError for an integer and TypeError for a slice. Py3/Py2/C/noC :) References: #5340 Change-Id: Id3cef452dc8a526b8371c90c5ca2bbb240b25c26
* Add immutabledict C codeMike Bayer2020-05-231-2/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Start trying to convert fundamental objects to C as we now rely on a fairly small core of things, and 1.4 is having problems with complexity added being slower than the performance gains we are trying to build in. immutabledict here does seem to bench as twice as fast as the Python one, see below. However, it does not appear to be used prominently enough to make any dent in the performance tests. at the very least it may provide us some more lift-and-copy code for more C extensions. import timeit from sqlalchemy.util._collections import not_immutabledict, immutabledict def run(dict_cls): for i in range(1000000): d1 = dict_cls({"x": 5, "y": 4}) d2 = d1.union({"x": 17, "new key": "some other value"}, None) assert list(d2) == ["x", "y", "new key"] print( timeit.timeit( "run(d)", "from __main__ import run, not_immutabledict as d", number=1 ) ) print( timeit.timeit( "run(d)", "from __main__ import run, immutabledict as d", number=1 ) ) output: python: 1.8799766399897635 C code: 0.8880784640205093 Change-Id: I29e7104dc21dcc7cdf895bf274003af2e219bf6d
* Performance fixes for new result setMike Bayer2020-05-211-0/+101
| | | | | | | | | | | A few small mistakes led to huge callcounts. Additionally, the warn-on-get behavior which is attempting to warn for deprecated access in SQLAlchemy 2.0 is very expensive; it's not clear if its feasible to have this warning or to somehow alter how it works. Fixes: #5340 Change-Id: I73bdd2d7b6f1b25cc0222accabd585cf761a5af4
* Disable "check unicode returns" under Python 3Mike Bayer2020-05-191-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | Disabled the "unicode returns" check that runs on dialect startup when running under Python 3, which for many years has occurred in order to test the current DBAPI's behavior for whether or not it returns Python Unicode or Py2K strings for the VARCHAR and NVARCHAR datatypes. The check still occurs by default under Python 2, however the mechanism to test the behavior will be removed in SQLAlchemy 2.0 when Python 2 support is also removed. This logic was very effective when it was needed, however now that Python 3 is standard, all DBAPIs are expected to return Python 3 strings for character datatypes. In the unlikely case that a third party DBAPI does not support this, the conversion logic within :class:`.String` is still available and the third party dialect may specify this in its upfront dialect flags by setting the dialect level flag ``returns_unicode_strings`` to one of :attr:`.String.RETURNS_CONDITIONAL` or :attr:`.String.RETURNS_BYTES`, both of which will enable Unicode conversion even under Python 3. As part of this change, disabling testing of the doctest tutorials under Python 2. Fixes: #5315 Change-Id: I1260e894611409d3b7fe1a92bd90c52043bbcf19
* Ensure order in doctest of core/tutorial.rstFederico Caselli2020-05-111-0/+3
| | | | | | | Fix doctest error caused by the missing order by clause in the tests introduced in Ia1bbe3248b4f7c74fbee06fedf76a6ce94cd28a6 Change-Id: I55b45690879ebbaa57bc62765fcdab06f5e9c6f3
* Merge "Add warn_deprecated_limited feature"mike bayer2020-05-061-0/+36
|\
| * Add warn_deprecated_limited featureGord Thompson2020-05-011-0/+36
| | | | | | | | | | Fixes: #5268 Change-Id: I2f976048af4f8d6dd03a14efa31d179bd7324ba6
* | Don't apply sets or similar to objects in IdentitySetMike Bayer2020-05-031-4/+23
| | | | | | | | | | | | | | | | | | | | | | Modified the internal "identity set" implementation, which is a set that hashes objects on their id() rather than their hash values, to not actually call the ``__hash__()`` method of the objects, which are typically user-mapped objects. Some methods were calling this method as a side effect of the implementation. Fixes: #5304 Change-Id: I0ed8762f47622215a54dcad9f210377b1becf8e8
* | Propose Result as immediate replacement for ResultProxyMike Bayer2020-05-012-139/+790
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As progress is made on the _future.Result, including breaking it out such that DBAPI behaviors are local to specific implementations, it becomes apparent that the Result object is a functional superset of ResultProxy and that basic operations like fetchone(), fetchall(), and fetchmany() behave pretty much exactly the same way on the new object. Reorganize things so that ResultProxy is now referred to as LegacyCursorResult, which subclasses CursorResult that represents the DBAPI-cursor version of Result, making use of a multiple inheritance pattern so that the functionality of Result is also available in non-DBAPI contexts, as will be necessary for some ORM patterns. Additionally propose the composition system for Result that will form the basis for ORM-alternative result systems such as horizontal sharding and dogpile cache. As ORM results will soon be coming directly from instances of Result, these extensions will instead build their own ResultFetchStrategies that perform the special steps to create composed or cached result sets. Also considering at the moment not emitting deprecation warnings for fetchXYZ() methods; the immediate issue is Keystone tests are calling upon it, but as the implementations here are proving to be not in any kind of conflict with how Result works, there's not too much issue leaving them around and deprecating at some later point. References: #5087 References: #4395 Fixes: #4959 Change-Id: I8091919d45421e3f53029b8660427f844fee0228
* Rename py.test to pytestGord Thompson2020-04-161-1/+1
| | | | Change-Id: I431e1ef41e26d490343204a75a5c097768749768
* Create initial 2.0 engine implementationMike Bayer2020-04-162-6/+44
| | | | | | | | | | | | | | | | | | | Implemented the SQLAlchemy 2 :func:`.future.create_engine` function which is used for forwards compatibility with SQLAlchemy 2. This engine features always-transactional behavior with autobegin. Allow execution options per statement execution. This includes that the before_execute() and after_execute() events now accept an additional dictionary with these options, empty if not passed; a legacy event decorator is added for backwards compatibility which now also emits a deprecation warning. Add some basic tests for execution, transactions, and the new result object. Build out on a new testing fixture that swaps in the future engine completely to start with. Change-Id: I70e7338bb3f0ce22d2f702537d94bb249bd9fb0a Fixes: #4644
* Fix typo in resultproxy.c and test compatibility with python 3.5Federico Caselli2020-03-281-0/+2
| | | | | | | | | | | - Fix typo in resultproxy.c that would error on windows. - add -Wundef to C flags when linux is detected so that undefined symbols emit a warning - a few adjustments for tests to succeed on python 3.5 - note minimum version still documented here as 3.4 but this should move to at least 3.5 if not 3.6 for SQLAlchemy 1.4 Change-Id: Ia93ee1cb5c52e51e72eb0a24c100421c5157d04b
* Improve the method ``__str__`` of :class:`ColumnCollection`Federico Caselli2020-03-241-0/+11
| | | | | | | | The change avoids confusing a :class:`ColumnCollection` with a python list since the previous string representation was the same. Fixes: #5191 Change-Id: Icdbc08f9991d31ce86372505e3614740eaee56e2
* Simplified module pre-loading strategy and made it linter friendlyFederico Caselli2020-03-071-0/+33
| | | | | | | | | | | | | | | | | Introduced a modules registry to register modules that should be lazily loaded in the package init. This ensures that they are in the system module cache, avoiding potential thread safety issues as when importing them directly in the function that uses them. The module registry is used to obtain these modules directly, ensuring that the all the lazily loaded modules are resolved at the proper time This replaces dependency_for decorator and the dependencies decorator logic, removing the need to pass the resolved modules as arguments of the decodated functions and removes possible errors caused by linters. Fixes: #4689 Fixes: #4656 Change-Id: I2e291eba4297867fc0ddb5d875b9f7af34751d01
* Ensure all nested exception throws have a causeMike Bayer2020-03-021-10/+18
| | | | | | | | | | | | | | | Applied an explicit "cause" to most if not all internally raised exceptions that are raised from within an internal exception catch, to avoid misleading stacktraces that suggest an error within the handling of an exception. While it would be preferable to suppress the internally caught exception in the way that the ``__suppress_context__`` attribute would, there does not as yet seem to be a way to do this without suppressing an enclosing user constructed context, so for now it exposes the internally caught exception as the cause so that full information about the context of the error is maintained. Fixes: #4849 Change-Id: I55a86b29023675d9e5e49bc7edc5a2dc0bcd4751
* Result initial introductionMike Bayer2020-02-211-18/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This builds on cc718cccc0bf8a01abdf4068c7ea4f3 which moved RowProxy to Row, allowing Row to be more like a named tuple. - KeyedTuple in ORM is replaced with Row - ResultSetMetaData broken out into "simple" and "cursor" versions for ORM and Core, as well as LegacyCursor version. - Row now has _mapping attribute that supplies full mapping behavior. Row and SimpleRow both have named tuple behavior otherwise. LegacyRow has some mapping features on the tuple which emit deprecation warnings (e.g. keys(), values(), etc). the biggest change for mapping->tuple is the behavior of __contains__ which moves from testing of "key in row" to "value in row". - ResultProxy breaks into ResultProxy and FutureResult (interim), the latter has the newer APIs. Made available to dialects using execution options. - internal reflection methods and most tests move off of implicit Row mapping behavior and move to row._mapping, result.mappings() method using future result - a new strategy system for cursor handling replaces the various subclasses of RowProxy - some execution context adjustments. We will leave EC in but refined things like get_result_proxy() and out parameter handling. Dialects for 1.4 will need to adjust from get_result_proxy() to get_result_cursor_strategy(), if they are using this method - out parameter handling now accommodated by get_out_parameter_values() EC method. Oracle changes for this. external dialect for DB2 for example will also need to adjust for this. - deprecate case_insensitive flag for engine / result, this feature is not used mapping-methods on Row are deprecated, and replaced with Row._mapping.<meth>, including: row.keys() -> use row._mapping.keys() row.items() -> use row._mapping.items() row.values() -> use row._mapping.values() key in row -> use key in row._mapping int in row -> use int < len(row) Fixes: #4710 Fixes: #4878 Change-Id: Ieb9085e9bcff564359095b754da9ae0af55679f0
* Query linter optionAlessio Bogon2020-01-221-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added "from linting" as a built-in feature to the SQL compiler. This allows the compiler to maintain graph of all the FROM clauses in a particular SELECT statement, linked by criteria in either the WHERE or in JOIN clauses that link these FROM clauses together. If any two FROM clauses have no path between them, a warning is emitted that the query may be producing a cartesian product. As the Core expression language as well as the ORM are built on an "implicit FROMs" model where a particular FROM clause is automatically added if any part of the query refers to it, it is easy for this to happen inadvertently and it is hoped that the new feature helps with this issue. The original recipe is from: https://github.com/sqlalchemy/sqlalchemy/wiki/FromLinter The linter is now enabled for all tests in the test suite as well. This has necessitated that a lot of the queries be adjusted to not include cartesian products. Part of the rationale for the linter to not be enabled for statement compilation only was to reduce the need for adjustment for the many test case statements throughout the test suite that are not real-world statements. This gerrit is adapted from Ib5946e57c9dba6da428c4d1dee6760b3e978dda0. Fixes: #4737 Change-Id: Ic91fd9774379f895d021c3ad564db6062299211c Closes: #4830 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4830 Pull-request-sha: f8a21aa6262d1bcc9ff0d11a2616e41fba97a47a
* Remove jython code, remove all jython / pypy symbolsMike Bayer2020-01-171-12/+5
| | | | | | | | | | | | | | | | | | | | | | Removed all dialect code related to support for Jython and zxJDBC. Jython has not been supported by SQLAlchemy for many years and it is not expected that the current zxJDBC code is at all functional; for the moment it just takes up space and adds confusion by showing up in documentation. At the moment, it appears that Jython has achieved Python 2.7 support in its releases but not Python 3. If Jython were to be supported again, the form it should take is against the Python 3 version of Jython, and the various zxJDBC stubs for various backends should be implemented as a third party dialect. Additionally modernized logic that distinguishes between "cpython" and "pypy" to instead look at platform.python_distribution() which reliably tells us if we are cPython or not; all booleans which previously checked for pypy and sometimes jython are now converted to be "not cpython", this impacts the test suite for tests that are cPython centric. Fixes: #5094 Change-Id: I226cb55827f997daf6b4f4a755c18e7f4eb8d9ad
* Test for short term reference cycles and resolve as many as possibleMike Bayer2019-12-301-13/+0
| | | | | | | | Added test support and repaired a wide variety of unnecessary reference cycles created for short-lived objects, mostly in the area of ORM queries. Fixes: #5056 Change-Id: Ifd93856eba550483f95f9ae63d49f36ab068b85a
* Fix WeakSequence circular referenceCarson Ip2019-12-271-0/+13
| | | | | | | | | | | | | | Fixed a reference cycle which could impact the GC behavior of the :class:`.WeakSequence` object, currently used within one place in certain mapper configurations. The issue only affects configuration-time structures. Pull request courtesy Carson Ip. Fixes: #5050 Closes: #5051 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5051 Pull-request-sha: db672f45f4f139722edd2dcc6b0c19892725c9de Change-Id: I72673a33e655c44b68283ec1a2d7358b904e90ae
* Skip sub-minute timezone tests for Python 3 < 3.7Mike Bayer2019-12-021-1/+6
| | | | | | | | | The datetime.timezone class as of Python 3.7, or SQLAlchemy's port for Python 2, supports seconds and microseconds. For Python 3.6 and earlier, it is not supported. Fixes: #5016 Change-Id: Ia347d5a9958c0fa53317b2e340162e2009667bc2
* Add DATETIMEOFFSET support for mssql+pyodbcGord Thompson2019-11-291-0/+62
| | | | | | | | | | | | | | | Repaired support for the :class:`.mssql.DATETIMEOFFSET` datatype on PyODBC, by adding PyODBC-level result handlers as it does not include native support for this datatype. This includes usage of the Python 3 "timezone" tzinfo subclass in order to set up a timezone, which on Python 2 makes use of a minimal backport of "timezone" in sqlalchemy.util. Fixes: #4983 Closes: #4986 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4986 Pull-request-sha: e3a5a8dda01c7580dc93271092c4f9beaac4be11 Change-Id: Ia4c7c0d1971c6c0492515bd5fa0b1799f07fee1c
* pass executemany context to _repr_paramsMike Bayer2019-10-111-0/+2
| | | | | | | | | | | | | | | | | | Fixed bug where parameter repr as used in logging and error reporting needs additional context in order to distinguish between a list of parameters for a single statement and a list of parameter lists, as the "list of lists" structure could also indicate a single parameter list where the first parameter itself is a list, such as for an array parameter. The engine/connection now passes in an additional boolean indicating how the parameters should be considered. The only SQLAlchemy backend that expects arrays as parameters is that of psycopg2 which uses pyformat parameters, so this issue has not been too apparent, however as other drivers that use positional gain more features it is important that this be supported. It also eliminates the need for the parameter repr function to guess based on the parameter structure passed. Fixes: #4902 Change-Id: I086246ee0eb51484adbefd83e07295fa56576c5f
* Add new "exec_once_unless_exception" system; apply to dialect.initializeMike Bayer2019-08-181-0/+136
| | | | | | | | | | | | | | | | | | Fixed an issue whereby if the dialect "initialize" process which occurs on first connect would encounter an unexpected exception, the initialize process would fail to complete and then no longer attempt on subsequent connection attempts, leaving the dialect in an un-initialized, or partially initialized state, within the scope of parameters that need to be established based on inspection of a live connection. The "invoke once" logic in the event system has been reworked to accommodate for this occurrence using new, private API features that establish an "exec once" hook that will continue to allow the initializer to fire off on subsequent connections, until it completes without raising an exception. This does not impact the behavior of the existing ``once=True`` flag within the event system. Fixes: #4807 Change-Id: Iec32999b61b6af4b38b6719e0c2651454619078c
* Rewrite pool reset_on_return parsing using a util functionMike Bayer2019-08-111-0/+63
| | | | | | | | Choosing a util.symbol() based on a user parameter is about to have another use case added as part of #4623, so add a generalized solution ahead of it. Change-Id: I420631f81af2ffc655995b9cce9ff2ac618c16d7
* Strong reference listen function wrapped by "once"Mike Bayer2019-08-041-0/+27
| | | | | | | | | | | | | | Fixed issue in event system where using the ``once=True`` flag with dynamically generated listener functions would cause event registration of future events to fail if those listener functions were garbage collected after they were used, due to an assumption that a listened function is strongly referenced. The "once" wrapped is now modified to strongly reference the inner function persistently, and documentation is updated that using "once" does not imply automatic de-registration of listener functions. Fixes: #4794 Change-Id: I06388083d1633dcc89e8919eb1e51270f966df38
* Allow duplicate columns in from clauses and selectablesMike Bayer2019-07-111-112/+447
| | | | | | | | | | | | | | | | | | The :func:`.select` construct and related constructs now allow for duplication of column labels and columns themselves in the columns clause, mirroring exactly how column expressions were passed in. This allows the tuples returned by an executed result to match what was SELECTed for in the first place, which is how the ORM :class:`.Query` works, so this establishes better cross-compatibility between the two constructs. Additionally, it allows column-positioning-sensitive structures such as UNIONs (i.e. :class:`.CompoundSelect`) to be more intuitively constructed in those cases where a particular column might appear in more than one place. To support this change, the :class:`.ColumnCollection` has been revised to support duplicate columns as well as to allow integer index access. Fixes: #4753 Change-Id: Ie09a8116f05c367995c1e43623c51e07971d3bf0
* SelectBase no longer a FromClauseMike Bayer2019-07-061-0/+24
| | | | | | | | | | | | | | | | | | | | As part of the SQLAlchemy 2.0 migration project, a conceptual change has been made to the role of the :class:`.SelectBase` class hierarchy, which is the root of all "SELECT" statement constructs, in that they no longer serve directly as FROM clauses, that is, they no longer subclass :class:`.FromClause`. For end users, the change mostly means that any placement of a :func:`.select` construct in the FROM clause of another :func:`.select` requires first that it be wrapped in a subquery first, which historically is through the use of the :meth:`.SelectBase.alias` method, and is now also available through the use of :meth:`.SelectBase.subquery`. This was usually a requirement in any case since several databases don't accept unnamed SELECT subqueries in their FROM clause in any case. See the documentation in this change for lots more detail. Fixes: #4617 Change-Id: I0f6174ee24b9a1a4529168e52e855e12abd60667
* Enable F841Mike Bayer2019-06-202-3/+3
| | | | | | | | | | | This is a very useful assertion which prevents unused variables from being set up allows code to be more readable and sometimes even more efficient. test suites seem to be where the most problems are and there do not seem to be documentation examples that are using this, or at least the linter is not taking effect within rst blocks. Change-Id: I2b3341d8dd14da34879d8425838e66a4b9f8e27d
* Add test to as_interface when no cls or methodsToni Karppi2019-02-211-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes #4511 ### Description The `as_interface` function requires that an argument be provided for at least one of the parameters `cls` or `methods`. There was no test that tested the case when neither of these arguments were provided. ### Checklist <!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once) --> This pull request is: - [ ] A documentation / typographical error fix - Good to go, no issue or tests are needed - [x] A short code fix - please include the issue number, and create an issue if none exists, which must include a complete example of the issue. one line code fixes without an issue and demonstration will not be accepted. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. one line code fixes without tests will not be accepted. - [ ] A new feature implementation - please include the issue number, and create an issue if none exists, which must include a complete example of how the feature would look. - Please include: `Fixes: #<issue number>` in the commit message - please include tests. **Have a nice day!** Closes: #4512 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4512 Pull-request-sha: 156f6af95ede730d2e1a9d2c935c830cf3cfc96e Change-Id: Ifa23dbe1dbe9a824fe680e4947b81c96a50c133f
* Include newlines in StatementError formattingNate Clark2019-02-201-29/+51
| | | | | | | | | | | | | | | | Revised the formatting for :class:`.StatementError` when stringified. Each error detail is broken up over multiple newlines instead of spaced out on a single line. Additionally, the SQL representation now stringifies the SQL statement rather than using ``repr()``, so that newlines are rendered as is. Pull request courtesy Nate Clark. Fixes: #4500 Closes: #4501 Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4501 Pull-request-sha: 60cc0ee68dc96b8f483a60d37bcb26b6c6d53efe Change-Id: I79d8418b7495e5691c9a56f41e79495c26a967ff
* move to inspect_getfullargspecMike Bayer2019-01-151-31/+132
| | | | | | | | Replace inspect_getargspec with inspect_getfullargspec including a compatibility fallback for Py2k and use getfullargspec fully. Change-Id: I92bce0aafc37ce1a360b4f61b75f5892d0911c7e
* Leave bytestring exception messages as bytestringsMike Bayer2019-01-101-0/+46
| | | | | | | | | | | | | | | | | Fixed a regression introduced in version 1.2 where a refactor of the :class:`.SQLAlchemyError` base exception class introduced an inappropriate coercion of a plain string message into Unicode under python 2k, which is not handled by the Python interpreter for characters outside of the platform's encoding (typically ascii). The :class:`.SQLAlchemyError` class now passes a bytestring through under Py2K for ``__str__()`` as is the behavior of exception objects in general under Py2K, does a safe coercion to unicode utf-8 with backslash fallback for ``__unicode__()``. For Py3K the message is typically unicode already, but if not is again safe-coerced with utf-8 with backslash fallback for the ``__str__()`` method. Fixes: #4429 Change-Id: I2289da3f2c45c7d0041fa43d838958f7614defc3
* Post black reformattingMike Bayer2019-01-066-35/+55
| | | | | | | | | | | | | Applied on top of a pure run of black -l 79 in I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes resolves all remaining flake8 conditions for those codes we have enabled in setup.cfg. Included are resolutions for all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
* Run black -l 79 against all source filesMike Bayer2019-01-066-930/+1042
| | | | | | | | | | | | | | This is a straight reformat run using black as is, with no edits applied at all. The black run will format code consistently, however in some cases that are prevalent in SQLAlchemy code it produces too-long lines. The too-long lines will be resolved in the following commit that will resolve all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
* Handle PostgreSQL enums in remote schemasMike Bayer2018-12-211-0/+119
| | | | | | | | | | | | | | | | | | Fixed issue where a :class:`.postgresql.ENUM` or a custom domain present in a remote schema would not be recognized within column reflection if the name of the enum/domain or the name of the schema required quoting. A new parsing scheme now fully parses out quoted or non-quoted tokens including support for SQL-escaped quotes. Fixed issue where multiple :class:`.postgresql.ENUM` objects referred to by the same :class:`.MetaData` object would fail to be created if multiple objects had the same name under different schema names. The internal memoization the Postgresql dialect uses to track if it has created a particular :class:`.postgresql.ENUM` in the database during a DDL creation sequence now takes schema name into account. Fixes: #4416 Change-Id: I8cf03069e10b12f409e9b6796e24fc5850979955
* Mention Properties keys in __dir__Korn, Uwe2018-08-011-1/+7
| | | | | | | | | | | | | | The Python builtin ``dir()`` is now supported for a SQLAlchemy "properties" object, such as that of a Core columns collection (e.g. ``.c``), ``mapper.attrs``, etc. Allows iPython autocompletion to work as well. Pull request courtesy Uwe Korn. Change-Id: I8696729542d1b74a566642a3a63fd500f64588cd Pull-request: https://github.com/zzzeek/sqlalchemy/pull/458 Mention Properties keys in __dir__ Change-Id: I88939955857c8df5eed0b87bc27c45357780b17d
* Implement remove() for _empty_collectionMike Bayer2018-02-191-0/+58
| | | | | | | | | | | Fixed regression caused in 1.2.3 due to fix from :ticket:`4181` where the changes to the event system involving :class:`.Engine` and :class:`.OptionEngine` did not accommodate for event removals, which would raise an ``AttributeError`` when invoked at the class level. Change-Id: I1c9083829d74dd710716d28b0eaca4fa15e86313 Fixes: #4190
* Implement an error lookupMike Bayer2017-12-271-7/+69
| | | | | | | | Add codes to commonly raised error messages and classes that link back to fixed documentation sections giving background on these messages. Change-Id: I78d0660add7026bb662e20305a59283b20616954
* Ensure Oracle index w/ col DESC etc. is reflectedMike Bayer2017-08-141-1/+36
| | | | | | | | | | | | | | | | | | | | Fixed bug where an index reflected under Oracle with an expression like "column DESC" would not be returned, if the table also had no primary key, as a result of logic that attempts to filter out the index implicitly added by Oracle onto the primary key columns. Reworked the "filter out the primary key index" logic in oracle get_indexes() to be clearer. This changeset also adds an internal check to ColumnCollection to accomodate for the case of a column being added twice, as well as adding a private _table argument to Index such that reflection can specify the Table explicitly. The _table argument can become part of public API in a later revision or release if needed. Change-Id: I745711e03b3e450b7f31185fc70e10d3823063fa Fixes: #4042
* Make all tests to be PEP8 compliantKhairi Hafsham2017-02-075-107/+140
| | | | | | | | tested using pycodestyle version 2.2.0 Fixes: #3885 Change-Id: I5df43adc3aefe318f9eeab72a078247a548ec566 Pull-request: https://github.com/zzzeek/sqlalchemy/pull/343
* Check for __module__ not present in util.wrap_callable()Mike Bayer2016-10-141-0/+13
| | | | | | | | | The newly added wrap_callable() function assumes __module__ is present when this is not the case for objects such as functools.partial. Change-Id: Ia226260e9a65419e26d5c1f7187512f7fd4bb7c1 Fixes: #3823
* Repair pickling for Properties objectPieter Mulder2016-06-291-0/+35
| | | | | | | | | | | | | Fixed bug whereby the ``__getstate__`` / ``__setstate__`` methods for sqlalchemy.util.Properties were non-working due to the transition in the 1.0 series to ``__slots__``. The issue potentially impacted some third-party applications. Pull request courtesy Pieter Mulder. Fixes: #3728 Change-Id: I01ebd425bbfe145747fea2edd0d2d412c74fd84d Pull-request: https://github.com/zzzeek/sqlalchemy/pull/286 (cherry picked from commit cab57e9bab04fbdea44690c08dff379a29eaab32)
View Lorry Controller Status