summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/engine/default.py
Commit message (Collapse)AuthorAgeFilesLines
* flake8 refactor - engineMike Bayer2019-01-051-6/+6
| | | | | | | | | | | | A full rewrite of all imports and pep8 formatting using zimports, black, commits are broken into sections. Directories included in this commit: lib/sqlalchemy/*.py lib/sqlalchemy/engine/ lib/sqlalchemy/pool/ lib/sqlalchemy/event/ Change-Id: Idf6a7ba01bacd00fd9271fefd7e1bbab6f91f22b (cherry picked from commit e85ed21d2cdfcb215ff363592daecf9fe0d9e576)
* - do a broken zimports run on libMike Bayer2019-01-051-6/+14
|
* step 1, straight black -l 79 runMike Bayer2019-01-051-173/+231
| | | | includes adjustment to setup.py to recognize __version__ correctly
* Maintain compiled_params / replacement_expressions within expanding INMike Bayer2018-12-211-31/+44
| | | | | | | | | | Fixed issue in "expanding IN" feature where using the same bound parameter name more than once in a query would lead to a KeyError within the process of rewriting the parameters in the query. Fixes: #4394 Change-Id: Ibcadce9fefbcb060266d9447c2044ee6efeccf5a (cherry picked from commit c495769751e8b19d54fb92388ced587b5d13b85d)
* Add do_setinputsizes event for cx_OracleMike Bayer2018-06-271-16/+24
| | | | | | | | | | | | | | Added a new event currently used only by the cx_Oracle dialect, :meth:`.DialectEvents.setiputsizes`. The event passes a dictionary of :class:`.BindParameter` objects to DBAPI-specific type objects that will be passed, after conversion to parameter names, to the cx_Oracle ``cursor.setinputsizes()`` method. This allows both visibility into the setinputsizes process as well as the ability to alter the behavior of what datatypes are passed to this method. Change-Id: I43b97c8e3c840cad6f01edb274dc9cfed19cb5fc Fixes: #4290 (cherry picked from commit c270efdfb38a266ac042be2a0d11b6ff7e5ee619)
* render WITH clause after INSERT for INSERT..SELECT on Oracle, MySQLMike Bayer2018-06-181-0/+1
| | | | | | | | | | | | | | | | | Fixed INSERT FROM SELECT with CTEs for the Oracle and MySQL dialects, where the CTE was being placed above the entire statement as is typical with other databases, however Oracle and MariaDB 10.2 wants the CTE underneath the "INSERT" segment. Note that the Oracle and MySQL dialects don't yet work when a CTE is applied to a subquery inside of an UPDATE or DELETE statement, as the CTE is still applied to the top rather than inside the subquery. Also adds test suite support CTEs against backends. Change-Id: I8ac337104d5c546dd4f0cd305632ffb56ac8bf90 Fixes: #4275 Fixes: #4230 (cherry picked from commit 3619edcb8aa3ceef2a44925b85315fc0e90c5982)
* Turn oracle BINARY_DOUBLE, BINARY_FLOAT, DOUBLE_PRECISION into floatsMike Bayer2018-05-251-1/+1
| | | | | | | | | | | | | | | | | | The Oracle BINARY_FLOAT and BINARY_DOUBLE datatypes now participate within cx_Oracle.setinputsizes(), passing along NATIVE_FLOAT, so as to support the NaN value. Additionally, :class:`.oracle.BINARY_FLOAT`, :class:`.oracle.BINARY_DOUBLE` and :class:`.oracle.DOUBLE_PRECISION` now subclass :class:`.Float`, since these are floating point datatypes, not decimal. These datatypes were already defaulting the :paramref:`.Float.asdecimal` flag to False in line with what :class:`.Float` already does. Added reflection capabilities for the :class:`.oracle.BINARY_FLOAT`, :class:`.oracle.BINARY_DOUBLE` datatypes. Change-Id: Id99b912e83052654a17d07dc92b4dcb958cb7600 Fixes: #4264 (cherry picked from commit 28c7450b61beeb0bfb3d082cfcd12493c182e0ee)
* call setinputsizes() for integer typesMike Bayer2018-05-181-15/+18
| | | | | | | | | | | | | | | | Altered the Oracle dialect such that when an :class:`.Integer` type is in use, the cx_Oracle.NUMERIC type is set up for setinputsizes(). In SQLAlchemy 1.1 and earlier, cx_Oracle.NUMERIC was passed for all numeric types unconditionally, and in 1.2 this was removed to allow for better numeric precision. However, for integers, some database/client setups will fail to coerce boolean values True/False into integers which introduces regressive behavior when using SQLAlchemy 1.2. Overall, the setinputsizes logic seems like it will need a lot more flexibility going forward so this is a start for that. Change-Id: Ida80cc2c2c37ffc0e05da4b5df2dadfab55a01f2 Fixes: #4259 (cherry picked from commit c7ae04d1c5c4aa6c6099584ae386d6ab9ef7b290)
* SQL Server is not native boolean; add new flag for CHECK constraintMike Bayer2018-05-101-0/+1
| | | | | | | | | | | | | | | Fixed a 1.2 regression caused by :ticket:`4061` where the SQL Server "BIT" type would be considered to be "native boolean". The goal here was to avoid creating a CHECK constraint on the column, however the bigger issue is that the BIT value does not behave like a true/false constant and cannot be interpreted as a standalone expression, e.g. "WHERE <column>". The SQL Server dialect now goes back to being non-native boolean, but with an extra flag that still avoids creating the CHECK constraint. Change-Id: I4765d2a2a00b0d14f50282603cc4d48d4739dac1 Fixes: #4250 (cherry picked from commit bd1d6012dcbe5fbc6d1097a79d85b972b0d4fd8a)
* Allow bind processors to work with expanding INMike Bayer2018-02-231-1/+1
| | | | | | | | | Fixed bug in new "expanding IN parameter" feature where the bind parameter processors for values wasn't working at all, tests failed to cover this pretty basic case which includes that ENUM values weren't working. Change-Id: I8e2420d7229a3e253e43b5227ebb98f9fe0bd14a Fixes: #4198
* happy new yearMike Bayer2018-01-121-1/+1
| | | | Change-Id: I3ef36bfd0cb0ba62b3123c8cf92370a43156cf8f
* Fix regexp for expanding INMike Bayer2017-12-011-1/+1
| | | | | | | | | Fixed bug in new "expanding bind parameter" feature whereby if multiple params were used in one statement, the regular expression would not match the parameter name correctly. Change-Id: Ifaf7d627aac4ead2a13c8dddccb5c515253d88e6 Fixes: #4140
* Add multivalued insert context for defaultsreview/mike_bayer/ticket_4075Mike Bayer2017-09-161-3/+84
| | | | | | | | | | | | | | Added a new method :class:`.DefaultExecutionContext.current_parameters` which is used within a function-based default value generator in order to retrieve the current parameters being passed to the statement. The new function differs from the ``.current_parameters`` attribute in that it also provides for optional grouping of parameters that correspond to a multi-valued "insert" construct. Previously it was not possible to identify the subset of parameters that were relevant to the function call. Change-Id: I6894c7b4a2bce3e83c3ade8af0e5b2f8df37b785 Fixes: #4075
* Refactor for cx_Oracle version 6oracle_numericMike Bayer2017-09-111-4/+6
| | | | | | | | | Drops support for cx_Oracle prior to version 5.x, reworks numeric and binary support. Fixes: #4064 Change-Id: Ib9ae9aba430c15cd2a6eeb4e5e3fd8e97b5fe480
* Add new sane_rowcount_w_returning flagMike Bayer2017-08-311-0/+4
| | | | | | | | | | | | | | | | | Added a new class of "rowcount support" for dialects that is specific to when "RETURNING", which on SQL Server looks like "OUTPUT inserted", is in use, as the PyODBC backend isn't able to give us rowcount on an UPDATE or DELETE statement when OUTPUT is in effect. This primarily affects the ORM when a flush is updating a row that contains server-calcluated values, raising an error if the backend does not return the expected row count. PyODBC now states that it supports rowcount except if OUTPUT.inserted is present, which is taken into account by the ORM during a flush as to whether it will look for a rowcount. ORM tests are implicit in existing tests run against PyODBC Fixes: #4062 Change-Id: Iff17cbe4c7a5742971ed85a4d58660c18cc569c2
* Add new "expanding" feature to bindparam()Mike Bayer2017-04-071-3/+110
| | | | | | | | | | | | | | | | Added a new kind of :func:`.bindparam` called "expanding". This is for use in ``IN`` expressions where the list of elements is rendered into individual bound parameters at statement execution time, rather than at statement compilation time. This allows both a single bound parameter name to be linked to an IN expression of multiple elements, as well as allows query caching to be used with IN expressions. The new feature allows the related features of "select in" loading and "polymorphic in" loading to make use of the baked query extension to reduce call overhead. This feature should be considered to be **experimental** for 1.2. Fixes: #3953 Change-Id: Ie708414a3ab9c0af29998a2c7f239ff7633b1f6e
* ResultProxy won't autoclose connection until state flag is setMike Bayer2017-04-031-4/+4
| | | | | | | | | | | | | | Changed the mechanics of :class:`.ResultProxy` to unconditionally delay the "autoclose" step until the :class:`.Connection` is done with the object; in the case where Postgresql ON CONFLICT with RETURNING returns no rows, autoclose was occurring in this previously non-existent use case, causing the usual autocommit behavior that occurs unconditionally upon INSERT/UPDATE/DELETE to fail. Change-Id: I235a25daf4381b31f523331f810ea04450349722 Fixes: #3955 (cherry picked from commit 8ee363e4917b0dcd64a83b6d26e465c9e61e0ea5) (cherry picked from commit f52fb5282a046d26b6ee2778e03b995eb117c2ee)
* Apply type processing to untyped preexec default clauseMike Bayer2017-03-301-1/+5
| | | | | | | | | | | | | | | | Fixed bug where a SQL-oriented Python-side column default could fail to be executed properly upon INSERT in the "pre-execute" codepath, if the SQL itself were an untyped expression, such as plain text. The "pre- execute" codepath is fairly uncommon however can apply to non-integer primary key columns with SQL defaults when RETURNING is not used. Tests exist here to ensure typing is applied to a typed expression for default, but in the case of an untyped SQL value, we know the type from the column, so apply this. Change-Id: I5d8b391611c137b9f700115a50a2bf5b30abfe94 Fixes: #3923
* Integrate "pre-ping" into connection pool.Mike Bayer2017-03-201-0/+20
| | | | | | | | | | | | | | | Added native "pessimistic disconnection" handling to the :class:`.Pool` object. The new parameter :paramref:`.Pool.pre_ping`, available from the engine as :paramref:`.create_engine.pool_pre_ping`, applies an efficient form of the "pre-ping" recipe featured in the pooling documentation, which upon each connection check out, emits a simple statement, typically "SELECT 1", to test the connection for liveness. If the existing connection is no longer able to respond to commands, the connection is transparently recycled, and all other connections made prior to the current timestamp are invalidated. Change-Id: I89700d0075e60abd2250e54b9bd14daf03c71c00 Fixes: #3919
* New features from python 2.7Катаев Денис2017-03-171-3/+3
| | | | | | | After bump minimum supported version to 2.7 (1da9d3752160430c91534a8868ceb8c5ad1451d4), we can use new syntax. Change-Id: Ib064c75a00562e641d132f9c57e5e69744200e05 Pull-request: https://github.com/zzzeek/sqlalchemy/pull/347
* Implement comments for tables, columnsFrazer McLean2017-03-171-0/+2
| | | | | | | | | | | | | | Added support for SQL comments on :class:`.Table` and :class:`.Column` objects, via the new :paramref:`.Table.comment` and :paramref:`.Column.comment` arguments. The comments are included as part of DDL on table creation, either inline or via an appropriate ALTER statement, and are also reflected back within table reflection, as well as via the :class:`.Inspector`. Supported backends currently include MySQL, Postgresql, and Oracle. Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Fixes: #1546 Change-Id: Ib90683850805a2b4ee198e420dc294f32f15d35d
* Consult compiled paramstyle on execute_compiledMike Bayer2017-03-151-1/+1
| | | | | | | | | | | | Fixed bug where in the unusual case of passing a :class:`.Compiled` object directly to :meth:`.Connection.execute`, the dialect with which the :class:`.Compiled` object were generated was not consulted for the paramstyle of the string statement, instead assuming it would match the dialect-level paramstyle, causing mismatches to occur. Change-Id: I114e4db2183fbb75bb7c0b0641f5a161855696ee Fixes: #3938
* Add "empty in" strategies; default to "static"Mike Bayer2017-03-141-0/+12
| | | | | | | | | | | | | | | The longstanding behavior of the :meth:`.Operators.in_` and :meth:`.Operators.not_in_` operators emitting a warning when the right-hand condition is an empty sequence has been revised; a new flag :paramref:`.create_engine.empty_in_strategy` allows an empty "IN" expression to generate a simple boolean expression, or to invoke the previous behavior of dis-equating the expression to itself, with or without a warning. The default behavior is now to emit the simple boolean expression, allowing an empty IN to be evaulated without any performance penalty. Change-Id: I65cc37f2d7cf65a59bf217136c42fee446929352 Fixes: #3907
* update for 2017 copyrightMike Bayer2017-01-041-1/+1
| | | | Change-Id: I4e8c2aa8fe817bb2af8707410fa0201f938781de
* Add _extend_on deduplicating set for metadata.reflect()Mike Bayer2016-11-231-2/+3
| | | | | | | | | | | | | The "extend_existing" option of :class:`.Table` reflection would cause indexes and constraints to be doubled up in the case that the parameter were used with :meth:`.MetaData.reflect` (as the automap extension does) due to tables being reflected both within the foreign key path as well as directly. A new de-duplicating set is passed through within the :meth:`.MetaData.reflect` sequence to prevent double reflection in this way. Change-Id: Ibf6650c1e76a44ccbe15765fd79df2fa53d6bac7 Fixes: #3861
* Add support for server side cursors to mysqldb and pymysqlRoman Podoliaka2016-11-101-2/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This allows to skip buffering of the results on the client side, e.g. the following snippet: table = sa.Table( 'testtbl', sa.MetaData(), sa.Column('id', sa.Integer, primary_key=True), sa.Column('a', sa.Integer), sa.Column('b', sa.String(512)) ) table.create(eng, checkfirst=True) with eng.connect() as conn: result = conn.execute(table.select().limit(1)).fetchone() if result is None: for _ in range(1000): conn.execute( table.insert(), [{'a': random.randint(1, 100000), 'b': ''.join(random.choice(string.ascii_letters) for _ in range(100))} for _ in range(1000)] ) with eng.connect() as conn: for row in conn.execution_options(stream_results=True).execute(table.select()): pass now uses ~23 MB of memory instead of ~327 MB on CPython 3.5.2 and PyMySQL 0.7.9. psycopg2 implementation and execution options (stream_results, server_side_cursors) are reused. Change-Id: I4dc23ce3094f027bdff51b896b050361991c62e2
* spelling: Postgresql -> PostgreSQLVille Skyttä2016-10-081-1/+1
|
* Propagate execution_options at compile stageMike Bayer2016-10-051-2/+2
| | | | | | | | Compiler can now set up execution options and additionally will propagate autocommit from embedded CTEs. Change-Id: I19db7b8fe4d84549ea95342e8d2040189fed1bbe Fixes: #3805
* Check row for None with implicit returning PK to accommodate ON CONFLICTMike Bayer2016-10-041-1/+5
| | | | | | | | | | An adjustment to ON CONFLICT such that the "inserted_primary_key" logic is able to accommodate the case where there's no INSERT or UPDATE and there's no net change. The value comes out as None in this case, rather than failing on an exception. Change-Id: I0794e95c3ca262cb1ab2387167d96b8984225fce Fixes: #3813
* Handle BaseException in all _handle_dbapi_errorMike Bayer2016-09-211-2/+2
| | | | | | | | | | | | | | | | | Tests illustrate that exceptions like GreenletExit and even KeyboardInterrupt can corrupt the state of a DBAPI connection like that of pymysql and mysqlclient. Intercept BaseException errors within the handle_error scheme and invalidate just the connection alone in this case, but not the whole pool. The change is backwards-incompatible with a program that currently intercepts ctrl-C within a database transaction and wants to continue working on that transaction. Ensure the event hook can be used to reverse this behavior. Change-Id: Ifaa013c13826d123eef34e32b7e79fff74f1b21b Fixes: #3803
* Check for supports_execution at ClauseElement baseMike Bayer2016-08-311-2/+3
| | | | | | | | | | Raise a more descriptive exception / message when ClauseElement or non-SQLAlchemy objects that are not "executable" are erroneously passed to ``.execute()``; a new exception ObjectNotExecutableError is raised consistently in all cases. Change-Id: I2dd393121e2c7e5b6b9e40286a2f25670876e8e4 Fixes: #3786
* Work w/ prefetch even for selects, if presentMike Bayer2016-07-121-22/+37
| | | | | | | | | | | | | | | | | | | Fixed bug in new CTE feature for update/insert/delete stated as a CTE inside of an enclosing statement (typically SELECT) whereby oninsert and onupdate values weren't called upon for the embedded statement. This is accomplished by consulting prefetch for all statements. The collection is also broken into separate insert/update collections so that we don't need to consult toplevel self.isinsert to determine if the prefetch is for an insert or an update. What we don't yet test for are CTE combinations that have both insert/update in one statement, though these should now work in theory provided the underlying database supports such a statement. Change-Id: I3b6a860e22c86743c91c56a7ec751ff706f66f64 Fixes: #3745
* - happy new yearMike Bayer2016-01-291-1/+1
|
* - calling str() on a core sql construct has been made more "friendly",Mike Bayer2016-01-191-0/+17
| | | | | | | | | | when the construct contains non-standard sql elements such as returning, array index operations, or dialect-specific or custom datatypes. a string is now returned in these cases rendering an approximation of the construct (typically the postgresql-style version of it) rather than raising an error. fixes #3631 - add within_group to top-level imports - add eq_ignore_whitespace to sqlalchemy.testing imports
* - A deep improvement to the recently added :meth:`.TextClause.columns`Mike Bayer2016-01-141-1/+2
| | | | | | | | | | | | | | | | | | | method, and its interaction with result-row processing, now allows the columns passed to the method to be positionally matched with the result columns in the statement, rather than matching on name alone. The advantage to this includes that when linking a textual SQL statement to an ORM or Core table model, no system of labeling or de-duping of common column names needs to occur, which also means there's no need to worry about how label names match to ORM columns and so-forth. In addition, the :class:`.ResultProxy` has been further enhanced to map column and string keys to a row with greater precision in some cases. fixes #3501 - reorganize the initialization of ResultMetaData for readability and complexity; use the name "cursor_description", define the task of "merging" cursor_description with compiled column information as its own function, and also define "name extraction" as a separate task. - fully change the name we use in the "ambiguous column" error to be the actual name that was ambiguous, modify the C ext also
* - reorganize schema_translate_map to be succinct and gain the performanceMike Bayer2016-01-111-3/+7
| | | | back by using an attrgetter for the default case
* - Multi-tenancy schema translation for :class:`.Table` objects is added.Mike Bayer2016-01-081-0/+8
| | | | | | | | | This supports the use case of an application that uses the same set of :class:`.Table` objects in many schemas, such as schema-per-user. A new execution option :paramref:`.Connection.execution_options.schema_translate_map` is added. fixes #2685 - latest tox doesn't like the {posargs} in the profile rerunner
* - Fixed 1.0 regression where the eager fetch of cursor.rowcount wasMike Bayer2016-01-051-0/+3
| | | | | | | | no longer called for an UPDATE or DELETE statement emitted via plain text or via the :func:`.text` construct, affecting those drivers that erase cursor.rowcount once the cursor is closed such as SQL Server ODBC and Firebird drivers. fixes #3622
* - Fixed bug where known boolean values used byMike Bayer2015-05-261-6/+5
| | | | | | | | :func:`.engine_from_config` were not being parsed correctly; these included ``pool_threadlocal`` and the psycopg2 argument ``use_native_unicode``. fixes #3435 - add legacy_schema_aliasing config parsing for mssql - move use_native_unicode config arg to the psycopg2 dialect
* - Added support for the case of the misbehaving DBAPI that hasMike Bayer2015-05-151-0/+9
| | | | | | | | | | | pep-249 exception names linked to exception classes of an entirely different name, preventing SQLAlchemy's own exception wrapping from wrapping the error appropriately. The SQLAlchemy dialect in use needs to implement a new accessor :attr:`.DefaultDialect.dbapi_exception_translation_map` to support this feature; this is implemented now for the py-postgresql dialect. fixes #3421
* - Fixed a regression where the "last inserted id" mechanics wouldMike Bayer2015-04-081-12/+20
| | | | | | fail to store the correct value for MSSQL on an INSERT where the primary key value was present in the insert params before execution. fixes #3360
* - The "auto close" for :class:`.ResultProxy` is now a "soft" close.Mike Bayer2015-03-171-4/+4
| | | | | | | | | | | That is, after exhausing all rows using the fetch methods, the DBAPI cursor is released as before and the object may be safely discarded, but the fetch methods may continue to be called for which they will return an end-of-result object (None for fetchone, empty list for fetchmany and fetchall). Only if :meth:`.ResultProxy.close` is called explicitly will these methods raise the "result is closed" error. fixes #3330 fixes #3329
* - copyright 2015Mike Bayer2015-03-101-1/+1
|
* fooMike Bayer2015-03-081-12/+3
|
* - the change for #918 was of course not nearly that simple.Mike Bayer2015-03-071-1/+1
| | | | | | | | | | | | | | | | | | The "wrapping" employed by the mssql and oracle dialects using the "iswrapper" argument was not being used intelligently by the compiler, and the result map was being written incorrectly, using *more* columns in the result map than were actually returned by the statement, due to "row number" columns that are inside the subquery. The compiler now writes out result map on the "top level" select in all cases fully, and for the mssql/oracle wrapping case extracts out the "proxied" columns in a second step, which only includes those columns that are proxied outwards to the top level. This change might have implications for 3rd party dialects that might be imitating oracle's approach. They can safely continue to use the "iswrapper" kw which is now ignored, but they may need to also add the _select_wraps argument as well.
* - The SQL compiler now generates the mapping of expected columnspositional_targetingMike Bayer2015-03-071-2/+9
| | | | | | | | | | | | | | | | | such that they are matched to the received result set positionally, rather than by name. Originally, this was seen as a way to handle cases where we had columns returned with difficult-to-predict names, though in modern use that issue has been overcome by anonymous labeling. In this version, the approach basically reduces function call count per-result by a few dozen calls, or more for larger sets of result columns. The approach still degrades into a modern version of the old approach if textual elements modify the result map, or if any discrepancy in size exists between the compiled set of columns versus what was received, so there's no issue for partially or fully textual compilation scenarios where these lists might not line up. fixes #918 - callcounts still need to be adjusted down for this so zoomark tests won't pass at the moment
* - A warning is emitted if the ``isolation_level`` parameter is usedMike Bayer2015-02-041-0/+6
| | | | | | | | | | | | | | | | with :meth:`.Connection.execution_options` when a :class:`.Transaction` is in play; DBAPIs and/or SQLAlchemy dialects such as psycopg2, MySQLdb may implicitly rollback or commit the transaction, or not change the setting til next transaction, so this is never safe. - Added new parameter :paramref:`.Session.connection.execution_options` which may be used to set up execution options on a :class:`.Connection` when it is first checked out, before the transaction has begun. This is used to set up options such as isolation level on the connection before the transaction starts. - added new documentation section detailing best practices for setting transaction isolation with sessions. fixes #3296
* - remove context-specific post-crud logic from Connection and inline post-crudMike Bayer2015-01-251-51/+76
| | | | | | | | | | logic to some degree in DefaultExecutionContext. In particular we are removing post_insert() which doesn't appear to be used based on a survey of prominent third party dialects. Callcounts aren't added to existing execute profiling tests and inserts might be a little better. - simplify the execution_options join in DEC. Callcounts don't appear affected.
* - The multi-values version of :meth:`.Insert.values` has beenMike Bayer2015-01-131-4/+5
| | | | | | | | | | | | repaired to work more usefully with tables that have Python- side default values and/or functions, as well as server-side defaults. The feature will now work with a dialect that uses "positional" parameters; a Python callable will also be invoked individually for each row just as is the case with an "executemany" style invocation; a server- side default column will no longer implicitly receive the value explicitly specified for the first row, instead refusing to invoke without an explicit value. fixes #3288
* - tiny refactors #1-#5Mike Bayer2014-09-051-96/+115
|
View Lorry Controller Status