summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/cextension
Commit message (Collapse)AuthorAgeFilesLines
* normalize execute style for events, 2.0Mike Bayer2020-08-201-6/+21
| | | | | | | | | | | | | | | The _execute_20 and exec_driver_sql methods should wrap up the parameters so that they represent the single list / single dictionary style of invocation into the legacy methods. then the before_ after_ execute event handlers should be receiving the parameter dictionary as a single dictionary. this requires that we break out distill_params to work differently if event handlers are present. additionally, add deprecation warnings for old argument passing styles. Change-Id: I97cb4d06adfcc6b889f10d01cc7775925cffb116
* Add future=True to create_engine/Session; unify select()Mike Bayer2020-07-081-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Several weeks of using the future_select() construct has led to the proposal there be just one select() construct again which features the new join() method, and otherwise accepts both the 1.x and 2.x argument styles. This would make migration simpler and reduce confusion. However, confusion may be increased by the fact that select().join() is different Current thinking is we may be better off with a few hard behavioral changes to old and relatively unknown APIs rather than trying to play both sides within two extremely similar but subtly different APIs. At the moment, the .join() thing seems to be the only behavioral change that occurs without the user taking any explicit steps. Session.execute() will still behave the old way as we are adding a future flag. This change also adds the "future" flag to Session() and session.execute(), so that interpretation of the incoming statement, as well as that the new style result is returned, does not occur for existing applications unless they add the use of this flag. The change in general is moving the "removed in 2.0" system further along where we want the test suite to fully pass even if the SQLALCHEMY_WARN_20 flag is set. Get many tests to pass when SQLALCHEMY_WARN_20 is set; this should be ongoing after this patch merges. Improve the RemovedIn20 warning; these are all deprecated "since" 1.4, so ensure that's what the messages read. Make sure the inforamtion link is on all warnings. Add deprecation warnings for parameters present and add warnings to all FromClause.select() types of methods. Fixes: #5379 Fixes: #5284 Change-Id: I765a0b912b3dcd0e995426427d8bb7997cbffd51 References: #5159
* Convert execution to move through SessionMike Bayer2020-05-251-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch replaces the ORM execution flow with a single pathway through Session.execute() for all queries, including Core and ORM. Currently included is full support for ORM Query, Query.from_statement(), select(), as well as the baked query and horizontal shard systems. Initial changes have also been made to the dogpile caching example, which like baked query makes use of a new ORM-specific execution hook that replaces the use of both QueryEvents.before_compile() as well as Query._execute_and_instances() as the central ORM interception hooks. select() and Query() constructs alike can be passed to Session.execute() where they will return ORM results in a Results object. This API is currently used internally by Query. Full support for Session.execute()->results to behave in a fully 2.0 fashion will be in later changesets. bulk update/delete with ORM support will also be delivered via the update() and delete() constructs, however these have not yet been adapted to the new system and may follow in a subsequent update. Performance is also beginning to lag as of this commit and some previous ones. It is hoped that a few central functions such as the coercions functions can be rewritten in C to re-gain performance. Additionally, query caching is now available and some subsequent patches will attempt to cache more of the per-execution work from the ORM layer, e.g. column getters and adapters. This patch also contains initial "turn on" of the caching system enginewide via the query_cache_size parameter to create_engine(). Still defaulting at zero for "no caching". The caching system still needs adjustments in order to gain adequate performance. Change-Id: I047a7ebb26aa85dc01f6789fac2bff561dcd555d
* Avoid proxy functions in row functionsFederico Caselli2020-05-231-5/+46
| | | | | | | | | | | This streamlines a bit for non-C implementations, however also adds and tests behavioral contracts that mappings should not allow integer or slice access and should behave like a Python mapping in that it raises KeyError for an integer and TypeError for a slice. Py3/Py2/C/noC :) References: #5340 Change-Id: Id3cef452dc8a526b8371c90c5ca2bbb240b25c26
* Merge "Add immutabledict C code"mike bayer2020-05-232-1/+476
|\
| * Add immutabledict C codeMike Bayer2020-05-233-2/+476
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Start trying to convert fundamental objects to C as we now rely on a fairly small core of things, and 1.4 is having problems with complexity added being slower than the performance gains we are trying to build in. immutabledict here does seem to bench as twice as fast as the Python one, see below. However, it does not appear to be used prominently enough to make any dent in the performance tests. at the very least it may provide us some more lift-and-copy code for more C extensions. import timeit from sqlalchemy.util._collections import not_immutabledict, immutabledict def run(dict_cls): for i in range(1000000): d1 = dict_cls({"x": 5, "y": 4}) d2 = d1.union({"x": 17, "new key": "some other value"}, None) assert list(d2) == ["x", "y", "new key"] print( timeit.timeit( "run(d)", "from __main__ import run, not_immutabledict as d", number=1 ) ) print( timeit.timeit( "run(d)", "from __main__ import run, immutabledict as d", number=1 ) ) output: python: 1.8799766399897635 C code: 0.8880784640205093 Change-Id: I29e7104dc21dcc7cdf895bf274003af2e219bf6d
* | Don't incref on new reference key_styleMike Bayer2020-05-221-1/+1
|/ | | | | | | | | | | in 4550983e0ce2f35b3585e53894c941c23693e71d we added a new attribute key_style. remove an erroneous Py_INCREF when we acquire it from PyLong_FromLong as we already own the reference. since this is a new reference we actualy need to Py_DECREF it because we aren't returning it. Change-Id: I61470513a173c76863ec6f7f5ff9b2ec13582f08
* Performance fixes for new result setMike Bayer2020-05-211-33/+104
| | | | | | | | | | | A few small mistakes led to huge callcounts. Additionally, the warn-on-get behavior which is attempting to warn for deprecated access in SQLAlchemy 2.0 is very expensive; it's not clear if its feasible to have this warning or to somehow alter how it works. Fixes: #5340 Change-Id: I73bdd2d7b6f1b25cc0222accabd585cf761a5af4
* Propose Result as immediate replacement for ResultProxyMike Bayer2020-05-011-20/+64
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As progress is made on the _future.Result, including breaking it out such that DBAPI behaviors are local to specific implementations, it becomes apparent that the Result object is a functional superset of ResultProxy and that basic operations like fetchone(), fetchall(), and fetchmany() behave pretty much exactly the same way on the new object. Reorganize things so that ResultProxy is now referred to as LegacyCursorResult, which subclasses CursorResult that represents the DBAPI-cursor version of Result, making use of a multiple inheritance pattern so that the functionality of Result is also available in non-DBAPI contexts, as will be necessary for some ORM patterns. Additionally propose the composition system for Result that will form the basis for ORM-alternative result systems such as horizontal sharding and dogpile cache. As ORM results will soon be coming directly from instances of Result, these extensions will instead build their own ResultFetchStrategies that perform the special steps to create composed or cached result sets. Also considering at the moment not emitting deprecation warnings for fetchXYZ() methods; the immediate issue is Keystone tests are calling upon it, but as the implementations here are proving to be not in any kind of conflict with how Result works, there's not too much issue leaving them around and deprecating at some later point. References: #5087 References: #4395 Fixes: #4959 Change-Id: I8091919d45421e3f53029b8660427f844fee0228
* Create initial 2.0 engine implementationMike Bayer2020-04-161-0/+7
| | | | | | | | | | | | | | | | | | | Implemented the SQLAlchemy 2 :func:`.future.create_engine` function which is used for forwards compatibility with SQLAlchemy 2. This engine features always-transactional behavior with autobegin. Allow execution options per statement execution. This includes that the before_execute() and after_execute() events now accept an additional dictionary with these options, empty if not passed; a legacy event decorator is added for backwards compatibility which now also emits a deprecation warning. Add some basic tests for execution, transactions, and the new result object. Build out on a new testing fixture that swaps in the future engine completely to start with. Change-Id: I70e7338bb3f0ce22d2f702537d94bb249bd9fb0a Fixes: #4644
* Fix typo in resultproxy.c and test compatibility with python 3.5Federico Caselli2020-03-281-2/+2
| | | | | | | | | | | - Fix typo in resultproxy.c that would error on windows. - add -Wundef to C flags when linux is detected so that undefined symbols emit a warning - a few adjustments for tests to succeed on python 3.5 - note minimum version still documented here as 3.4 but this should move to at least 3.5 if not 3.6 for SQLAlchemy 1.4 Change-Id: Ia93ee1cb5c52e51e72eb0a24c100421c5157d04b
* Ensure all nested exception throws have a causeMike Bayer2020-03-021-1/+1
| | | | | | | | | | | | | | | Applied an explicit "cause" to most if not all internally raised exceptions that are raised from within an internal exception catch, to avoid misleading stacktraces that suggest an error within the handling of an exception. While it would be preferable to suppress the internally caught exception in the way that the ``__suppress_context__`` attribute would, there does not as yet seem to be a way to do this without suppressing an enclosing user constructed context, so for now it exposes the internally caught exception as the cause so that full information about the context of the error is maintained. Fixes: #4849 Change-Id: I55a86b29023675d9e5e49bc7edc5a2dc0bcd4751
* Result initial introductionMike Bayer2020-02-211-56/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This builds on cc718cccc0bf8a01abdf4068c7ea4f3 which moved RowProxy to Row, allowing Row to be more like a named tuple. - KeyedTuple in ORM is replaced with Row - ResultSetMetaData broken out into "simple" and "cursor" versions for ORM and Core, as well as LegacyCursor version. - Row now has _mapping attribute that supplies full mapping behavior. Row and SimpleRow both have named tuple behavior otherwise. LegacyRow has some mapping features on the tuple which emit deprecation warnings (e.g. keys(), values(), etc). the biggest change for mapping->tuple is the behavior of __contains__ which moves from testing of "key in row" to "value in row". - ResultProxy breaks into ResultProxy and FutureResult (interim), the latter has the newer APIs. Made available to dialects using execution options. - internal reflection methods and most tests move off of implicit Row mapping behavior and move to row._mapping, result.mappings() method using future result - a new strategy system for cursor handling replaces the various subclasses of RowProxy - some execution context adjustments. We will leave EC in but refined things like get_result_proxy() and out parameter handling. Dialects for 1.4 will need to adjust from get_result_proxy() to get_result_cursor_strategy(), if they are using this method - out parameter handling now accommodated by get_out_parameter_values() EC method. Oracle changes for this. external dialect for DB2 for example will also need to adjust for this. - deprecate case_insensitive flag for engine / result, this feature is not used mapping-methods on Row are deprecated, and replaced with Row._mapping.<meth>, including: row.keys() -> use row._mapping.keys() row.items() -> use row._mapping.items() row.values() -> use row._mapping.values() key in row -> use key in row._mapping int in row -> use int < len(row) Fixes: #4710 Fixes: #4878 Change-Id: Ieb9085e9bcff564359095b754da9ae0af55679f0
* Fix cext for Python 2; ensure C extensions build successfullyMike Bayer2020-01-041-1/+15
| | | | | | | | | | | | | | | | | | | | The C extensions have been broken since cc718cccc0bf8a01abdf4068c however CI did not find this, because the build degraded to non-C extensions without failing. Ensure that if cext is set, there is no fallback to non-cext build if the C extension build fails. Repair C related issues introduced in cc718cccc0bf8a01abdf4068c. As C extensions have been silently failing on 2.7 for some commits, the callcounts also needed to be adjusted for recent performance-related changes. That in turn required a fix to the profiling decorator to use signature rewriting in order to support py.test's fixture mechanism under Python 2, usage introduced under profiling in 89bf6d80a9. Fixes: #5076 Change-Id: Id968f10c85d6bf489298b1c318a1f869ad3e7d80
* happy new yearMike Bayer2020-01-013-3/+3
| | | | Change-Id: I08440dc25e40ea1ccea1778f6ee9e28a00808235
* Run row value processors up frontMike Bayer2019-10-011-263/+426
| | | | | | | | | | | | as part of a larger series of changes to generalize row-tuples, RowProxy becomes plain Row and is no longer a "proxy"; the DBAPI row is now copied directly into the Row when constructed, result handling occurs at once. Subsequent changes will break out Row into a new version that behaves fully a tuple. Change-Id: I2ffa156afce5d21c38f28e54c3a531f361345dd5
* happy new yearMike Bayer2019-01-113-3/+3
| | | | Change-Id: I6a71f4924d046cf306961c58dffccf21e9c03911
* Use INITERROR macro in utils.cAndru19992018-06-131-8/+10
| | | | | | | | | | Remove py3k check where we initialize the module and instead make this look like the same init sequence as resultproxy.c, processors.c Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Change-Id: Ia6352e50eaf760d95ab2bbf66d90c023c37f1193 Pull-request: https://github.com/zzzeek/sqlalchemy/pull/429
* happy new yearMike Bayer2018-01-123-3/+3
| | | | Change-Id: I3ef36bfd0cb0ba62b3123c8cf92370a43156cf8f
* - update copyright in .c filesMike Bayer2017-01-173-3/+3
| | | | Change-Id: If905d1bc026b688ec7203674ff14c72bc4906abf
* Make boolean processors consistent between Py/C; coerce to 1/0Mike Bayer2016-06-231-12/+2
| | | | | | | | | | | | | The processing performed by the :class:`.Boolean` datatype for backends that only feature integer types has been made consistent between the pure Python and C-extension versions, in that the C-extension version will accept any integer value from the database as a boolean, not just zero and one; additionally, non-boolean integer values being sent to the database are coerced to exactly zero or one, instead of being passed as the original integer value. Change-Id: I01e647547fd7047bd549dd70e1fa202c51e8328b Fixes: #3730
* Merge remote-tracking branch 'origin/pr/231' into pr231Mike Bayer2016-03-301-0/+4
|\
| * - properly handle negative indexes in RowProxy.__getitem__()pr/231Lele Gaifax2016-01-281-0/+4
| |
* | - happy new yearMike Bayer2016-01-293-3/+3
|/
* - reinstate "dont set up integer index in keymap if we're on cexts",Mike Bayer2016-01-271-1/+1
| | | | | and this time also fix the cext itself to properly handle int vs. long on py2k
* - A deep improvement to the recently added :meth:`.TextClause.columns`Mike Bayer2016-01-141-4/+9
| | | | | | | | | | | | | | | | | | | method, and its interaction with result-row processing, now allows the columns passed to the method to be positionally matched with the result columns in the statement, rather than matching on name alone. The advantage to this includes that when linking a textual SQL statement to an ORM or Core table model, no system of labeling or de-duping of common column names needs to occur, which also means there's no need to worry about how label names match to ORM columns and so-forth. In addition, the :class:`.ResultProxy` has been further enhanced to map column and string keys to a row with greater precision in some cases. fixes #3501 - reorganize the initialization of ResultMetaData for readability and complexity; use the name "cursor_description", define the task of "merging" cursor_description with compiled column information as its own function, and also define "name extraction" as a separate task. - fully change the name we use in the "ambiguous column" error to be the actual name that was ambiguous, modify the C ext also
* - copyright 2015Mike Bayer2015-03-103-3/+3
|
* - rework Oracle to no longer do its own unicode conversion; this has been ↵Mike Bayer2014-01-171-0/+41
| | | | | | | | | | | observed to be very slow. this now has the effect of producing "conditional" unicode conversion for the Oracle backend, as it still returns NVARCHAR etc. as unicode [ticket:2911] - add new "conditional" functionality to unicode processors; the C-level function now uses PyUnicode_Check() as a fast alternative to the isinstance() check in Python
* Use PyMODINIT_FUNCpr/55cgohlke2014-01-081-1/+1
|
* Use PyMODINIT_FUNCcgohlke2014-01-081-1/+1
|
* Use PyMODINIT_FUNCcgohlke2014-01-081-1/+1
|
* - happy new yearMike Bayer2014-01-053-3/+3
|
* - The C extensions are ported to Python 3 and will build underMike Bayer2013-07-263-45/+352
| | | | any supported CPython 2 or 3 environment. [ticket:2161]
* happy new year (see #2645)Diana Clarke2013-01-013-3/+3
|
* fix some warningsMike Bayer2012-08-281-3/+11
|
* - [bug] Fixed cextension bug whereby theMike Bayer2012-08-221-3/+10
| | | | | | | | | | | | | | | | | | | "ambiguous column error" would fail to function properly if the given index were a Column object and not a string. Note there are still some column-targeting issues here which are fixed in 0.8. [ticket:2553] - find more cases where column targeting is being inaccurate, add more information to result_map to better differentiate "ambiguous" results from "present" or "not present". In particular, result_map is sensitive to dupes, even though no error is raised; the conflicting columns are added to the "obj" member of the tuple so that the two are both directly accessible in the result proxy - handwringing over the damn "name fallback" thing in results. can't really make it perfect yet - fix up oracle returning clause. not sure why its guarding against labels, remove that for now and see what the bot says.
* - add new C extension "utils", so far includes distill_paramsMike Bayer2012-08-073-0/+191
| | | | | - repair test_processors which wasn't hitting the python functions - add another suite to test_processors that does distill_params
* - break out engine/base.py into base, interfaces, result, util.Mike Bayer2012-08-071-7/+7
| | | | - remove deprecated 0.7 engine methods
* - [bug] Fixed memory leak in C version ofMike Bayer2012-06-011-6/+18
| | | | | | | | result proxy whereby DBAPIs which don't deliver pure Python tuples for result rows would fail to decrement refcounts correctly. The most prominently affected DBAPI is pyodbc. [ticket:2489]
* - [bug] Fixed issue whereby attribute-basedMike Bayer2012-03-141-1/+10
| | | | | | | | column access on a row would raise AttributeError with non-C version, NoSuchColumnError with C version. Now raises AttributeError in both cases. [ticket:2398]
* - [bug] Fixed bug in C extensions wherebyMike Bayer2012-03-131-15/+10
| | | | | | | | string format would not be applied to a Numeric value returned as integer; this affected primarily SQLite which does not maintain numeric scale settings. [ticket:2432]
* - [bug] Fixed memory leak in core which wouldMike Bayer2012-03-101-0/+6
| | | | | | | occur when C extensions were used with particular types of result fetches, in particular when orm query.count() were called. [ticket:2427]
* o null check PyObject_Repr resultsPhilip Jenvey2012-01-232-7/+20
| | | | o limit size of strings passed to PyErr_Format
* - [bug] Fixed bug whereby a table-bound ColumnMike Bayer2012-01-221-1/+1
| | | | | | | | | | | object named "<a>_<b>" which matched a column labeled as "<tablename>_<colname>" could match inappropriately when targeting in a result set row. [ticket:2377] - requires that we change the tuple format in RowProxy. Makes an improvement to the cases tested against an unpickled RowProxy as well though doesn't solve the problem there entirely.
* - [bug] Improved error messages when a non-stringMike Bayer2012-01-221-6/+45
| | | | | | | | | or invalid string is passed to any of the date/time processors used by SQLite, including C and Python versions. [ticket:2382] - changed the import model of processors.py so that we can get at the pure python versions and C versions simultaneously in tests.
* - Added some defs to the resultproxy.c extension so thatMike Bayer2011-01-301-0/+2
| | | | | the extension compiles and runs on Python 2.4. [ticket:2023]
* fixed a small potential memory leak in UnicodeResultProcessor (for some weirdGaëtan de Menten2010-11-272-7/+15
| | | | | reason, it didn't actually leak in my tests) by providing a dealloc method to the type, and added a test to ensure it stays that way. Closes #1981.
* Fix memory leaks in the cprocessors DecimalResultProcessor, including tests. ↵Taavi Burns2010-11-191-1/+10
| | | | [ticket:1978]
* - Implemented sequence check capability for the CMike Bayer2010-11-091-1/+7
| | | | | | version of RowProxy, as well as 2.7 style "collections.Sequence" registration for RowProxy. [ticket:1871]
* take2 on #1781.Gaëtan de Menten2010-04-271-1/+2
|
View Lorry Controller Status