summaryrefslogtreecommitdiff
path: root/lib/sqlalchemy/dialects/postgresql/array.py
Commit message (Collapse)AuthorAgeFilesLines
* Remove old versionadded and versionchangedFederico Caselli2023-04-121-7/+0
| | | | | | | Removed versionadded and versionchanged for version prior to 1.2 since they are no longer useful. Change-Id: I5c53d1188bc5fec3ab4be39ef761650ed8fa6d3e
* happy new year 2023Mike Bayer2023-01-031-1/+1
| | | | Change-Id: I625af65b3fb1815b1af17dc2ef47dd697fdc3fb1
* Try running pyupgrade on the codeFederico Caselli2022-11-161-1/+1
| | | | | | | | command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>" pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not exists in sqlalchemy fixtures Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
* apply consistent ORM mutable notes for all mutable SQL typesMike Bayer2022-08-301-0/+25
| | | | | | | | | | | | in https://github.com/sqlalchemy/sqlalchemy/discussions/8447 I was surprised that we didnt have any notes about using Mutable for ARRAY classes, since we have them for HSTORE and JSON. Add a consistent topic box for these so we have something to point towards. Change-Id: Idfa1b2cbee67024545f4fa299e4c875075ec7d3f
* implement literal stringification for arraysMike Bayer2022-06-151-31/+20
| | | | | | | | | | | | | | | as we already implement stringification for the contents, provide a bracketed syntax for default and ARRAY literal for PG specifically. ARRAY literal seems much simpler to render than their quoted syntax which requires double quotes for strings. also open up testing for pg8000 which has likely been fine with arrays for awhile now, bump the version pin also. Fixes: #8138 Change-Id: Id85b052b0a9564d6aa1489160e58b7359f130fdd
* add typing for PG UUID, other typesMike Bayer2022-05-271-1/+7
| | | | | | | note that UUID will be generalized into core with #7212. Fixes: #6402 Change-Id: I90f0052ca74367c2c2f1ce2f8a90e81d173d1430
* inline mypy config; files ignoring type errors for the momentMike Bayer2022-04-281-0/+2
| | | | | | | | | | | | | | | | | | | to simplify pyproject.toml change the remaining files that aren't going to be typed on this first pass (unless of course someone wants to type some of these) to include # mypy: ignore-errors. for the moment, only a handful of ORM modules are to have more type checking implemented. It's important that ignore-errors is used and not "# type: ignore", as in the latter case, mypy doesn't even read the existing types in the file, which makes it impossible to type any files that refer to those modules at all. to simplify ongoing typing work use inline mypy config for remaining files that are "done" for now, indicating the level of type checking they currently have. Change-Id: I98669c1a305c2f0adba85d10b5425541f3fe9533
* implement multi-element expression constructsMike Bayer2022-04-131-12/+20
| | | | | | | | | | | | | | | | | | | | | | Improved the construction of SQL binary expressions to allow for very long expressions against the same associative operator without special steps needed in order to avoid high memory use and excess recursion depth. A particular binary operation ``A op B`` can now be joined against another element ``op C`` and the resulting structure will be "flattened" so that the representation as well as SQL compilation does not require recursion. To implement this more cleanly, the biggest change here is that column-oriented lists of things are broken away from ClauseList in a new class ExpressionClauseList, that also forms the basis of BooleanClauseList. ClauseList is still used for the generic "comma-separated list" of things such as Tuple and things like ORDER BY, as well as in some API endpoints. Also adds __slots__ to the TypeEngine-bound Comparator classes. Still can't really do __slots__ on ClauseElement. Fixes: #7744 Change-Id: I81a8ceb6f8f3bb0fe52d58f3cb42e4b6c2bc9018
* restore empty list logic to ARRAY of ENUM parsingMike Bayer2022-01-201-1/+2
| | | | | | | | | | Fixed regression where the change in :ticket:`7148` to repair ENUM handling in PostgreSQL broke the use case of an empty ARRAY of ENUM, preventing rows that contained an empty array from being handled correctly when fetching results. Fixes: #7590 Change-Id: I43a35ef25281a6e0a26b698efebef6ba12a63e8c
* initial reorganize for static typingMike Bayer2022-01-121-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | start applying foundational annotations to key elements. two main elements addressed here: 1. removal of public_factory() and replacement with explicit functions. this just works much better with typing. 2. typing support for column expressions and operators. The biggest part of this involves stubbing out all the ColumnOperators methods under ColumnElement in a TYPE_CHECKING section. Took me a while to see this method vs. much more complicated things I thought I needed. Also for this version implementing #7519, ColumnElement types against the Python type and not TypeEngine. it is hoped this leads to easier transferrence between ORM/Core as well as eventual support for result set typing. Not clear yet how well this approach will work and what new issues it may introduce. given the current approach we now get full, rich typing for scenarios like this: from sqlalchemy import column, Integer, String, Boolean c1 = column('a', String) c2 = column('a', Integer) expr1 = c2.in_([1, 2, 3]) expr2 = c2 / 5 expr3 = -c2 expr4_a = ~(c2 == 5) expr4_b = ~column('q', Boolean) expr5 = c1 + 'x' expr6 = c2 + 10 Fixes: #7519 Fixes: #6810 Change-Id: I078d9f57955549f6f7868314287175f6c61c44cb
* happy new year 2022Mike Bayer2022-01-061-1/+1
| | | | Change-Id: I49abf2607e0eb0623650efdf0091b1fb3db737ea
* Improve array of enum handling.Federico Caselli2022-01-041-2/+27
| | | | | | | Fixed handling of array of enum values which require escape characters. Fixes: #7418 Change-Id: I50525846f6029dfea9a8ad1cb913424d168d5f62
* Warn when caching is disabled / documentMike Bayer2021-12-061-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds new warnings for all elements that don't indicate their caching behavior, including user-defined ClauseElement subclasses and third party dialects. it additionally adds new documentation to discuss an apparent performance degradation in 1.4 when caching is disabled as a result in the significant expense incurred by ORM lazy loaders, which in 1.3 used BakedQuery so were actually cached. As a result of adding the warnings, a fair degree of lesser used SQL expression objects identified that they did not define caching behavior so would have been producing ``[no key]``, including PostgreSQL constructs ``hstore`` and ``array``. These have been amended to use inherit cache where appropriate. "on conflict" constructs in PostgreSQL, MySQL, SQLite still explicitly don't generate a cache key at this time. The change also adds a test for all constructs via assert_compile() to assert they will not generate cache warnings. Fixes: #7394 Change-Id: I85958affbb99bfad0f5efa21bc8f2a95e7e46981
* Merge "propose emulated setinputsizes embedded in the compiler" into mainmike bayer2021-11-251-3/+0
|\
| * propose emulated setinputsizes embedded in the compilerMike Bayer2021-11-231-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a new system so that PostgreSQL and other dialects have a reliable way to add casts to bound parameters in SQL statements, replacing previous use of setinputsizes() for PG dialects. rationale: 1. psycopg3 will be using the same SQLAlchemy-side "setinputsizes" as asyncpg, so we will be seeing a lot more of this 2. the full rendering that SQLAlchemy's compilation is performing is in the engine log as well as error messages. Without this, we introduce three levels of SQL rendering, the compiler, the hidden "setinputsizes" in SQLAlchemy, and then whatever the DBAPI driver does. With this new approach, users reporting bugs etc. will be less confused that there are as many as two separate layers of "hidden rendering"; SQLAlchemy's rendering is again fully transparent 3. calling upon a setinputsizes() method for every statement execution is expensive. this way, the work is done behind the caching layer 4. for "fast insertmany()", I also want there to be a fast approach towards setinputsizes. As it was, we were going to be taking a SQL INSERT with thousands of bound parameter placeholders and running a whole second pass on it to apply typecasts. this way, we will at least be able to build the SQL string once without a huge second pass over the whole string 5. psycopg2 can use this same system for its ARRAY casts 6. the general need for PostgreSQL to have lots of type casts is now mostly in the base PostgreSQL dialect and works independently of a DBAPI being present. dependence on DBAPI symbols that aren't complete / consistent / hashable is removed I was originally going to try to build this into bind_expression(), but it was revealed this worked poorly with custom bind_expression() as well as empty sets. the current impl also doesn't need to run a second expression pass over the POSTCOMPILE sections, which came out better than I originally thought it would. Change-Id: I363e6d593d059add7bcc6d1f6c3f91dd2e683c0c
* | Clean up most py3k compatFederico Caselli2021-11-241-2/+2
|/ | | | Change-Id: I8172fdcc3103ff92aa049827728484c8779af6b7
* Improve array support on pg8000Federico Caselli2021-10-271-1/+1
| | | | | | References: #6023 Change-Id: I0f6cbc34b3c0bfc0b8c86b3ebe4531e23039b6c0
* repair any_() / all_() "implicit flip" behavior for NoneMike Bayer2021-10-041-14/+4
| | | | | | | | | | | | | | Fixed an inconsistency in the any_() / all_() functions / methods where the special behavior these functions have of "flipping" the expression such that the "ANY" / "ALL" expression is always on the right side would not function if the comparison were against the None value, that is, "column.any_() == None" should produce the same SQL expression as "null() == column.any_()". Added more docs to clarify this as well, plus mentions that any_() / all_() generally supersede the ARRAY version "any()" / "all()". Fixes: #7140 Change-Id: Ia5d55414ba40eb3fbda3598931fdd24c9b4a4411
* Fixes: #6930jonathan vanasco2021-09-081-0/+3
| | | | | | Note in docstrings that ignored kwargs are required for API conformance. Change-Id: Icc9a8c63c0936a7c5255841ef49d10a83496763a
* fix linter JOIN logic; fix PostgreSQL ARRAY op comparisonMike Bayer2021-08-151-3/+3
| | | | | | | | | | | | | | | | | | | Adjusted the "from linter" warning feature to accommodate for a chain of joins more than one level deep where the ON clauses don't explicitly match up the targets, such as an expression such as "ON TRUE". This mode of use is intended to cancel the cartesian product warning simply by the fact that there's a JOIN from "a to b", which was not working for the case where the chain of joins had more than one element. this incurs a bit more compiler overhead that comes out in profiling but is not extensive. Added the "is_comparison" flag to the PostgreSQL "overlaps", "contained_by", "contains" operators, so that they work in relevant ORM contexts as well as in conjunction with the "from linter" feature. Fixes: #6886 Change-Id: I078dc3fe6d4f7871ffe4ebac3e71e62f3f213d12
* Replace all http:// links to https://Federico Caselli2021-07-041-1/+1
| | | | | | Also replace http://pypi.python.org/pypi with https://pypi.org/project Change-Id: I84b5005c39969a82140706472989f2a30b0c7685
* CAST the elements in ARRAYs when using psycopg2Federico Caselli2021-03-151-10/+1
| | | | | | | | | | | | Adjusted the psycopg2 dialect to emit an explicit PostgreSQL-style cast for bound parameters that contain ARRAY elements. This allows the full range of datatypes to function correctly within arrays. The asyncpg dialect already generated these internal casts in the final statement. This also includes support for array slice updates as well as the PostgreSQL-specific :meth:`_postgresql.ARRAY.contains` method. Fixes: #6023 Change-Id: Ia7519ac4371a635f05ac69a3a4d0f4e6d2f04cad
* Implement support for functions as FROM with columns clause supportMike Bayer2021-02-031-0/+2
| | | | | | | | | | | | | | | | Implemented support for "table valued functions" along with additional syntaxes supported by PostgreSQL, one of the most commonly requested features. Table valued functions are SQL functions that return lists of values or rows, and are prevalent in PostgreSQL in the area of JSON functions, where the "table value" is commonly referred towards as the "record" datatype. Table valued functions are also supported by Oracle and SQL Server. Moved from I5b093b72533ef695293e737eb75850b9713e5e03 due to accidental push Fixes: #3566 Change-Id: Iea36d04c80a5ed3509dcdd9ebf0701687143fef5
* happy new yearMike Bayer2021-01-041-1/+1
| | | | Change-Id: Ic5bb19ca8be3cb47c95a0d3315d84cb484bac47c
* Update select usage to use the new 1.4 formatFederico Caselli2020-09-081-5/+3
| | | | | | | | | | | | | | | | This change includes mainly that the bracketed use within select() is moved to positional, and keyword arguments are removed from calls to the select() function. it does not yet fully address other issues such as keyword arguments passed to the table.select(). Additionally, allows False / None to both be considered as "disable" for all of select.correlate(), select.correlate_except(), query.correlate(), which establishes consistency with passing of ``False`` for the legact select(correlate=False) argument. Change-Id: Ie6c6e6abfbd3d75d4c8de504c0cf0159e6999108
* Create a real type for Tuple() and handle appropriately in compilerMike Bayer2020-08-171-6/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | Improved the :func:`_sql.tuple_` construct such that it behaves predictably when used in a columns-clause context. The SQL tuple is not supported as a "SELECT" columns clause element on most backends; on those that do (PostgreSQL, not surprisingly), the Python DBAPI does not have a "nested type" concept so there are still challenges in fetching rows for such an object. Use of :func:`_sql.tuple_` in a :func:`_sql.select` or :class:`_orm.Query` will now raise a :class:`_exc.CompileError` at the point at which the :func:`_sql.tuple_` object is seen as presenting itself for fetching rows (i.e., if the tuple is in the columns clause of a subquery, no error is raised). For ORM use,the :class:`_orm.Bundle` object is an explicit directive that a series of columns should be returned as a sub-tuple per row and is suggested by the error message. Additionally ,the tuple will now render with parenthesis in all contexts. Previously, the parenthesization would not render in a columns context leading to non-defined behavior. As part of this change, Tuple receives a dedicated datatype which appears to allow us the very desirable change of removing the bindparam._expanding_in_types attribute as well as ClauseList._tuple_values (which might already have not been needed due to #4645). Fixes: #5127 Change-Id: Iecafa0e0aac2f1f37ec8d0e1631d562611c90200
* Support `ARRAY` of `Enum`, `JSON` or `JSONB`Federico Caselli2020-04-201-12/+41
| | | | | | | | | | | | | | Added support for columns or type :class:`.ARRAY` of :class:`.Enum`, :class:`.JSON` or :class:`_postgresql.JSONB` in PostgreSQL. Previously a workaround was required in these use cases. Raise an explicit :class:`.exc.CompileError` when adding a table with a column of type :class:`.ARRAY` of :class:`.Enum` configured with :paramref:`.Enum.native_enum` set to ``False`` when :paramref:`.Enum.create_constraint` is not set to ``False`` Fixes: #5265 Fixes: #5266 Change-Id: I83a2d20a599232b7066d0839f3e55ff8b78cd8fc
* Run search and replace of symbolic module namesMike Bayer2020-04-141-18/+22
| | | | | | | | Replaces a wide array of Sphinx-relative doc references with an abbreviated absolute form now supported by zzzeeksphinx. Change-Id: I94bffcc3f37885ffdde6238767224296339698a2
* happy new yearMike Bayer2020-01-011-1/+1
| | | | Change-Id: I08440dc25e40ea1ccea1778f6ee9e28a00808235
* Remove erroneous assertion from array._bind_paramMike Bayer2019-08-261-3/+0
| | | | | | | | | | | Fixed bug where Postgresql operators such as :meth:`.postgresql.ARRAY.Comparator.contains` and :meth:`.postgresql.ARRAY.Comparator.contained_by` would fail to function correctly for non-integer values when used against a :class:`.postgresql.array` object, due to an erroneous assert statement. Fixes: #4822 Change-Id: I886aca4c86dc7d64e9d0dfc1d910a0ae64d775a1
* Support multidimensional array literals in PostgresqlMike Bayer2019-07-081-2/+28
| | | | | | | | | Added support for multidimensional Postgresql array literals via nesting the :class:`.postgresql.array` object within another one. The multidimensional array type is detected automatically. Fixes: #4756 Change-Id: Ie2107ad3cf291112f6ca330dc90dc15a0a940cee
* Remove version directives for 0.6, 0.7, 0.8Mike Bayer2019-01-151-4/+2
| | | | | | | | | - fix a few "seealso"s - ComparableProprerty's "superseded in 0.7" becomes deprecated in 0.7 Backport to currently maintained doc versions 1.2, 1.1 Change-Id: Ib1fcb2df8673dbe5c4ffc47f3896a60d1dfcb4b2
* happy new yearMike Bayer2019-01-111-1/+1
| | | | Change-Id: I6a71f4924d046cf306961c58dffccf21e9c03911
* Post black reformattingMike Bayer2019-01-061-9/+12
| | | | | | | | | | | | | Applied on top of a pure run of black -l 79 in I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes resolves all remaining flake8 conditions for those codes we have enabled in setup.cfg. Included are resolutions for all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
* Run black -l 79 against all source filesMike Bayer2019-01-061-27/+52
| | | | | | | | | | | | | | This is a straight reformat run using black as is, with no edits applied at all. The black run will format code consistently, however in some cases that are prevalent in SQLAlchemy code it produces too-long lines. The too-long lines will be resolved in the following commit that will resolve all remaining flake8 issues including shadowed builtins, long lines, import order, unused imports, duplicate imports, and docstring issues. Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
* happy new yearMike Bayer2018-01-121-1/+1
| | | | Change-Id: I3ef36bfd0cb0ba62b3123c8cf92370a43156cf8f
* Set up base ARRAY to be compatible with postgresql.ARRAY.Mike Bayer2017-04-111-14/+3
| | | | | | | | | | For some reason, when ARRAY was added to the base it was never linked to postgresql.ARRAY. Link the two types and also make base ARRAY the schema event target so that it supports the same features as postgresql.ARRAY. Change-Id: I82fa6c9d2b8c5028dba3a009715f7bc296b2bc0b Fixes: #3964
* update for 2017 copyrightMike Bayer2017-01-041-1/+1
| | | | Change-Id: I4e8c2aa8fe817bb2af8707410fa0201f938781de
* spelling: Postgresql -> PostgreSQLVille Skyttä2016-10-081-3/+3
|
* Fix typo in commentkeeyip chan2016-09-111-2/+2
|
* Spelling fixespr/299Ville Skyttä2016-08-081-2/+2
|
* - happy new yearMike Bayer2016-01-291-1/+1
|
* - Added :class:`.mysql.JSON` for MySQL 5.7. The JSON type providesMike Bayer2016-01-061-17/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | persistence of JSON values in MySQL as well as basic operator support of "getitem" and "getpath", making use of the ``JSON_EXTRACT`` function in order to refer to individual paths in a JSON structure. fixes #3547 - Added a new type to core :class:`.types.JSON`. This is the base of the PostgreSQL :class:`.postgresql.JSON` type as well as that of the new :class:`.mysql.JSON` type, so that a PG/MySQL-agnostic JSON column may be used. The type features basic index and path searching support. fixes #3619 - reorganization of migration docs etc. to try to refer both to the fixes to JSON that helps Postgresql while at the same time indicating these are new features of the new base JSON type. - a rework of the Array/Indexable system some more, moving things that are specific to Array out of Indexable. - new operators for JSON indexing added to core so that these can be compiled by the PG and MySQL dialects individually - rename sqltypes.Array to sqltypes.ARRAY - as there is no generic Array implementation, this is an uppercase type for now, consistent with the new sqltypes.JSON type that is also not a generic implementation. There may need to be some convention change to handle the case of datatypes that aren't generic, rely upon DB-native implementations, but aren't necessarily all named the same thing.
* - The use of a :class:`.postgresql.ARRAY` object that refersMike Bayer2015-09-191-1/+14
| | | | | | | to a :class:`.types.Enum` or :class:`.postgresql.ENUM` subtype will now emit the expected "CREATE TYPE" and "DROP TYPE" DDL when the type is used within a "CREATE TABLE" or "DROP TABLE". fixes #2729
* - add a postgresql-specific form of array_agg() that injects theMike Bayer2015-08-271-2/+2
| | | | ARRAY type, references #3132
* - build out a new base type for Array, as well as new any/all operatorsMike Bayer2015-08-251-176/+46
| | | | | | - any/all work for Array as well as subqueries, accepted by MySQL - Postgresql ARRAY now subclasses Array - fixes #3516
* - document workaround type for ARRAY of ENUM, fixes #3467Mike Bayer2015-08-181-0/+4
|
* - merge of ticket_3499 indexed access branchMike Bayer2015-08-171-0/+419
- The "hashable" flag on special datatypes such as :class:`.postgresql.ARRAY`, :class:`.postgresql.JSON` and :class:`.postgresql.HSTORE` is now set to False, which allows these types to be fetchable in ORM queries that include entities within the row. fixes #3499 - The Postgresql :class:`.postgresql.ARRAY` type now supports multidimensional indexed access, e.g. expressions such as ``somecol[5][6]`` without any need for explicit casts or type coercions, provided that the :paramref:`.postgresql.ARRAY.dimensions` parameter is set to the desired number of dimensions. fixes #3487 - The return type for the :class:`.postgresql.JSON` and :class:`.postgresql.JSONB` when using indexed access has been fixed to work like Postgresql itself, and returns an expression that itself is of type :class:`.postgresql.JSON` or :class:`.postgresql.JSONB`. Previously, the accessor would return :class:`.NullType` which disallowed subsequent JSON-like operators to be used. part of fixes #3503 - The :class:`.postgresql.JSON`, :class:`.postgresql.JSONB` and :class:`.postgresql.HSTORE` datatypes now allow full control over the return type from an indexed textual access operation, either ``column[someindex].astext`` for a JSON type or ``column[someindex]`` for an HSTORE type, via the :paramref:`.postgresql.JSON.astext_type` and :paramref:`.postgresql.HSTORE.text_type` parameters. also part of fixes #3503 - The :attr:`.postgresql.JSON.Comparator.astext` modifier no longer calls upon :meth:`.ColumnElement.cast` implicitly, as PG's JSON/JSONB types allow cross-casting between each other as well. Code that makes use of :meth:`.ColumnElement.cast` on JSON indexed access, e.g. ``col[someindex].cast(Integer)``, will need to be changed to call :attr:`.postgresql.JSON.Comparator.astext` explicitly. This is part of the refactor in references #3503 for consistency in operator use.
View Lorry Controller Status