summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorMike Bayer <mike_mp@zzzcomputing.com>2014-11-06 17:15:30 -0500
committerMike Bayer <mike_mp@zzzcomputing.com>2014-11-06 17:15:30 -0500
commit590498bf844e7dcdcf41d3ac786b4cccbebd2d43 (patch)
tree0455eea3f8555a4b78ec7fa015b06d9ffc88d47f /doc
parentb9d430af752b7cc955932a54a8f8db18f46d89a6 (diff)
parent8200c2cd35b3e85a636baabe8324b9ecbbd8fedf (diff)
downloadsqlalchemy-590498bf844e7dcdcf41d3ac786b4cccbebd2d43.tar.gz
Merge branch 'master' into ticket_3100
Diffstat (limited to 'doc')
-rw-r--r--doc/build/changelog/changelog_09.rst178
-rw-r--r--doc/build/changelog/changelog_10.rst349
-rw-r--r--doc/build/changelog/migration_10.rst1030
-rw-r--r--doc/build/core/defaults.rst4
-rw-r--r--doc/build/core/sqlelement.rst5
-rw-r--r--doc/build/dialects/index.rst1
-rw-r--r--doc/build/dialects/sqlite.rst7
-rw-r--r--doc/build/faq.rst102
-rw-r--r--doc/build/orm/extensions/declarative.rst1
-rw-r--r--doc/build/orm/inheritance.rst6
-rw-r--r--doc/build/orm/loading.rst26
-rw-r--r--doc/build/orm/relationships.rst135
-rw-r--r--doc/build/orm/tutorial.rst7
13 files changed, 1624 insertions, 227 deletions
diff --git a/doc/build/changelog/changelog_09.rst b/doc/build/changelog/changelog_09.rst
index 329e054b0..8ed2ea776 100644
--- a/doc/build/changelog/changelog_09.rst
+++ b/doc/build/changelog/changelog_09.rst
@@ -11,7 +11,185 @@
:start-line: 5
.. changelog::
+ :version: 0.9.9
+
+ .. change::
+ :tags: bug, orm, sqlite
+ :versions: 1.0.0
+ :tickets: 3241
+
+ Fixed bug regarding expression mutations which could express
+ itself as a "Could not locate column" error when using
+ :class:`.Query` to select from multiple, anonymous column
+ entities when querying against SQLite, as a side effect of the
+ "join rewriting" feature used by the SQLite dialect.
+
+ .. change::
+ :tags: feature, sqlite
+ :versions: 1.0.0
+
+ Added a new SQLite backend for the SQLCipher backend. This backend
+ provides for encrypted SQLite databases using the pysqlcipher Python
+ driver, which is very similar to the pysqlite driver.
+
+ .. seealso::
+
+ :mod:`~sqlalchemy.dialects.sqlite.pysqlcipher`
+
+ .. change::
+ :tags: bug, orm
+ :tickets: 3232
+ :versions: 1.0.0
+
+ Fixed bug where the ON clause for :meth:`.Query.join`,
+ and :meth:`.Query.outerjoin` to a single-inheritance subclass
+ using ``of_type()`` would not render the "single table criteria" in
+ the ON clause if the ``from_joinpoint=True`` flag were set.
+
+.. changelog::
:version: 0.9.8
+ :released: October 13, 2014
+
+ .. change::
+ :tags: bug, mysql, mysqlconnector
+ :versions: 1.0.0
+
+ Mysqlconnector as of version 2.0, probably as a side effect of
+ the python 3 merge, now does not expect percent signs (e.g.
+ as used as the modulus operator and others) to be doubled,
+ even when using the "pyformat" bound parameter format (this
+ change is not documented by Mysqlconnector). The dialect now
+ checks for py2k and for mysqlconnector less than version 2.0
+ when detecting if the modulus operator should be rendered as
+ ``%%`` or ``%``.
+
+ .. change::
+ :tags: bug, mysql, mysqlconnector
+ :versions: 1.0.0
+
+ Unicode SQL is now passed for MySQLconnector version 2.0 and above;
+ for Py2k and MySQL < 2.0, strings are encoded.
+
+
+ .. change::
+ :tags: bug, oracle
+ :versions: 1.0.0
+ :tickets: 2138
+
+ Fixed long-standing bug in Oracle dialect where bound parameter
+ names that started with numbers would not be quoted, as Oracle
+ doesn't like numerics in bound parameter names.
+
+ .. change::
+ :tags: bug, sql
+ :versions: 1.0.0
+ :tickets: 3195
+
+ Fixed bug where a fair number of SQL elements within
+ the sql package would fail to ``__repr__()`` successfully,
+ due to a missing ``description`` attribute that would then invoke
+ a recursion overflow when an internal AttributeError would then
+ re-invoke ``__repr__()``.
+
+ .. change::
+ :tags: bug, declarative, orm
+ :versions: 1.0.0
+ :tickets: 3185
+
+ Fixed "'NoneType' object has no attribute 'concrete'" error
+ when using :class:`.AbstractConcreteBase` in conjunction with
+ a subclass that declares ``__abstract__``.
+
+ .. change::
+ :tags: bug, engine
+ :versions: 1.0.0
+ :tickets: 3200
+
+ The execution options passed to an :class:`.Engine` either via
+ :paramref:`.create_engine.execution_options` or
+ :meth:`.Engine.update_execution_options` are not passed to the
+ special :class:`.Connection` used to initialize the dialect
+ within the "first connect" event; dialects will usually
+ perform their own queries in this phase, and none of the
+ current available options should be applied here. In
+ particular, the "autocommit" option was causing an attempt to
+ autocommit within this initial connect which would fail with
+ an AttributeError due to the non-standard state of the
+ :class:`.Connection`.
+
+ .. change::
+ :tags: bug, sqlite
+ :versions: 1.0.0
+ :tickets: 3211
+
+ When selecting from a UNION using an attached database file,
+ the pysqlite driver reports column names in cursor.description
+ as 'dbname.tablename.colname', instead of 'tablename.colname' as
+ it normally does for a UNION (note that it's supposed to just be
+ 'colname' for both, but we work around it). The column translation
+ logic here has been adjusted to retrieve the rightmost token, rather
+ than the second token, so it works in both cases. Workaround
+ courtesy Tony Roberts.
+
+ .. change::
+ :tags: bug, postgresql
+ :versions: 1.0.0
+ :tickets: 3021
+
+ A revisit to this issue first patched in 0.9.5, apparently
+ psycopg2's ``.closed`` accessor is not as reliable as we assumed,
+ so we have added an explicit check for the exception messages
+ "SSL SYSCALL error: Bad file descriptor" and
+ "SSL SYSCALL error: EOF detected" when detecting an
+ is-disconnect scenario. We will continue to consult psycopg2's
+ connection.closed as a first check.
+
+ .. change::
+ :tags: bug, orm, engine
+ :versions: 1.0.0
+ :tickets: 3197
+
+ Fixed bug that affected generally the same classes of event
+ as that of :ticket:`3199`, when the ``named=True`` parameter
+ would be used. Some events would fail to register, and others
+ would not invoke the event arguments correctly, generally in the
+ case of when an event was "wrapped" for adaption in some other way.
+ The "named" mechanics have been rearranged to not interfere with
+ the argument signature expected by internal wrapper functions.
+
+ .. change::
+ :tags: bug, declarative
+ :versions: 1.0.0
+ :tickets: 3208
+
+ Fixed an unlikely race condition observed in some exotic end-user
+ setups, where the attempt to check for "duplicate class name" in
+ declarative would hit upon a not-totally-cleaned-up weak reference
+ related to some other class being removed; the check here now ensures
+ the weakref still references an object before calling upon it further.
+
+ .. change::
+ :tags: bug, orm
+ :versions: 1.0.0
+ :tickets: 3199
+
+ Fixed bug that affected many classes of event, particularly
+ ORM events but also engine events, where the usual logic of
+ "de duplicating" a redundant call to :func:`.event.listen`
+ with the same arguments would fail, for those events where the
+ listener function is wrapped. An assertion would be hit within
+ registry.py. This assertion has now been integrated into the
+ deduplication check, with the added bonus of a simpler means
+ of checking deduplication across the board.
+
+ .. change::
+ :tags: bug, mssql
+ :versions: 1.0.0
+ :tickets: 3151
+
+ Fixed the version string detection in the pymssql dialect to
+ work with Microsoft SQL Azure, which changes the word "SQL Server"
+ to "SQL Azure".
.. change::
:tags: bug, orm
diff --git a/doc/build/changelog/changelog_10.rst b/doc/build/changelog/changelog_10.rst
index 9c7f207cc..e63e023d9 100644
--- a/doc/build/changelog/changelog_10.rst
+++ b/doc/build/changelog/changelog_10.rst
@@ -22,6 +22,350 @@
on compatibility concerns, see :doc:`/changelog/migration_10`.
.. change::
+ :tags: bug, sql
+ :pullreq: github:146
+
+ Fixed the name of the :paramref:`.PoolEvents.reset.dbapi_connection`
+ parameter as passed to this event; in particular this affects
+ usage of the "named" argument style for this event. Pull request
+ courtesy Jason Goldberger.
+
+ .. change::
+ :tags: feature, sql
+ :pullreq: github:139
+
+ Added a new parameter :paramref:`.Table.tometadata.name` to
+ the :meth:`.Table.tometadata` method. Similar to
+ :paramref:`.Table.tometadata.schema`, this argument causes the newly
+ copied :class:`.Table` to take on the new name instead of
+ the existing one. An interesting capability this adds is that of
+ copying a :class:`.Table` object to the *same* :class:`.MetaData`
+ target with a new name. Pull request courtesy n.d. parker.
+
+ .. change::
+ :tags: bug, orm
+ :pullreq: github:137
+
+ Repaired support of the ``copy.deepcopy()`` call when used by the
+ :class:`.orm.util.CascadeOptions` argument, which occurs
+ if ``copy.deepcopy()`` is being used with :func:`.relationship`
+ (not an officially supported use case). Pull request courtesy
+ duesenfranz.
+
+ .. change::
+ :tags: bug, sql
+ :tickets: 3170
+
+ Reversing a change that was made in 0.9, the "singleton" nature
+ of the "constants" :func:`.null`, :func:`.true`, and :func:`.false`
+ has been reverted. These functions returning a "singleton" object
+ had the effect that different instances would be treated as the
+ same regardless of lexical use, which in particular would impact
+ the rendering of the columns clause of a SELECT statement.
+
+ .. seealso::
+
+ :ref:`bug_3170`
+
+ .. change::
+ :tags: bug, orm
+ :tickets: 3139
+
+ Fixed bug where :meth:`.Session.expunge` would not fully detach
+ the given object if the object had been subject to a delete
+ operation that was flushed, but not committed. This would also
+ affect related operations like :func:`.make_transient`.
+
+ .. seealso::
+
+ :ref:`bug_3139`
+
+ .. change::
+ :tags: bug, orm
+ :tickets: 3230
+
+ A warning is emitted in the case of multiple relationships that
+ ultimately will populate a foreign key column in conflict with
+ another, where the relationships are attempting to copy values
+ from different source columns. This occurs in the case where
+ composite foreign keys with overlapping columns are mapped to
+ relationships that each refer to a different referenced column.
+ A new documentation section illustrates the example as well as how
+ to overcome the issue by specifying "foreign" columns specifically
+ on a per-relationship basis.
+
+ .. seealso::
+
+ :ref:`relationship_overlapping_foreignkeys`
+
+ .. change::
+ :tags: feature, sql
+ :tickets: 3172
+
+ Exception messages have been spiffed up a bit. The SQL statement
+ and parameters are not displayed if None, reducing confusion for
+ error messages that weren't related to a statement. The full
+ module and classname for the DBAPI-level exception is displayed,
+ making it clear that this is a wrapped DBAPI exception. The
+ statement and parameters themselves are bounded within a bracketed
+ sections to better isolate them from the error message and from
+ each other.
+
+ .. change::
+ :tags: bug, orm
+ :tickets: 3228
+
+ The :meth:`.Query.update` method will now convert string key
+ names in the given dictionary of values into mapped attribute names
+ against the mapped class being updated. Previously, string names
+ were taken in directly and passed to the core update statement without
+ any means to resolve against the mapped entity. Support for synonyms
+ and hybrid attributes as the subject attributes of
+ :meth:`.Query.update` are also supported.
+
+ .. seealso::
+
+ :ref:`bug_3228`
+
+ .. change::
+ :tags: bug, orm
+ :tickets: 3035
+
+ Improvements to the mechanism used by :class:`.Session` to locate
+ "binds" (e.g. engines to use), such engines can be associated with
+ mixin classes, concrete subclasses, as well as a wider variety
+ of table metadata such as joined inheritance tables.
+
+ .. seealso::
+
+ :ref:`bug_3035`
+
+ .. change::
+ :tags: bug, general
+ :tickets: 3218
+
+ The ``__module__`` attribute is now set for all those SQL and
+ ORM functions that are derived as "public factory" symbols, which
+ should assist with documentation tools being able to report on the
+ target module.
+
+ .. change::
+ :tags: feature, sql
+
+ :meth:`.Insert.from_select` now includes Python and SQL-expression
+ defaults if otherwise unspecified; the limitation where non-
+ server column defaults aren't included in an INSERT FROM
+ SELECT is now lifted and these expressions are rendered as
+ constants into the SELECT statement.
+
+ .. seealso::
+
+ :ref:`feature_insert_from_select_defaults`
+
+ .. change::
+ :tags: bug, orm
+ :tickets: 3233
+
+ Fixed bug in single table inheritance where a chain of joins
+ that included the same single inh entity more than once
+ (normally this should raise an error) could, in some cases
+ depending on what was being joined "from", implicitly alias the
+ second case of the single inh entity, producing
+ a query that "worked". But as this implicit aliasing is not
+ intended in the case of single table inheritance, it didn't
+ really "work" fully and was very misleading, since it wouldn't
+ always appear.
+
+ .. seealso::
+
+ :ref:`bug_3233`
+
+
+ .. change::
+ :tags: bug, orm
+ :tickets: 3222
+
+ The ON clause rendered when using :meth:`.Query.join`,
+ :meth:`.Query.outerjoin`, or the standalone :func:`.orm.join` /
+ :func:`.orm.outerjoin` functions to a single-inheritance subclass will
+ now include the "single table criteria" in the ON clause even
+ if the ON clause is otherwise hand-rolled; it is now added to the
+ criteria using AND, the same way as if joining to a single-table
+ target using relationship or similar.
+
+ This is sort of in-between feature and bug.
+
+ .. seealso::
+
+ :ref:`migration_3222`
+
+ .. change::
+ :tags: feature, sql
+ :tickets: 3184
+ :pullreq: bitbucket:30
+
+ The :class:`.UniqueConstraint` construct is now included when
+ reflecting a :class:`.Table` object, for databases where this
+ is applicable. In order to achieve this
+ with sufficient accuracy, MySQL and Postgresql now contain features
+ that correct for the duplication of indexes and unique constraints
+ when reflecting tables, indexes, and constraints.
+ In the case of MySQL, there is not actually a "unique constraint"
+ concept independent of a "unique index", so for this backend
+ :class:`.UniqueConstraint` continues to remain non-present for a
+ reflected :class:`.Table`. For Postgresql, the query used to
+ detect indexes against ``pg_index`` has been improved to check for
+ the same construct in ``pg_constraint``, and the implicitly
+ constructed unique index is not included with a
+ reflected :class:`.Table`.
+
+ In both cases, the :meth:`.Inspector.get_indexes` and the
+ :meth:`.Inspector.get_unique_constraints` methods return both
+ constructs individually, but include a new token
+ ``duplicates_constraint`` in the case of Postgresql or
+ ``duplicates_index`` in the case
+ of MySQL to indicate when this condition is detected.
+ Pull request courtesy Johannes Erdfelt.
+
+ .. seealso::
+
+ :ref:`feature_3184`
+
+ .. change::
+ :tags: feature, postgresql
+ :pullreq: github:134
+
+ Added support for the FILTER keyword as applied to aggregate
+ functions, supported by Postgresql 9.4. Pull request
+ courtesy Ilja Everilä.
+
+ .. seealso::
+
+ :ref:`feature_gh134`
+
+ .. change::
+ :tags: bug, sql, engine
+ :tickets: 3215
+
+ Fixed bug where a "branched" connection, that is the kind you get
+ when you call :meth:`.Connection.connect`, would not share invalidation
+ status with the parent. The architecture of branching has been tweaked
+ a bit so that the branched connection defers to the parent for
+ all invalidation status and operations.
+
+ .. change::
+ :tags: bug, sql, engine
+ :tickets: 3190
+
+ Fixed bug where a "branched" connection, that is the kind you get
+ when you call :meth:`.Connection.connect`, would not share transaction
+ status with the parent. The architecture of branching has been tweaked
+ a bit so that the branched connection defers to the parent for
+ all transactional status and operations.
+
+ .. change::
+ :tags: bug, declarative
+ :tickets: 2670
+
+ A relationship set up with :class:`.declared_attr` on
+ a :class:`.AbstractConcreteBase` base class will now be configured
+ on the abstract base mapping automatically, in addition to being
+ set up on descendant concrete classes as usual.
+
+ .. seealso::
+
+ :ref:`feature_3150`
+
+ .. change::
+ :tags: feature, declarative
+ :tickets: 3150
+
+ The :class:`.declared_attr` construct has newly improved
+ behaviors and features in conjunction with declarative. The
+ decorated function will now have access to the final column
+ copies present on the local mixin when invoked, and will also
+ be invoked exactly once for each mapped class, the returned result
+ being memoized. A new modifier :attr:`.declared_attr.cascading`
+ is added as well.
+
+ .. seealso::
+
+ :ref:`feature_3150`
+
+ .. change::
+ :tags: feature, ext
+ :tickets: 3210
+
+ The :mod:`sqlalchemy.ext.automap` extension will now set
+ ``cascade="all, delete-orphan"`` automatically on a one-to-many
+ relationship/backref where the foreign key is detected as containing
+ one or more non-nullable columns. This argument is present in the
+ keywords passed to :func:`.automap.generate_relationship` in this
+ case and can still be overridden. Additionally, if the
+ :class:`.ForeignKeyConstraint` specifies ``ondelete="CASCADE"``
+ for a non-nullable or ``ondelete="SET NULL"`` for a nullable set
+ of columns, the argument ``passive_deletes=True`` is also added to the
+ relationship. Note that not all backends support reflection of
+ ondelete, but backends that do include Postgresql and MySQL.
+
+ .. change::
+ :tags: feature, sql
+ :tickets: 3206
+
+ Added new method :meth:`.Select.with_statement_hint` and ORM
+ method :meth:`.Query.with_statement_hint` to support statement-level
+ hints that are not specific to a table.
+
+ .. change::
+ :tags: bug, sqlite
+ :tickets: 3203
+ :pullreq: bitbucket:31
+
+ SQLite now supports reflection of unique constraints from
+ temp tables; previously, this would fail with a TypeError.
+ Pull request courtesy Johannes Erdfelt.
+
+ .. seealso::
+
+ :ref:`change_3204` - changes regarding SQLite temporary
+ table and view reflection.
+
+ .. change::
+ :tags: bug, sqlite
+ :tickets: 3204
+
+ Added :meth:`.Inspector.get_temp_table_names` and
+ :meth:`.Inspector.get_temp_view_names`; currently, only the
+ SQLite and Oracle dialects support these methods. The return of
+ temporary table and view names has been **removed** from SQLite and
+ Oracle's version of :meth:`.Inspector.get_table_names` and
+ :meth:`.Inspector.get_view_names`; other database backends cannot
+ support this information (such as MySQL), and the scope of operation
+ is different in that the tables can be local to a session and
+ typically aren't supported in remote schemas.
+
+ .. seealso::
+
+ :ref:`change_3204`
+
+ .. change::
+ :tags: feature, postgresql
+ :tickets: 2891
+ :pullreq: github:128
+
+ Support has been added for reflection of materialized views
+ and foreign tables, as well as support for materialized views
+ within :meth:`.Inspector.get_view_names`, and a new method
+ :meth:`.PGInspector.get_foreign_table_names` available on the
+ Postgresql version of :class:`.Inspector`. Pull request courtesy
+ Rodrigo Menezes.
+
+ .. seealso::
+
+ :ref:`feature_2891`
+
+
+ .. change::
:tags: feature, orm
Added new event handlers :meth:`.AttributeEvents.init_collection`
@@ -268,6 +612,11 @@
default, or a server-side default "eagerly" fetched via RETURNING.
.. change::
+ :tags: feature, oracle
+
+ Added support for the Oracle table option ON COMMIT.
+
+ .. change::
:tags: feature, postgresql
:tickets: 2051
diff --git a/doc/build/changelog/migration_10.rst b/doc/build/changelog/migration_10.rst
index 6a48b31fa..bc7fa139f 100644
--- a/doc/build/changelog/migration_10.rst
+++ b/doc/build/changelog/migration_10.rst
@@ -8,7 +8,7 @@ What's New in SQLAlchemy 1.0?
undergoing maintenance releases as of May, 2014,
and SQLAlchemy version 1.0, as of yet unreleased.
- Document last updated: September 7, 2014
+ Document last updated: October 23, 2014
Introduction
============
@@ -25,6 +25,141 @@ potentially backwards-incompatible changes.
New Features
============
+.. _feature_3150:
+
+Improvements to declarative mixins, ``@declared_attr`` and related features
+----------------------------------------------------------------------------
+
+The declarative system in conjunction with :class:`.declared_attr` has been
+overhauled to support new capabilities.
+
+A function decorated with :class:`.declared_attr` is now called only **after**
+any mixin-based column copies are generated. This means the function can
+call upon mixin-established columns and will receive a reference to the correct
+:class:`.Column` object::
+
+ class HasFooBar(object):
+ foobar = Column(Integer)
+
+ @declared_attr
+ def foobar_prop(cls):
+ return column_property('foobar: ' + cls.foobar)
+
+ class SomeClass(HasFooBar, Base):
+ __tablename__ = 'some_table'
+ id = Column(Integer, primary_key=True)
+
+Above, ``SomeClass.foobar_prop`` will be invoked against ``SomeClass``,
+and ``SomeClass.foobar`` will be the final :class:`.Column` object that is
+to be mapped to ``SomeClass``, as opposed to the non-copied object present
+directly on ``HasFooBar``, even though the columns aren't mapped yet.
+
+The :class:`.declared_attr` function now **memoizes** the value
+that's returned on a per-class basis, so that repeated calls to the same
+attribute will return the same value. We can alter the example to illustrate
+this::
+
+ class HasFooBar(object):
+ @declared_attr
+ def foobar(cls):
+ return Column(Integer)
+
+ @declared_attr
+ def foobar_prop(cls):
+ return column_property('foobar: ' + cls.foobar)
+
+ class SomeClass(HasFooBar, Base):
+ __tablename__ = 'some_table'
+ id = Column(Integer, primary_key=True)
+
+Previously, ``SomeClass`` would be mapped with one particular copy of
+the ``foobar`` column, but the ``foobar_prop`` by calling upon ``foobar``
+a second time would produce a different column. The value of
+``SomeClass.foobar`` is now memoized during declarative setup time, so that
+even before the attribute is mapped by the mapper, the interim column
+value will remain consistent no matter how many times the
+:class:`.declared_attr` is called upon.
+
+The two behaviors above should help considerably with declarative definition
+of many types of mapper properties that derive from other attributes, where
+the :class:`.declared_attr` function is called upon from other
+:class:`.declared_attr` functions locally present before the class is
+actually mapped.
+
+For a pretty slim edge case where one wishes to build a declarative mixin
+that establishes distinct columns per subclass, a new modifier
+:attr:`.declared_attr.cascading` is added. With this modifier, the
+decorated function will be invoked individually for each class in the
+mapped inheritance hierarchy. While this is already the behavior for
+special attributes such as ``__table_args__`` and ``__mapper_args__``,
+for columns and other properties the behavior by default assumes that attribute
+is affixed to the base class only, and just inherited from subclasses.
+With :attr:`.declared_attr.cascading`, individual behaviors can be
+applied::
+
+ class HasSomeAttribute(object):
+ @declared_attr.cascading
+ def some_id(cls):
+ if has_inherited_table(cls):
+ return Column(ForeignKey('myclass.id'), primary_key=True)
+ else:
+ return Column(Integer, primary_key=True)
+
+ return Column('id', Integer, primary_key=True)
+
+ class MyClass(HasSomeAttribute, Base):
+ ""
+ # ...
+
+ class MySubClass(MyClass):
+ ""
+ # ...
+
+.. seealso::
+
+ :ref:`mixin_inheritance_columns`
+
+Finally, the :class:`.AbstractConcreteBase` class has been reworked
+so that a relationship or other mapper property can be set up inline
+on the abstract base::
+
+ from sqlalchemy import Column, Integer, ForeignKey
+ from sqlalchemy.orm import relationship
+ from sqlalchemy.ext.declarative import (declarative_base, declared_attr,
+ AbstractConcreteBase)
+
+ Base = declarative_base()
+
+ class Something(Base):
+ __tablename__ = u'something'
+ id = Column(Integer, primary_key=True)
+
+
+ class Abstract(AbstractConcreteBase, Base):
+ id = Column(Integer, primary_key=True)
+
+ @declared_attr
+ def something_id(cls):
+ return Column(ForeignKey(Something.id))
+
+ @declared_attr
+ def something(cls):
+ return relationship(Something)
+
+
+ class Concrete(Abstract):
+ __tablename__ = u'cca'
+ __mapper_args__ = {'polymorphic_identity': 'cca', 'concrete': True}
+
+
+The above mapping will set up a table ``cca`` with both an ``id`` and
+a ``something_id`` column, and ``Concrete`` will also have a relationship
+``something``. The new feature is that ``Abstract`` will also have an
+independently configured relationship ``something`` that builds against
+the polymorphic union of the base.
+
+:ticket:`3150` :ticket:`2670` :ticket:`3149` :ticket:`2952` :ticket:`3050`
+
.. _feature_3034:
Select/Query LIMIT / OFFSET may be specified as an arbitrary SQL expression
@@ -37,7 +172,7 @@ any SQL expression, in addition to integer values, as arguments. The ORM
this is used to allow a bound parameter to be passed, which can be substituted
with a value later::
- sel = select([table]).limit(bindparam('mylimit')).offset(bindparam('myoffset'))
+ sel = select([table]).limit(bindparam('mylimit')).offset(bindparam('myoffset'))
Dialects which don't support non-integer LIMIT or OFFSET expressions may continue
to not support this behavior; third party dialects may also need modification
@@ -50,6 +185,170 @@ wishes to support the new feature should now call upon the ``._limit_clause``
and ``._offset_clause`` attributes to receive the full SQL expression, rather
than the integer value.
+.. _change_2051:
+
+.. _feature_insert_from_select_defaults:
+
+INSERT FROM SELECT now includes Python and SQL-expression defaults
+-------------------------------------------------------------------
+
+:meth:`.Insert.from_select` now includes Python and SQL-expression defaults if
+otherwise unspecified; the limitation where non-server column defaults
+aren't included in an INSERT FROM SELECT is now lifted and these
+expressions are rendered as constants into the SELECT statement::
+
+ from sqlalchemy import Table, Column, MetaData, Integer, select, func
+
+ m = MetaData()
+
+ t = Table(
+ 't', m,
+ Column('x', Integer),
+ Column('y', Integer, default=func.somefunction()))
+
+ stmt = select([t.c.x])
+ print t.insert().from_select(['x'], stmt)
+
+Will render::
+
+ INSERT INTO t (x, y) SELECT t.x, somefunction() AS somefunction_1
+ FROM t
+
+The feature can be disabled using
+:paramref:`.Insert.from_select.include_defaults`.
+
+New Postgresql Table options
+-----------------------------
+
+Added support for PG table options TABLESPACE, ON COMMIT,
+WITH(OUT) OIDS, and INHERITS, when rendering DDL via
+the :class:`.Table` construct.
+
+.. seealso::
+
+ :ref:`postgresql_table_options`
+
+:ticket:`2051`
+
+.. _feature_get_enums:
+
+New get_enums() method with Postgresql Dialect
+----------------------------------------------
+
+The :func:`.inspect` method returns a :class:`.PGInspector` object in the
+case of Postgresql, which includes a new :meth:`.PGInspector.get_enums`
+method that returns information on all available ``ENUM`` types::
+
+ from sqlalchemy import inspect, create_engine
+
+ engine = create_engine("postgresql+psycopg2://host/dbname")
+ insp = inspect(engine)
+ print(insp.get_enums())
+
+.. seealso::
+
+ :meth:`.PGInspector.get_enums`
+
+.. _feature_2891:
+
+Postgresql Dialect reflects Materialized Views, Foreign Tables
+--------------------------------------------------------------
+
+Changes are as follows:
+
+* the :class:`Table` construct with ``autoload=True`` will now match a name
+ that exists in the database as a materialized view or foriegn table.
+
+* :meth:`.Inspector.get_view_names` will return plain and materialized view
+ names.
+
+* :meth:`.Inspector.get_table_names` does **not** change for Postgresql, it
+ continues to return only the names of plain tables.
+
+* A new method :meth:`.PGInspector.get_foreign_table_names` is added which
+ will return the names of tables that are specifically marked as "foreign"
+ in the Postgresql schema tables.
+
+The change to reflection involves adding ``'m'`` and ``'f'`` to the list
+of qualifiers we use when querying ``pg_class.relkind``, but this change
+is new in 1.0.0 to avoid any backwards-incompatible surprises for those
+running 0.9 in production.
+
+:ticket:`2891`
+
+.. _feature_gh134:
+
+Postgresql FILTER keyword
+-------------------------
+
+The SQL standard FILTER keyword for aggregate functions is now supported
+by Postgresql as of 9.4. SQLAlchemy allows this using
+:meth:`.FunctionElement.filter`::
+
+ func.count(1).filter(True)
+
+.. seealso::
+
+ :meth:`.FunctionElement.filter`
+
+ :class:`.FunctionFilter`
+
+.. _feature_3184:
+
+UniqueConstraint is now part of the Table reflection process
+------------------------------------------------------------
+
+A :class:`.Table` object populated using ``autoload=True`` will now
+include :class:`.UniqueConstraint` constructs as well as
+:class:`.Index` constructs. This logic has a few caveats for
+Postgresql and Mysql:
+
+Postgresql
+^^^^^^^^^^
+
+Postgresql has the behavior such that when a UNIQUE constraint is
+created, it implicitly creates a UNIQUE INDEX corresponding to that
+constraint as well. The :meth:`.Inspector.get_indexes` and the
+:meth:`.Inspector.get_unique_constraints` methods will continue to
+**both** return these entries distinctly, where
+:meth:`.Inspector.get_indexes` now features a token
+``duplicates_constraint`` within the index entry indicating the
+corresponding constraint when detected. However, when performing
+full table reflection using ``Table(..., autoload=True)``, the
+:class:`.Index` construct is detected as being linked to the
+:class:`.UniqueConstraint`, and is **not** present within the
+:attr:`.Table.indexes` collection; only the :class:`.UniqueConstraint`
+will be present in the :attr:`.Table.constraints` collection. This
+deduplication logic works by joining to the ``pg_constraint`` table
+when querying ``pg_index`` to see if the two constructs are linked.
+
+MySQL
+^^^^^
+
+MySQL does not have separate concepts for a UNIQUE INDEX and a UNIQUE
+constraint. While it supports both syntaxes when creating tables and indexes,
+it does not store them any differently. The
+:meth:`.Inspector.get_indexes`
+and the :meth:`.Inspector.get_unique_constraints` methods will continue to
+**both** return an entry for a UNIQUE index in MySQL,
+where :meth:`.Inspector.get_unique_constraints` features a new token
+``duplicates_index`` within the constraint entry indicating that this is a
+dupe entry corresponding to that index. However, when performing
+full table reflection using ``Table(..., autoload=True)``,
+the :class:`.UniqueConstraint` construct is
+**not** part of the fully reflected :class:`.Table` construct under any
+circumstances; this construct is always represented by a :class:`.Index`
+with the ``unique=True`` setting present in the :attr:`.Table.indexes`
+collection.
+
+.. seealso::
+
+ :ref:`postgresql_index_reflection`
+
+ :ref:`mysql_unique_constraints`
+
+:ticket:`3184`
+
Behavioral Improvements
=======================
@@ -82,35 +381,35 @@ that a raw load of rows now populates ORM-based objects around 25% faster.
Assuming a 1M row table, a script like the following illustrates the type
of load that's improved the most::
- import time
- from sqlalchemy import Integer, Column, create_engine, Table
- from sqlalchemy.orm import Session
- from sqlalchemy.ext.declarative import declarative_base
+ import time
+ from sqlalchemy import Integer, Column, create_engine, Table
+ from sqlalchemy.orm import Session
+ from sqlalchemy.ext.declarative import declarative_base
- Base = declarative_base()
+ Base = declarative_base()
- class Foo(Base):
- __table__ = Table(
- 'foo', Base.metadata,
- Column('id', Integer, primary_key=True),
- Column('a', Integer(), nullable=False),
- Column('b', Integer(), nullable=False),
- Column('c', Integer(), nullable=False),
- )
+ class Foo(Base):
+ __table__ = Table(
+ 'foo', Base.metadata,
+ Column('id', Integer, primary_key=True),
+ Column('a', Integer(), nullable=False),
+ Column('b', Integer(), nullable=False),
+ Column('c', Integer(), nullable=False),
+ )
- engine = create_engine(
- 'mysql+mysqldb://scott:tiger@localhost/test', echo=True)
+ engine = create_engine(
+ 'mysql+mysqldb://scott:tiger@localhost/test', echo=True)
- sess = Session(engine)
+ sess = Session(engine)
- now = time.time()
+ now = time.time()
- # avoid using all() so that we don't have the overhead of building
- # a large list of full objects in memory
- for obj in sess.query(Foo).yield_per(100).limit(1000000):
- pass
+ # avoid using all() so that we don't have the overhead of building
+ # a large list of full objects in memory
+ for obj in sess.query(Foo).yield_per(100).limit(1000000):
+ pass
- print("Total time: %d" % (time.time() - now))
+ print("Total time: %d" % (time.time() - now))
Local MacBookPro results bench from 19 seconds for 0.9 down to 14 seconds for
1.0. The :meth:`.Query.yield_per` call is always a good idea when batching
@@ -121,7 +420,6 @@ MacBookPro is 31 seconds on 0.9 and 26 seconds on 1.0, the extra time spent
setting up very large memory buffers.
-
.. _feature_3176:
New KeyedTuple implementation dramatically faster
@@ -130,7 +428,7 @@ New KeyedTuple implementation dramatically faster
We took a look into the :class:`.KeyedTuple` implementation in the hopes
of improving queries like this::
- rows = sess.query(Foo.a, Foo.b, Foo.c).all()
+ rows = sess.query(Foo.a, Foo.b, Foo.c).all()
The :class:`.KeyedTuple` class is used rather than Python's
``collections.namedtuple()``, because the latter has a very complex
@@ -146,30 +444,73 @@ which scenario. In the "sweet spot", where we are both creating a good number
of new types as well as fetching a good number of rows, the lightweight
object totally smokes both namedtuple and KeyedTuple::
- -----------------
- size=10 num=10000 # few rows, lots of queries
- namedtuple: 3.60302400589 # namedtuple falls over
- keyedtuple: 0.255059957504 # KeyedTuple very fast
- lw keyed tuple: 0.582715034485 # lw keyed trails right on KeyedTuple
- -----------------
- size=100 num=1000 # <--- sweet spot
- namedtuple: 0.365247011185
- keyedtuple: 0.24896979332
- lw keyed tuple: 0.0889317989349 # lw keyed blows both away!
- -----------------
- size=10000 num=100
- namedtuple: 0.572599887848
- keyedtuple: 2.54251694679
- lw keyed tuple: 0.613876104355
- -----------------
- size=1000000 num=10 # few queries, lots of rows
- namedtuple: 5.79669594765 # namedtuple very fast
- keyedtuple: 28.856498003 # KeyedTuple falls over
- lw keyed tuple: 6.74346804619 # lw keyed trails right on namedtuple
+ -----------------
+ size=10 num=10000 # few rows, lots of queries
+ namedtuple: 3.60302400589 # namedtuple falls over
+ keyedtuple: 0.255059957504 # KeyedTuple very fast
+ lw keyed tuple: 0.582715034485 # lw keyed trails right on KeyedTuple
+ -----------------
+ size=100 num=1000 # <--- sweet spot
+ namedtuple: 0.365247011185
+ keyedtuple: 0.24896979332
+ lw keyed tuple: 0.0889317989349 # lw keyed blows both away!
+ -----------------
+ size=10000 num=100
+ namedtuple: 0.572599887848
+ keyedtuple: 2.54251694679
+ lw keyed tuple: 0.613876104355
+ -----------------
+ size=1000000 num=10 # few queries, lots of rows
+ namedtuple: 5.79669594765 # namedtuple very fast
+ keyedtuple: 28.856498003 # KeyedTuple falls over
+ lw keyed tuple: 6.74346804619 # lw keyed trails right on namedtuple
:ticket:`3176`
+.. _bug_3035:
+
+Session.get_bind() handles a wider variety of inheritance scenarios
+-------------------------------------------------------------------
+
+The :meth:`.Session.get_bind` method is invoked whenever a query or unit
+of work flush process seeks to locate the database engine that corresponds
+to a particular class. The method has been improved to handle a variety
+of inheritance-oriented scenarios, including:
+
+* Binding to a Mixin or Abstract Class::
+
+ class MyClass(SomeMixin, Base):
+ __tablename__ = 'my_table'
+ # ...
+
+ session = Session(binds={SomeMixin: some_engine})
+
+
+* Binding to inherited concrete subclasses individually based on table::
+
+ class BaseClass(Base):
+ __tablename__ = 'base'
+
+ # ...
+
+ class ConcreteSubClass(BaseClass):
+ __tablename__ = 'concrete'
+
+ # ...
+
+ __mapper_args__ = {'concrete': True}
+
+
+ session = Session(binds={
+ base_table: some_engine,
+ concrete_table: some_other_engine
+ })
+
+
+:ticket:`3035`
+
+
.. _feature_3178:
New systems to safely emit parameterized warnings
@@ -195,27 +536,27 @@ them as duplicates.
To illustrate, the following test script will show only ten warnings being
emitted for ten of the parameter sets, out of a total of 1000::
- from sqlalchemy import create_engine, Unicode, select, cast
- import random
- import warnings
+ from sqlalchemy import create_engine, Unicode, select, cast
+ import random
+ import warnings
- e = create_engine("sqlite://")
+ e = create_engine("sqlite://")
- # Use the "once" filter (which is also the default for Python
- # warnings). Exactly ten of these warnings will
- # be emitted; beyond that, the Python warnings registry will accumulate
- # new values as dupes of one of the ten existing.
- warnings.filterwarnings("once")
+ # Use the "once" filter (which is also the default for Python
+ # warnings). Exactly ten of these warnings will
+ # be emitted; beyond that, the Python warnings registry will accumulate
+ # new values as dupes of one of the ten existing.
+ warnings.filterwarnings("once")
- for i in range(1000):
- e.execute(select([cast(
- ('foo_%d' % random.randint(0, 1000000)).encode('ascii'), Unicode)]))
+ for i in range(1000):
+ e.execute(select([cast(
+ ('foo_%d' % random.randint(0, 1000000)).encode('ascii'), Unicode)]))
The format of the warning here is::
- /path/lib/sqlalchemy/sql/sqltypes.py:186: SAWarning: Unicode type received
- non-unicode bind param value 'foo_4852'. (this warning may be
- suppressed after 10 occurrences)
+ /path/lib/sqlalchemy/sql/sqltypes.py:186: SAWarning: Unicode type received
+ non-unicode bind param value 'foo_4852'. (this warning may be
+ suppressed after 10 occurrences)
:ticket:`3178`
@@ -233,15 +574,15 @@ However, as these objects are class-bound descriptors, they must be accessed
at the attribute. Below this is illustared using the
:attr:`.Mapper.all_orm_descriptors` namespace::
- class SomeObject(Base):
- # ...
+ class SomeObject(Base):
+ # ...
- @hybrid_property
- def some_prop(self):
- return self.value + 5
+ @hybrid_property
+ def some_prop(self):
+ return self.value + 5
- inspect(SomeObject).all_orm_descriptors.some_prop.info['foo'] = 'bar'
+ inspect(SomeObject).all_orm_descriptors.some_prop.info['foo'] = 'bar'
It is also available as a constructor argument for all :class:`.SchemaItem`
objects (e.g. :class:`.ForeignKey`, :class:`.UniqueConstraint` etc.) as well
@@ -258,26 +599,26 @@ Change to single-table-inheritance criteria when using from_self(), count()
Given a single-table inheritance mapping, such as::
- class Widget(Base):
- __table__ = 'widget_table'
+ class Widget(Base):
+ __table__ = 'widget_table'
- class FooWidget(Widget):
- pass
+ class FooWidget(Widget):
+ pass
Using :meth:`.Query.from_self` or :meth:`.Query.count` against a subclass
would produce a subquery, but then add the "WHERE" criteria for subtypes
to the outside::
- sess.query(FooWidget).from_self().all()
+ sess.query(FooWidget).from_self().all()
rendering::
- SELECT
- anon_1.widgets_id AS anon_1_widgets_id,
- anon_1.widgets_type AS anon_1_widgets_type
- FROM (SELECT widgets.id AS widgets_id, widgets.type AS widgets_type,
- FROM widgets) AS anon_1
- WHERE anon_1.widgets_type IN (?)
+ SELECT
+ anon_1.widgets_id AS anon_1_widgets_id,
+ anon_1.widgets_type AS anon_1_widgets_type
+ FROM (SELECT widgets.id AS widgets_id, widgets.type AS widgets_type,
+ FROM widgets) AS anon_1
+ WHERE anon_1.widgets_type IN (?)
The issue with this is that if the inner query does not specify all
columns, then we can't add the WHERE clause on the outside (it actually tries,
@@ -286,28 +627,93 @@ apparently goes way back to 0.6.5 with the note "may need to make more
adjustments to this". Well, those adjustments have arrived! So now the
above query will render::
- SELECT
- anon_1.widgets_id AS anon_1_widgets_id,
- anon_1.widgets_type AS anon_1_widgets_type
- FROM (SELECT widgets.id AS widgets_id, widgets.type AS widgets_type,
- FROM widgets
- WHERE widgets.type IN (?)) AS anon_1
+ SELECT
+ anon_1.widgets_id AS anon_1_widgets_id,
+ anon_1.widgets_type AS anon_1_widgets_type
+ FROM (SELECT widgets.id AS widgets_id, widgets.type AS widgets_type,
+ FROM widgets
+ WHERE widgets.type IN (?)) AS anon_1
So that queries that don't include "type" will still work!::
- sess.query(FooWidget.id).count()
+ sess.query(FooWidget.id).count()
Renders::
- SELECT count(*) AS count_1
- FROM (SELECT widgets.id AS widgets_id
- FROM widgets
- WHERE widgets.type IN (?)) AS anon_1
+ SELECT count(*) AS count_1
+ FROM (SELECT widgets.id AS widgets_id
+ FROM widgets
+ WHERE widgets.type IN (?)) AS anon_1
:ticket:`3177`
+.. _migration_3222:
+
+
+single-table-inheritance criteria added to all ON clauses unconditionally
+-------------------------------------------------------------------------
+
+When joining to a single-table inheritance subclass target, the ORM always adds
+the "single table criteria" when joining on a relationship. Given a
+mapping as::
+
+ class Widget(Base):
+ __tablename__ = 'widget'
+ id = Column(Integer, primary_key=True)
+ type = Column(String)
+ related_id = Column(ForeignKey('related.id'))
+ related = relationship("Related", backref="widget")
+ __mapper_args__ = {'polymorphic_on': type}
+
+
+ class FooWidget(Widget):
+ __mapper_args__ = {'polymorphic_identity': 'foo'}
+
+
+ class Related(Base):
+ __tablename__ = 'related'
+ id = Column(Integer, primary_key=True)
+
+It's been the behavior for quite some time that a JOIN on the relationship
+will render a "single inheritance" clause for the type::
+
+ s.query(Related).join(FooWidget, Related.widget).all()
+
+SQL output::
+
+ SELECT related.id AS related_id
+ FROM related JOIN widget ON related.id = widget.related_id AND widget.type IN (:type_1)
+
+Above, because we joined to a subclass ``FooWidget``, :meth:`.Query.join`
+knew to add the ``AND widget.type IN ('foo')`` criteria to the ON clause.
+
+The change here is that the ``AND widget.type IN()`` criteria is now appended
+to *any* ON clause, not just those generated from a relationship,
+including one that is explicitly stated::
+
+ # ON clause will now render as
+ # related.id = widget.related_id AND widget.type IN (:type_1)
+ s.query(Related).join(FooWidget, FooWidget.related_id == Related.id).all()
+
+As well as the "implicit" join when no ON clause of any kind is stated::
+
+ # ON clause will now render as
+ # related.id = widget.related_id AND widget.type IN (:type_1)
+ s.query(Related).join(FooWidget).all()
+
+Previously, the ON clause for these would not include the single-inheritance
+criteria. Applications that are already adding this criteria to work around
+this will want to remove its explicit use, though it should continue to work
+fine if the criteria happens to be rendered twice in the meantime.
+
+.. seealso::
+
+ :ref:`bug_3233`
+
+:ticket:`3222`
+
.. _bug_3188:
ColumnProperty constructs work a lot better with aliases, order_by
@@ -319,67 +725,67 @@ as the "order by label" logic introduced in 0.9 (see :ref:`migration_1068`).
Given a mapping like the following::
- class A(Base):
- __tablename__ = 'a'
+ class A(Base):
+ __tablename__ = 'a'
- id = Column(Integer, primary_key=True)
+ id = Column(Integer, primary_key=True)
- class B(Base):
- __tablename__ = 'b'
+ class B(Base):
+ __tablename__ = 'b'
- id = Column(Integer, primary_key=True)
- a_id = Column(ForeignKey('a.id'))
+ id = Column(Integer, primary_key=True)
+ a_id = Column(ForeignKey('a.id'))
- A.b = column_property(
- select([func.max(B.id)]).where(B.a_id == A.id).correlate(A)
- )
+ A.b = column_property(
+ select([func.max(B.id)]).where(B.a_id == A.id).correlate(A)
+ )
A simple scenario that included "A.b" twice would fail to render
correctly::
- print sess.query(A, a1).order_by(a1.b)
+ print sess.query(A, a1).order_by(a1.b)
This would order by the wrong column::
- SELECT a.id AS a_id, (SELECT max(b.id) AS max_1 FROM b
- WHERE b.a_id = a.id) AS anon_1, a_1.id AS a_1_id,
- (SELECT max(b.id) AS max_2
- FROM b WHERE b.a_id = a_1.id) AS anon_2
- FROM a, a AS a_1 ORDER BY anon_1
+ SELECT a.id AS a_id, (SELECT max(b.id) AS max_1 FROM b
+ WHERE b.a_id = a.id) AS anon_1, a_1.id AS a_1_id,
+ (SELECT max(b.id) AS max_2
+ FROM b WHERE b.a_id = a_1.id) AS anon_2
+ FROM a, a AS a_1 ORDER BY anon_1
New output::
- SELECT a.id AS a_id, (SELECT max(b.id) AS max_1
- FROM b WHERE b.a_id = a.id) AS anon_1, a_1.id AS a_1_id,
- (SELECT max(b.id) AS max_2
- FROM b WHERE b.a_id = a_1.id) AS anon_2
- FROM a, a AS a_1 ORDER BY anon_2
+ SELECT a.id AS a_id, (SELECT max(b.id) AS max_1
+ FROM b WHERE b.a_id = a.id) AS anon_1, a_1.id AS a_1_id,
+ (SELECT max(b.id) AS max_2
+ FROM b WHERE b.a_id = a_1.id) AS anon_2
+ FROM a, a AS a_1 ORDER BY anon_2
There were also many scenarios where the "order by" logic would fail
to order by label, for example if the mapping were "polymorphic"::
- class A(Base):
- __tablename__ = 'a'
+ class A(Base):
+ __tablename__ = 'a'
- id = Column(Integer, primary_key=True)
- type = Column(String)
+ id = Column(Integer, primary_key=True)
+ type = Column(String)
- __mapper_args__ = {'polymorphic_on': type, 'with_polymorphic': '*'}
+ __mapper_args__ = {'polymorphic_on': type, 'with_polymorphic': '*'}
The order_by would fail to use the label, as it would be anonymized due
to the polymorphic loading::
- SELECT a.id AS a_id, a.type AS a_type, (SELECT max(b.id) AS max_1
- FROM b WHERE b.a_id = a.id) AS anon_1
- FROM a ORDER BY (SELECT max(b.id) AS max_2
- FROM b WHERE b.a_id = a.id)
+ SELECT a.id AS a_id, a.type AS a_type, (SELECT max(b.id) AS max_1
+ FROM b WHERE b.a_id = a.id) AS anon_1
+ FROM a ORDER BY (SELECT max(b.id) AS max_2
+ FROM b WHERE b.a_id = a.id)
Now that the order by label tracks the anonymized label, this now works::
- SELECT a.id AS a_id, a.type AS a_type, (SELECT max(b.id) AS max_1
- FROM b WHERE b.a_id = a.id) AS anon_1
- FROM a ORDER BY anon_1
+ SELECT a.id AS a_id, a.type AS a_type, (SELECT max(b.id) AS max_1
+ FROM b WHERE b.a_id = a.id) AS anon_1
+ FROM a ORDER BY anon_1
Included in these fixes are a variety of heisenbugs that could corrupt
the state of an ``aliased()`` construct such that the labeling logic
@@ -387,11 +793,88 @@ would again fail; these have also been fixed.
:ticket:`3148` :ticket:`3188`
+.. _bug_3170:
+
+null(), false() and true() constants are no longer singletons
+-------------------------------------------------------------
+
+These three constants were changed to return a "singleton" value
+in 0.9; unfortunately, that would lead to a query like the following
+to not render as expected::
+
+ select([null(), null()])
+
+rendering only ``SELECT NULL AS anon_1``, because the two :func:`.null`
+constructs would come out as the same ``NULL`` object, and
+SQLAlchemy's Core model is based on object identity in order to
+determine lexical significance. The change in 0.9 had no
+importance other than the desire to save on object overhead; in general,
+an unnamed construct needs to stay lexically unique so that it gets
+labeled uniquely.
+
+:ticket:`3170`
+
.. _behavioral_changes_orm_10:
Behavioral Changes - ORM
========================
+.. _bug_3228:
+
+query.update() now resolves string names into mapped attribute names
+--------------------------------------------------------------------
+
+The documentation for :meth:`.Query.update` states that the given
+``values`` dictionary is "a dictionary with attributes names as keys",
+implying that these are mapped attribute names. Unfortunately, the function
+was designed more in mind to receive attributes and SQL expressions and
+not as much strings; when strings
+were passed, these strings would be passed through straight to the core
+update statement without any resolution as far as how these names are
+represented on the mapped class, meaning the name would have to match that
+of a table column exactly, not how an attribute of that name was mapped
+onto the class.
+
+The string names are now resolved as attribute names in earnest::
+
+ class User(Base):
+ __tablename__ = 'user'
+
+ id = Column(Integer, primary_key=True)
+ name = Column('user_name', String(50))
+
+Above, the column ``user_name`` is mapped as ``name``. Previously,
+a call to :meth:`.Query.update` that was passed strings would have to
+have been called as follows::
+
+ session.query(User).update({'user_name': 'moonbeam'})
+
+The given string is now resolved against the entity::
+
+ session.query(User).update({'name': 'moonbeam'})
+
+It is typically preferable to use the attribute directly, to avoid any
+ambiguity::
+
+ session.query(User).update({User.name: 'moonbeam'})
+
+The change also indicates that synonyms and hybrid attributes can be referred
+to by string name as well::
+
+ class User(Base):
+ __tablename__ = 'user'
+
+ id = Column(Integer, primary_key=True)
+ name = Column('user_name', String(50))
+
+ @hybrid_property
+ def fullname(self):
+ return self.name
+
+ session.query(User).update({'fullname': 'moonbeam'})
+
+:ticket:`3228`
+
.. _migration_3061:
Changes to attribute events and other operations regarding attributes that have no pre-existing value
@@ -406,13 +889,13 @@ for :func:`.attributes.get_history` and related functions.
Given an object with no state::
- >>> obj = Foo()
+ >>> obj = Foo()
It has always been SQLAlchemy's behavior such that if we access a scalar
or many-to-one attribute that was never set, it is returned as ``None``::
- >>> obj.someattr
- None
+ >>> obj.someattr
+ None
This value of ``None`` is in fact now part of the state of ``obj``, and is
not unlike as though we had set the attribute explicitly, e.g.
@@ -420,31 +903,31 @@ not unlike as though we had set the attribute explicitly, e.g.
differently as far as history and events. It would not emit any attribute
event, and additionally if we view history, we see this::
- >>> inspect(obj).attrs.someattr.history
- History(added=(), unchanged=[None], deleted=()) # 0.9 and below
+ >>> inspect(obj).attrs.someattr.history
+ History(added=(), unchanged=[None], deleted=()) # 0.9 and below
That is, it's as though the attribute were always ``None`` and were
never changed. This is explicitly different from if we had set the
attribute first instead::
- >>> obj = Foo()
- >>> obj.someattr = None
- >>> inspect(obj).attrs.someattr.history
- History(added=[None], unchanged=(), deleted=()) # all versions
+ >>> obj = Foo()
+ >>> obj.someattr = None
+ >>> inspect(obj).attrs.someattr.history
+ History(added=[None], unchanged=(), deleted=()) # all versions
The above means that the behavior of our "set" operation can be corrupted
by the fact that the value was accessed via "get" earlier. In 1.0, this
inconsistency has been resolved, by no longer actually setting anything
when the default "getter" is used.
- >>> obj = Foo()
- >>> obj.someattr
- None
- >>> inspect(obj).attrs.someattr.history
- History(added=(), unchanged=(), deleted=()) # 1.0
- >>> obj.someattr = None
- >>> inspect(obj).attrs.someattr.history
- History(added=[None], unchanged=(), deleted=())
+ >>> obj = Foo()
+ >>> obj.someattr
+ None
+ >>> inspect(obj).attrs.someattr.history
+ History(added=(), unchanged=(), deleted=()) # 1.0
+ >>> obj.someattr = None
+ >>> inspect(obj).attrs.someattr.history
+ History(added=[None], unchanged=(), deleted=())
The reason the above behavior hasn't had much impact is because the
INSERT statement in relational databases considers a missing value to be
@@ -469,6 +952,39 @@ symbol, and no change to the object's state occurs.
:ticket:`3061`
+.. _bug_3139:
+
+session.expunge() will fully detach an object that's been deleted
+-----------------------------------------------------------------
+
+The behavior of :meth:`.Session.expunge` had a bug that caused an
+inconsistency in behavior regarding deleted objects. The
+:func:`.object_session` function as well as the :attr:`.InstanceState.session`
+attribute would still report object as belonging to the :class:`.Session`
+subsequent to the expunge::
+
+ u1 = sess.query(User).first()
+ sess.delete(u1)
+
+ sess.flush()
+
+ assert u1 not in sess
+ assert inspect(u1).session is sess # this is normal before commit
+
+ sess.expunge(u1)
+
+ assert u1 not in sess
+ assert inspect(u1).session is None # would fail
+
+Note that it is normal for ``u1 not in sess`` to be True while
+``inspect(u1).session`` still refers to the session, while the transaction
+is ongoing subsequent to the delete operation and :meth:`.Session.expunge`
+has not been called; the full detachment normally completes once the
+transaction is committed. This issue would also impact functions
+that rely on :meth:`.Session.expunge` such as :func:`.make_transient`.
+
+:ticket:`3139`
+
.. _migration_yield_per_eager_loading:
Joined/Subquery eager loading explicitly disallowed with yield_per
@@ -482,17 +998,102 @@ with yield-per (subquery loading could be in theory, however).
When this error is raised, the :func:`.lazyload` option can be sent with
an asterisk::
- q = sess.query(Object).options(lazyload('*')).yield_per(100)
+ q = sess.query(Object).options(lazyload('*')).yield_per(100)
or use :meth:`.Query.enable_eagerloads`::
- q = sess.query(Object).enable_eagerloads(False).yield_per(100)
+ q = sess.query(Object).enable_eagerloads(False).yield_per(100)
The :func:`.lazyload` option has the advantage that additional many-to-one
joined loader options can still be used::
- q = sess.query(Object).options(
- lazyload('*'), joinedload("some_manytoone")).yield_per(100)
+ q = sess.query(Object).options(
+ lazyload('*'), joinedload("some_manytoone")).yield_per(100)
+
+.. _bug_3233:
+
+Single inheritance join targets will no longer sometimes implicitly alias themselves
+------------------------------------------------------------------------------------
+
+This is a bug where an unexpected and inconsistent behavior would occur
+in some scenarios when joining to a single-table-inheritance entity. The
+difficulty this might cause is that the query is supposed to raise an error,
+as it is invalid SQL, however the bug would cause an alias to be added which
+makes the query "work". The issue is confusing because this aliasing
+is not applied consistently and could change based on the nature of the query
+preceding the join.
+
+A simple example is::
+
+ from sqlalchemy import Integer, Column, String, ForeignKey
+ from sqlalchemy.orm import Session, relationship
+ from sqlalchemy.ext.declarative import declarative_base
+
+ Base = declarative_base()
+
+ class A(Base):
+ __tablename__ = "a"
+
+ id = Column(Integer, primary_key=True)
+ type = Column(String)
+
+ __mapper_args__ = {'polymorphic_on': type, 'polymorphic_identity': 'a'}
+
+
+ class ASub1(A):
+ __mapper_args__ = {'polymorphic_identity': 'asub1'}
+
+
+ class ASub2(A):
+ __mapper_args__ = {'polymorphic_identity': 'asub2'}
+
+
+ class B(Base):
+ __tablename__ = 'b'
+
+ id = Column(Integer, primary_key=True)
+
+ a_id = Column(Integer, ForeignKey("a.id"))
+
+ a = relationship("A", primaryjoin="B.a_id == A.id", backref='b')
+
+ s = Session()
+
+ print s.query(ASub1).join(B, ASub1.b).join(ASub2, B.a)
+
+ print s.query(ASub1).join(B, ASub1.b).join(ASub2, ASub2.id == B.a_id)
+
+The two queries at the bottom are equivalent, and should both render
+the identical SQL::
+
+ SELECT a.id AS a_id, a.type AS a_type
+ FROM a JOIN b ON b.a_id = a.id JOIN a ON b.a_id = a.id AND a.type IN (:type_1)
+ WHERE a.type IN (:type_2)
+
+The above SQL is invalid, as it renders "a" within the FROM list twice.
+The bug however would occur with the second query only and render this instead::
+
+ SELECT a.id AS a_id, a.type AS a_type
+ FROM a JOIN b ON b.a_id = a.id JOIN a AS a_1
+ ON a_1.id = b.a_id AND a_1.type IN (:type_1)
+ WHERE a_1.type IN (:type_2)
+
+Where above, the second join to "a" is aliased. While this seems convenient,
+it's not how single-inheritance queries work in general and is misleading
+and inconsistent.
+
+The net effect is that applications which were relying on this bug will now
+have an error raised by the database. The solution is to use the expected
+form. When referring to multiple subclasses of a single-inheritance
+entity in a query, you must manually use aliases to disambiguate the table,
+as all the subclasses normally refer to the same table::
+
+ asub2_alias = aliased(ASub2)
+
+ print s.query(ASub1).join(B, ASub1.b).join(asub2_alias, B.a.of_type(asub2_alias))
+
+:ticket:`3233`
+
.. _migration_migration_deprecated_orm_events:
@@ -546,7 +1147,7 @@ The unused ``result`` member is now removed::
.. seealso::
- :ref:`bundles`
+ :ref:`bundles`
.. _migration_3008:
@@ -565,12 +1166,12 @@ As introduced in :ref:`feature_2976` from version 0.9, the behavior of
join eager load will use a right-nested join. ``"nested"`` is now implied
when using ``innerjoin=True``::
- query(User).options(
- joinedload("orders", innerjoin=False).joinedload("items", innerjoin=True))
+ query(User).options(
+ joinedload("orders", innerjoin=False).joinedload("items", innerjoin=True))
With the new default, this will render the FROM clause in the form::
- FROM users LEFT OUTER JOIN (orders JOIN items ON <onclause>) ON <onclause>
+ FROM users LEFT OUTER JOIN (orders JOIN items ON <onclause>) ON <onclause>
That is, using a right-nested join for the INNER join so that the full
result of ``users`` can be returned. The use of an INNER join is more efficient
@@ -579,13 +1180,13 @@ optimization parameter to take effect in all cases.
To get the older behavior, use ``innerjoin="unnested"``::
- query(User).options(
- joinedload("orders", innerjoin=False).joinedload("items", innerjoin="unnested"))
+ query(User).options(
+ joinedload("orders", innerjoin=False).joinedload("items", innerjoin="unnested"))
This will avoid right-nested joins and chain the joins together using all
OUTER joins despite the innerjoin directive::
- FROM users LEFT OUTER JOIN orders ON <onclause> LEFT OUTER JOIN items ON <onclause>
+ FROM users LEFT OUTER JOIN orders ON <onclause> LEFT OUTER JOIN items ON <onclause>
As noted in the 0.9 notes, the only database backend that has difficulty
with right-nested joins is SQLite; SQLAlchemy as of 0.9 converts a right-nested
@@ -593,7 +1194,7 @@ join into a subquery as a join target on SQLite.
.. seealso::
- :ref:`feature_2976` - description of the feature as introduced in 0.9.4.
+ :ref:`feature_2976` - description of the feature as introduced in 0.9.4.
:ticket:`3008`
@@ -638,15 +1239,15 @@ with SQL expressions into many functions, such as :meth:`.Select.where`,
Note that by "SQL expressions" we mean a **full fragment of a SQL string**,
such as::
- # the argument sent to where() is a full SQL expression
- stmt = select([sometable]).where("somecolumn = 'value'")
+ # the argument sent to where() is a full SQL expression
+ stmt = select([sometable]).where("somecolumn = 'value'")
and we are **not talking about string arguments**, that is, the normal
behavior of passing string values that become parameterized::
- # This is a normal Core expression with a string argument -
- # we aren't talking about this!!
- stmt = select([sometable]).where(sometable.c.somecolumn == 'value')
+ # This is a normal Core expression with a string argument -
+ # we aren't talking about this!!
+ stmt = select([sometable]).where(sometable.c.somecolumn == 'value')
The Core tutorial has long featured an example of the use of this technique,
using a :func:`.select` construct where virtually all components of it
@@ -660,25 +1261,25 @@ So the change here is to encourage the user to qualify textual strings when
composing SQL that is partially or fully composed from textual fragments.
When composing a select as below::
- stmt = select(["a", "b"]).where("a = b").select_from("sometable")
+ stmt = select(["a", "b"]).where("a = b").select_from("sometable")
The statement is built up normally, with all the same coercions as before.
However, one will see the following warnings emitted::
- SAWarning: Textual column expression 'a' should be explicitly declared
- with text('a'), or use column('a') for more specificity
- (this warning may be suppressed after 10 occurrences)
+ SAWarning: Textual column expression 'a' should be explicitly declared
+ with text('a'), or use column('a') for more specificity
+ (this warning may be suppressed after 10 occurrences)
- SAWarning: Textual column expression 'b' should be explicitly declared
- with text('b'), or use column('b') for more specificity
- (this warning may be suppressed after 10 occurrences)
+ SAWarning: Textual column expression 'b' should be explicitly declared
+ with text('b'), or use column('b') for more specificity
+ (this warning may be suppressed after 10 occurrences)
- SAWarning: Textual SQL expression 'a = b' should be explicitly declared
- as text('a = b') (this warning may be suppressed after 10 occurrences)
+ SAWarning: Textual SQL expression 'a = b' should be explicitly declared
+ as text('a = b') (this warning may be suppressed after 10 occurrences)
- SAWarning: Textual SQL FROM expression 'sometable' should be explicitly
- declared as text('sometable'), or use table('sometable') for more
- specificity (this warning may be suppressed after 10 occurrences)
+ SAWarning: Textual SQL FROM expression 'sometable' should be explicitly
+ declared as text('sometable'), or use table('sometable') for more
+ specificity (this warning may be suppressed after 10 occurrences)
These warnings attempt to show exactly where the issue is by displaying
the parameters as well as where the string was received.
@@ -688,14 +1289,14 @@ one wishes the warnings to be exceptions, the
`Python Warnings Filter <https://docs.python.org/2/library/warnings.html>`_
should be used::
- import warnings
- warnings.simplefilter("error") # all warnings raise an exception
+ import warnings
+ warnings.simplefilter("error") # all warnings raise an exception
Given the above warnings, our statement works just fine, but
to get rid of the warnings we would rewrite our statement as follows::
- from sqlalchemy import select, text
- stmt = select([
+ from sqlalchemy import select, text
+ stmt = select([
text("a"),
text("b")
]).where(text("a = b")).select_from(text("sometable"))
@@ -703,10 +1304,10 @@ to get rid of the warnings we would rewrite our statement as follows::
and as the warnings suggest, we can give our statement more specificity
about the text if we use :func:`.column` and :func:`.table`::
- from sqlalchemy import select, text, column, table
+ from sqlalchemy import select, text, column, table
- stmt = select([column("a"), column("b")]).\
- where(text("a = b")).select_from(table("sometable"))
+ stmt = select([column("a"), column("b")]).\
+ where(text("a = b")).select_from(table("sometable"))
Where note also that :func:`.table` and :func:`.column` can now
be imported from "sqlalchemy" without the "sql" part.
@@ -723,10 +1324,10 @@ of this change we have enhanced its functionality. When we have a
:func:`.select` or :class:`.Query` that refers to some column name or named
label, we might want to GROUP BY and/or ORDER BY known columns or labels::
- stmt = select([
- user.c.name,
- func.count(user.c.id).label("id_count")
- ]).group_by("name").order_by("id_count")
+ stmt = select([
+ user.c.name,
+ func.count(user.c.id).label("id_count")
+ ]).group_by("name").order_by("id_count")
In the above statement we expect to see "ORDER BY id_count", as opposed to a
re-statement of the function. The string argument given is actively
@@ -734,24 +1335,24 @@ matched to an entry in the columns clause during compilation, so the above
statement would produce as we expect, without warnings (though note that
the ``"name"`` expression has been resolved to ``users.name``!)::
- SELECT users.name, count(users.id) AS id_count
- FROM users GROUP BY users.name ORDER BY id_count
+ SELECT users.name, count(users.id) AS id_count
+ FROM users GROUP BY users.name ORDER BY id_count
However, if we refer to a name that cannot be located, then we get
the warning again, as below::
- stmt = select([
+ stmt = select([
user.c.name,
func.count(user.c.id).label("id_count")
]).order_by("some_label")
The output does what we say, but again it warns us::
- SAWarning: Can't resolve label reference 'some_label'; converting to
- text() (this warning may be suppressed after 10 occurrences)
+ SAWarning: Can't resolve label reference 'some_label'; converting to
+ text() (this warning may be suppressed after 10 occurrences)
- SELECT users.name, count(users.id) AS id_count
- FROM users ORDER BY some_label
+ SELECT users.name, count(users.id) AS id_count
+ FROM users ORDER BY some_label
The above behavior applies to all those places where we might want to refer
to a so-called "label reference"; ORDER BY and GROUP BY, but also within an
@@ -761,7 +1362,7 @@ Postgresql syntax).
We can still specify any arbitrary expression for ORDER BY or others using
:func:`.text`::
- stmt = select([users]).order_by(text("some special expression"))
+ stmt = select([users]).order_by(text("some special expression"))
The upshot of the whole change is that SQLAlchemy now would like us
to tell it when a string is sent that this string is explicitly
@@ -822,7 +1423,7 @@ data is needed.
A :class:`.Table` can be set up for reflection by passing
:paramref:`.Table.autoload_with` alone::
- my_table = Table('my_table', metadata, autoload_with=some_engine)
+ my_table = Table('my_table', metadata, autoload_with=some_engine)
:ticket:`3027`
@@ -831,39 +1432,6 @@ A :class:`.Table` can be set up for reflection by passing
Dialect Changes
===============
-.. _change_2051:
-
-New Postgresql Table options
------------------------------
-
-Added support for PG table options TABLESPACE, ON COMMIT,
-WITH(OUT) OIDS, and INHERITS, when rendering DDL via
-the :class:`.Table` construct.
-
-.. seealso::
-
- :ref:`postgresql_table_options`
-
-:ticket:`2051`
-
-.. _feature_get_enums:
-
-New get_enums() method with Postgresql Dialect
-----------------------------------------------
-
-The :func:`.inspect` method returns a :class:`.PGInspector` object in the
-case of Postgresql, which includes a new :meth:`.PGInspector.get_enums`
-method that returns information on all available ``ENUM`` types::
-
- from sqlalchemy import inspect, create_engine
-
- engine = create_engine("postgresql+psycopg2://host/dbname")
- insp = inspect(engine)
- print(insp.get_enums())
-
-.. seealso::
-
- :meth:`.PGInspector.get_enums`
MySQL internal "no such table" exceptions not passed to event handlers
----------------------------------------------------------------------
@@ -925,6 +1493,26 @@ when using ODBC to avoid this issue entirely.
:ticket:`3182`
+.. _change_3204:
+
+SQLite/Oracle have distinct methods for temporary table/view name reporting
+---------------------------------------------------------------------------
+
+The :meth:`.Inspector.get_table_names` and :meth:`.Inspector.get_view_names`
+methods in the case of SQLite/Oracle would also return the names of temporary
+tables and views, which is not provided by any other dialect (in the case
+of MySQL at least it is not even possible). This logic has been moved
+out to two new methods :meth:`.Inspector.get_temp_table_names` and
+:meth:`.Inspector.get_temp_view_names`.
+
+Note that reflection of a specific named temporary table or temporary view,
+either by ``Table('name', autoload=True)`` or via methods like
+:meth:`.Inspector.get_columns` continues to function for most if not all
+dialects. For SQLite specifically, there is a bug fix for UNIQUE constraint
+reflection from temp tables as well, which is :ticket:`3203`.
+
+:ticket:`3204`
+
.. _change_2984:
Drizzle Dialect is now an External Dialect
diff --git a/doc/build/core/defaults.rst b/doc/build/core/defaults.rst
index 166273c18..1d55cd6c6 100644
--- a/doc/build/core/defaults.rst
+++ b/doc/build/core/defaults.rst
@@ -1,6 +1,8 @@
+.. module:: sqlalchemy.schema
+
.. _metadata_defaults_toplevel:
+
.. _metadata_defaults:
-.. module:: sqlalchemy.schema
Column Insert/Update Defaults
==============================
diff --git a/doc/build/core/sqlelement.rst b/doc/build/core/sqlelement.rst
index 61600e927..44a969dbb 100644
--- a/doc/build/core/sqlelement.rst
+++ b/doc/build/core/sqlelement.rst
@@ -35,6 +35,8 @@ used to construct any kind of typed SQL expression.
.. autodata:: func
+.. autofunction:: funcfilter
+
.. autofunction:: label
.. autofunction:: literal
@@ -109,6 +111,9 @@ used to construct any kind of typed SQL expression.
.. autoclass:: sqlalchemy.sql.elements.False_
:members:
+.. autoclass:: FunctionFilter
+ :members:
+
.. autoclass:: Label
:members:
diff --git a/doc/build/dialects/index.rst b/doc/build/dialects/index.rst
index da2699b2b..463f00612 100644
--- a/doc/build/dialects/index.rst
+++ b/doc/build/dialects/index.rst
@@ -50,6 +50,7 @@ Production Ready
developed jointly by IBM and SQLAlchemy developers.
* `redshift-sqlalchemy <https://pypi.python.org/pypi/redshift-sqlalchemy>`_ - driver for Amazon Redshift, adapts
the existing Postgresql/psycopg2 driver.
+* `sqlalchemy_exasol <https://github.com/blue-yonder/sqlalchemy_exasol>`_ - driver for EXASolution.
* `sqlalchemy-sqlany <https://github.com/sqlanywhere/sqlalchemy-sqlany>`_ - driver for SAP Sybase SQL
Anywhere, developed by SAP.
* `sqlalchemy-monetdb <https://github.com/gijzelaerr/sqlalchemy-monetdb>`_ - driver for MonetDB.
diff --git a/doc/build/dialects/sqlite.rst b/doc/build/dialects/sqlite.rst
index 21fd4e3aa..a18b0ba7b 100644
--- a/doc/build/dialects/sqlite.rst
+++ b/doc/build/dialects/sqlite.rst
@@ -28,4 +28,9 @@ they originate from :mod:`sqlalchemy.types` or from the local dialect::
Pysqlite
--------
-.. automodule:: sqlalchemy.dialects.sqlite.pysqlite \ No newline at end of file
+.. automodule:: sqlalchemy.dialects.sqlite.pysqlite
+
+Pysqlcipher
+-----------
+
+.. automodule:: sqlalchemy.dialects.sqlite.pysqlcipher \ No newline at end of file
diff --git a/doc/build/faq.rst b/doc/build/faq.rst
index 487f5b953..54f761b3b 100644
--- a/doc/build/faq.rst
+++ b/doc/build/faq.rst
@@ -603,6 +603,108 @@ The same idea applies to all the other arguments, such as ``foreign_keys``::
foo = relationship(Dest, foreign_keys=[foo_id, bar_id])
+.. _faq_subqueryload_limit_sort:
+
+Why is ``ORDER BY`` required with ``LIMIT`` (especially with ``subqueryload()``)?
+---------------------------------------------------------------------------------
+
+A relational database can return rows in any
+arbitrary order, when an explicit ordering is not set.
+While this ordering very often corresponds to the natural
+order of rows within a table, this is not the case for all databases and
+all queries. The consequence of this is that any query that limits rows
+using ``LIMIT`` or ``OFFSET`` should **always** specify an ``ORDER BY``.
+Otherwise, it is not deterministic which rows will actually be returned.
+
+When we use a SQLAlchemy method like :meth:`.Query.first`, we are in fact
+applying a ``LIMIT`` of one to the query, so without an explicit ordering
+it is not deterministic what row we actually get back.
+While we may not notice this for simple queries on databases that usually
+returns rows in their natural
+order, it becomes much more of an issue if we also use :func:`.orm.subqueryload`
+to load related collections, and we may not be loading the collections
+as intended.
+
+SQLAlchemy implements :func:`.orm.subqueryload` by issuing a separate query,
+the results of which are matched up to the results from the first query.
+We see two queries emitted like this:
+
+.. sourcecode:: python+sql
+
+ >>> session.query(User).options(subqueryload(User.addresses)).all()
+ {opensql}-- the "main" query
+ SELECT users.id AS users_id
+ FROM users
+ {stop}
+ {opensql}-- the "load" query issued by subqueryload
+ SELECT addresses.id AS addresses_id,
+ addresses.user_id AS addresses_user_id,
+ anon_1.users_id AS anon_1_users_id
+ FROM (SELECT users.id AS users_id FROM users) AS anon_1
+ JOIN addresses ON anon_1.users_id = addresses.user_id
+ ORDER BY anon_1.users_id
+
+The second query embeds the first query as a source of rows.
+When the inner query uses ``OFFSET`` and/or ``LIMIT`` without ordering,
+the two queries may not see the same results:
+
+.. sourcecode:: python+sql
+
+ >>> user = session.query(User).options(subqueryload(User.addresses)).first()
+ {opensql}-- the "main" query
+ SELECT users.id AS users_id
+ FROM users
+ LIMIT 1
+ {stop}
+ {opensql}-- the "load" query issued by subqueryload
+ SELECT addresses.id AS addresses_id,
+ addresses.user_id AS addresses_user_id,
+ anon_1.users_id AS anon_1_users_id
+ FROM (SELECT users.id AS users_id FROM users LIMIT 1) AS anon_1
+ JOIN addresses ON anon_1.users_id = addresses.user_id
+ ORDER BY anon_1.users_id
+
+Depending on database specifics, there is
+a chance we may get the a result like the following for the two queries::
+
+ -- query #1
+ +--------+
+ |users_id|
+ +--------+
+ | 1|
+ +--------+
+
+ -- query #2
+ +------------+-----------------+---------------+
+ |addresses_id|addresses_user_id|anon_1_users_id|
+ +------------+-----------------+---------------+
+ | 3| 2| 2|
+ +------------+-----------------+---------------+
+ | 4| 2| 2|
+ +------------+-----------------+---------------+
+
+Above, we receive two ``addresses`` rows for ``user.id`` of 2, and none for
+1. We've wasted two rows and failed to actually load the collection. This
+is an insidious error because without looking at the SQL and the results, the
+ORM will not show that there's any issue; if we access the ``addresses``
+for the ``User`` we have, it will emit a lazy load for the collection and we
+won't see that anything actually went wrong.
+
+The solution to this problem is to always specify a deterministic sort order,
+so that the main query always returns the same set of rows. This generally
+means that you should :meth:`.Query.order_by` on a unique column on the table.
+The primary key is a good choice for this::
+
+ session.query(User).options(subqueryload(User.addresses)).order_by(User.id).first()
+
+Note that :func:`.joinedload` does not suffer from the same problem because
+only one query is ever issued, so the load query cannot be different from the
+main query.
+
+.. seealso::
+
+ :ref:`subqueryload_ordering`
+
Performance
===========
diff --git a/doc/build/orm/extensions/declarative.rst b/doc/build/orm/extensions/declarative.rst
index 636bb451b..7d9e634b5 100644
--- a/doc/build/orm/extensions/declarative.rst
+++ b/doc/build/orm/extensions/declarative.rst
@@ -13,6 +13,7 @@ API Reference
.. autofunction:: as_declarative
.. autoclass:: declared_attr
+ :members:
.. autofunction:: sqlalchemy.ext.declarative.api._declarative_constructor
diff --git a/doc/build/orm/inheritance.rst b/doc/build/orm/inheritance.rst
index 642f3420c..9f01a3e24 100644
--- a/doc/build/orm/inheritance.rst
+++ b/doc/build/orm/inheritance.rst
@@ -45,6 +45,12 @@ this column is to act as the **discriminator**, and stores a value
which indicates the type of object represented within the row. The column may
be of any datatype, though string and integer are the most common.
+.. warning::
+
+ Currently, **only one discriminator column may be set**, typically
+ on the base-most class in the hierarchy. "Cascading" polymorphic columns
+ are not yet supported.
+
The discriminator column is only needed if polymorphic loading is
desired, as is usually the case. It is not strictly necessary that
it be present directly on the base mapped table, and can instead be defined on a
diff --git a/doc/build/orm/loading.rst b/doc/build/orm/loading.rst
index 6c2fac004..b2d8124e2 100644
--- a/doc/build/orm/loading.rst
+++ b/doc/build/orm/loading.rst
@@ -120,6 +120,32 @@ query options:
# set children to load eagerly with a second statement
session.query(Parent).options(subqueryload('children')).all()
+.. _subqueryload_ordering:
+
+The Importance of Ordering
+--------------------------
+
+A query which makes use of :func:`.subqueryload` in conjunction with a
+limiting modifier such as :meth:`.Query.first`, :meth:`.Query.limit`,
+or :meth:`.Query.offset` should **always** include :meth:`.Query.order_by`
+against unique column(s) such as the primary key, so that the additional queries
+emitted by :func:`.subqueryload` include
+the same ordering as used by the parent query. Without it, there is a chance
+that the inner query could return the wrong rows::
+
+ # incorrect, no ORDER BY
+ session.query(User).options(subqueryload(User.addresses)).first()
+
+ # incorrect if User.name is not unique
+ session.query(User).options(subqueryload(User.addresses)).order_by(User.name).first()
+
+ # correct
+ session.query(User).options(subqueryload(User.addresses)).order_by(User.name, User.id).first()
+
+.. seealso::
+
+ :ref:`faq_subqueryload_limit_sort` - detailed example
+
Loading Along Paths
-------------------
diff --git a/doc/build/orm/relationships.rst b/doc/build/orm/relationships.rst
index c65f06cbc..f512251a7 100644
--- a/doc/build/orm/relationships.rst
+++ b/doc/build/orm/relationships.rst
@@ -1079,12 +1079,15 @@ The above relationship will produce a join like::
ON host_entry_1.ip_address = CAST(host_entry.content AS INET)
An alternative syntax to the above is to use the :func:`.foreign` and
-:func:`.remote` :term:`annotations`, inline within the :paramref:`~.relationship.primaryjoin` expression.
+:func:`.remote` :term:`annotations`,
+inline within the :paramref:`~.relationship.primaryjoin` expression.
This syntax represents the annotations that :func:`.relationship` normally
applies by itself to the join condition given the :paramref:`~.relationship.foreign_keys` and
-:paramref:`~.relationship.remote_side` arguments; the functions are provided in the API in the
-rare case that :func:`.relationship` can't determine the exact location
-of these features on its own::
+:paramref:`~.relationship.remote_side` arguments. These functions may
+be more succinct when an explicit join condition is present, and additionally
+serve to mark exactly the column that is "foreign" or "remote" independent
+of whether that column is stated multiple times or within complex
+SQL expressions::
from sqlalchemy.orm import foreign, remote
@@ -1157,6 +1160,130 @@ Will render as::
flag to assist in the creation of :func:`.relationship` constructs using
custom operators.
+.. _relationship_overlapping_foreignkeys:
+
+Overlapping Foreign Keys
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+A rare scenario can arise when composite foreign keys are used, such that
+a single column may be the subject of more than one column
+referred to via foreign key constraint.
+
+Consider an (admittedly complex) mapping such as the ``Magazine`` object,
+referred to both by the ``Writer`` object and the ``Article`` object
+using a composite primary key scheme that includes ``magazine_id``
+for both; then to make ``Article`` refer to ``Writer`` as well,
+``Article.magazine_id`` is involved in two separate relationships;
+``Article.magazine`` and ``Article.writer``::
+
+ class Magazine(Base):
+ __tablename__ = 'magazine'
+
+ id = Column(Integer, primary_key=True)
+
+
+ class Article(Base):
+ __tablename__ = 'article'
+
+ article_id = Column(Integer)
+ magazine_id = Column(ForeignKey('magazine.id'))
+ writer_id = Column()
+
+ magazine = relationship("Magazine")
+ writer = relationship("Writer")
+
+ __table_args__ = (
+ PrimaryKeyConstraint('article_id', 'magazine_id'),
+ ForeignKeyConstraint(
+ ['writer_id', 'magazine_id'],
+ ['writer.id', 'writer.magazine_id']
+ ),
+ )
+
+
+ class Writer(Base):
+ __tablename__ = 'writer'
+
+ id = Column(Integer, primary_key=True)
+ magazine_id = Column(ForeignKey('magazine.id'), primary_key=True)
+ magazine = relationship("Magazine")
+
+When the above mapping is configured, we will see this warning emitted::
+
+ SAWarning: relationship 'Article.writer' will copy column
+ writer.magazine_id to column article.magazine_id,
+ which conflicts with relationship(s): 'Article.magazine'
+ (copies magazine.id to article.magazine_id). Consider applying
+ viewonly=True to read-only relationships, or provide a primaryjoin
+ condition marking writable columns with the foreign() annotation.
+
+What this refers to originates from the fact that ``Article.magazine_id`` is
+the subject of two different foreign key constraints; it refers to
+``Magazine.id`` directly as a source column, but also refers to
+``Writer.magazine_id`` as a source column in the context of the
+composite key to ``Writer``. If we associate an ``Article`` with a
+particular ``Magazine``, but then associate the ``Article`` with a
+``Writer`` that's associated with a *different* ``Magazine``, the ORM
+will overwrite ``Article.magazine_id`` non-deterministically, silently
+changing which magazine we refer towards; it may
+also attempt to place NULL into this columnn if we de-associate a
+``Writer`` from an ``Article``. The warning lets us know this is the case.
+
+To solve this, we need to break out the behavior of ``Article`` to include
+all three of the following features:
+
+1. ``Article`` first and foremost writes to
+ ``Article.magazine_id`` based on data persisted in the ``Article.magazine``
+ relationship only, that is a value copied from ``Magazine.id``.
+
+2. ``Article`` can write to ``Article.writer_id`` on behalf of data
+ persisted in the ``Article.writer`` relationship, but only the
+ ``Writer.id`` column; the ``Writer.magazine_id`` column should not
+ be written into ``Article.magazine_id`` as it ultimately is sourced
+ from ``Magazine.id``.
+
+3. ``Article`` takes ``Article.magazine_id`` into account when loading
+ ``Article.writer``, even though it *doesn't* write to it on behalf
+ of this relationship.
+
+To get just #1 and #2, we could specify only ``Article.writer_id`` as the
+"foreign keys" for ``Article.writer``::
+
+ class Article(Base):
+ # ...
+
+ writer = relationship("Writer", foreign_keys='Article.writer_id')
+
+However, this has the effect of ``Article.writer`` not taking
+``Article.magazine_id`` into account when querying against ``Writer``:
+
+.. sourcecode:: sql
+
+ SELECT article.article_id AS article_article_id,
+ article.magazine_id AS article_magazine_id,
+ article.writer_id AS article_writer_id
+ FROM article
+ JOIN writer ON writer.id = article.writer_id
+
+Therefore, to get at all of #1, #2, and #3, we express the join condition
+as well as which columns to be written by combining
+:paramref:`~.relationship.primaryjoin` fully, along with either the
+:paramref:`~.relationship.foreign_keys` argument, or more succinctly by
+annotating with :func:`~.orm.foreign`::
+
+ class Article(Base):
+ # ...
+
+ writer = relationship(
+ "Writer",
+ primaryjoin="and_(Writer.id == foreign(Article.writer_id), "
+ "Writer.magazine_id == Article.magazine_id)")
+
+.. versionchanged:: 1.0.0 the ORM will attempt to warn when a column is used
+ as the synchronization target from more than one relationship
+ simultaneously.
+
+
Non-relational Comparisons / Materialized Path
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/build/orm/tutorial.rst b/doc/build/orm/tutorial.rst
index f1b6a4499..8871ce765 100644
--- a/doc/build/orm/tutorial.rst
+++ b/doc/build/orm/tutorial.rst
@@ -1631,6 +1631,13 @@ very easy to use:
>>> jack.addresses
[<Address(email_address='jack@google.com')>, <Address(email_address='j25@yahoo.com')>]
+.. note::
+
+ :func:`.subqueryload` when used in conjunction with limiting such as
+ :meth:`.Query.first`, :meth:`.Query.limit` or :meth:`.Query.offset`
+ should also include :meth:`.Query.order_by` on a unique column in order to
+ ensure correct results. See :ref:`subqueryload_ordering`.
+
Joined Load
-------------