diff options
author | Mike Bayer <mike_mp@zzzcomputing.com> | 2013-10-06 20:29:08 -0400 |
---|---|---|
committer | Mike Bayer <mike_mp@zzzcomputing.com> | 2013-10-06 20:29:08 -0400 |
commit | 1b25ed907fb7311d28d2273c9b9858b50c1a7afc (patch) | |
tree | 74bd8df8638dbd1f1e48b1ca660963944be0be3d | |
parent | d79e1d69a6b2d0d1cc18d3d9d0283ef4a77925bc (diff) | |
download | sqlalchemy-1b25ed907fb7311d28d2273c9b9858b50c1a7afc.tar.gz |
- merge ticket_1418 branch, [ticket:1418]
- The system of loader options has been entirely rearchitected to build
upon a much more comprehensive base, the :class:`.Load` object. This
base allows any common loader option like :func:`.joinedload`,
:func:`.defer`, etc. to be used in a "chained" style for the purpose
of specifying options down a path, such as ``joinedload("foo").subqueryload("bar")``.
The new system supersedes the usage of dot-separated path names,
multiple attributes within options, and the usage of ``_all()`` options.
- Added a new load option :func:`.orm.load_only`. This allows a series
of column names to be specified as loading "only" those attributes,
deferring the rest.
26 files changed, 2847 insertions, 1903 deletions
diff --git a/doc/build/changelog/changelog_09.rst b/doc/build/changelog/changelog_09.rst index b09ecefae..ec6f086f3 100644 --- a/doc/build/changelog/changelog_09.rst +++ b/doc/build/changelog/changelog_09.rst @@ -14,6 +14,30 @@ .. change:: :tags: feature, orm + :tickets: 1418 + + Added a new load option :func:`.orm.load_only`. This allows a series + of column names to be specified as loading "only" those attributes, + deferring the rest. + + .. change:: + :tags: feature, orm + :tickets: 1418 + + The system of loader options has been entirely rearchitected to build + upon a much more comprehensive base, the :class:`.Load` object. This + base allows any common loader option like :func:`.joinedload`, + :func:`.defer`, etc. to be used in a "chained" style for the purpose + of specifying options down a path, such as ``joinedload("foo").subqueryload("bar")``. + The new system supersedes the usage of dot-separated path names, + multiple attributes within options, and the usage of ``_all()`` options. + + .. seealso:: + + :ref:`feature_1418` + + .. change:: + :tags: feature, orm :tickets: 2824 The :func:`.composite` construct now maintains the return object diff --git a/doc/build/changelog/migration_09.rst b/doc/build/changelog/migration_09.rst index 2952c67b0..da01dc5c8 100644 --- a/doc/build/changelog/migration_09.rst +++ b/doc/build/changelog/migration_09.rst @@ -385,6 +385,150 @@ such as listener targets, to be garbage collected when they go out of scope. :ticket:`2268` +.. _feature_1418: + +New Query Options API; ``load_only()`` option +--------------------------------------------- + +The system of loader options such as :func:`.orm.joinedload`, +:func:`.orm.subqueryload`, :func:`.orm.lazyload`, :func:`.orm.defer`, etc. +all build upon a new system known as :class:`.Load`. :class:`.Load` provides +a "method chained" (a.k.a. :term:`generative`) approach to loader options, so that +instead of joining together long paths using dots or multiple attribute names, +an explicit loader style is given for each path. + +While the new way is slightly more verbose, it is simpler to understand +in that there is no ambiguity in what options are being applied to which paths; +it simplifies the method signatures of the options and provides greater flexibility +particularly for column-based options. The old systems are to remain functional +indefinitely as well and all styles can be mixed. + +**Old Way** + +To set a certain style of loading along every link in a multi-element path, the ``_all()`` +option has to be used:: + + query(User).options(joinedload_all("orders.items.keywords")) + +**New Way** + +Loader options are now chainable, so the same ``joinedload(x)`` method is applied +equally to each link, without the need to keep straight between +:func:`.joinedload` and :func:`.joinedload_all`:: + + query(User).options(joinedload("orders").joinedload("items").joinedload("keywords")) + +**Old Way** + +Setting an option on path that is based on a subclass requires that all +links in the path be spelled out as class bound attributes, since the +:meth:`.PropComparator.of_type` method needs to be called:: + + session.query(Company).\ + options( + subqueryload_all( + Company.employees.of_type(Engineer), + Engineer.machines + ) + ) + +**New Way** + +Only those elements in the path that actually need :meth:`.PropComparator.of_type` +need to be set as a class-bound attribute, string-based names can be resumed +afterwards:: + + session.query(Company).\ + options( + subqueryload(Company.employees.of_type(Engineer)). + subqueryload("machines") + ) + ) + +**Old Way** + +Setting the loader option on the last link in a long path uses a syntax +that looks a lot like it should be setting the option for all links in the +path, causing confusion:: + + query(User).options(subqueryload("orders.items.keywords")) + +**New Way** + +A path can now be spelled out using :func:`.defaultload` for entries in the +path where the existing loader style should be unchanged. More verbose +but the intent is clearer:: + + query(User).options(defaultload("orders").defaultload("items").subqueryload("keywords")) + + +The dotted style can still be taken advantage of, particularly in the case +of skipping over several path elements:: + + query(User).options(defaultload("orders.items").subqueryload("keywords")) + +**Old Way** + +The :func:`.defer` option on a path needed to be spelled out with the full +path for each column:: + + query(User).options(defer("orders.description"), defer("orders.isopen")) + +**New Way** + +A single :class:`.Load` object that arrives at the target path can have +:meth:`.Load.defer` called upon it repeatedly:: + + query(User).options(defaultload("orders").defer("description").defer("isopen")) + +The Load Class +^^^^^^^^^^^^^^^ + +The :class:`.Load` class can be used directly to provide a "bound" target, +especially when multiple parent entities are present:: + + from sqlalchemy.orm import Load + + query(User, Address).options(Load(Address).joinedload("entries")) + +Load Only +^^^^^^^^^ + +A new option :func:`.load_only` achieves a "defer everything but" style of load, +loading only the given columns and deferring the rest:: + + from sqlalchemy.orm import load_only + + query(User).options(load_only("name", "fullname")) + + # specify explicit parent entity + query(User, Address).options(Load(User).load_only("name", "fullname")) + + # specify path + query(User).options(joinedload(User.addresses).load_only("email_address")) + +Class-specific Wildcards +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Using :class:`.Load`, a wildcard may be used to set the loading for all +relationships (or perhaps columns) on a given entity, without affecting any +others:: + + # lazyload all User relationships + query(User).options(Load(User).lazyload("*")) + + # undefer all User columns + query(User).options(Load(User).undefer("*")) + + # lazyload all Address relationships + query(User).options(defaultload(User.addresses).lazyload("*")) + + # undefer all Address columns + query(User).options(defaultload(User.addresses).undefer("*")) + + +:ticket:`1418` + .. _feature_722: diff --git a/doc/build/orm/inheritance.rst b/doc/build/orm/inheritance.rst index a82fcf675..e6c1e378b 100644 --- a/doc/build/orm/inheritance.rst +++ b/doc/build/orm/inheritance.rst @@ -478,8 +478,11 @@ Below we load ``Company`` rows while eagerly loading related ``Engineer`` objects, querying the ``employee`` and ``engineer`` tables simultaneously:: session.query(Company).\ - options(subqueryload_all(Company.employees.of_type(Engineer), - Engineer.machines)) + options( + subqueryload(Company.employees.of_type(Engineer)). + subqueryload("machines") + ) + ) .. versionadded:: 0.8 :func:`.joinedload` and :func:`.subqueryload` support diff --git a/doc/build/orm/loading.rst b/doc/build/orm/loading.rst index e84179558..ccdb53cee 100644 --- a/doc/build/orm/loading.rst +++ b/doc/build/orm/loading.rst @@ -1,3 +1,5 @@ +.. _loading_toplevel: + .. currentmodule:: sqlalchemy.orm Relationship Loading Techniques @@ -82,24 +84,25 @@ The default **loader strategy** for any :func:`~sqlalchemy.orm.relationship` is configured by the ``lazy`` keyword argument, which defaults to ``select`` - this indicates a "select" statement . Below we set it as ``joined`` so that the ``children`` relationship is eager -loading, using a join: - -.. sourcecode:: python+sql +loaded using a JOIN:: # load the 'children' collection using LEFT OUTER JOIN - mapper(Parent, parent_table, properties={ - 'children': relationship(Child, lazy='joined') - }) + class Parent(Base): + __tablename__ = 'parent' + + id = Column(Integer, primary_key=True) + children = relationship("Child", lazy='joined') We can also set it to eagerly load using a second query for all collections, -using ``subquery``: +using ``subquery``:: -.. sourcecode:: python+sql + # load the 'children' collection using a second query which + # JOINS to a subquery of the original + class Parent(Base): + __tablename__ = 'parent' - # load the 'children' attribute using a join to a subquery - mapper(Parent, parent_table, properties={ - 'children': relationship(Child, lazy='subquery') - }) + id = Column(Integer, primary_key=True) + children = relationship("Child", lazy='subquery') When querying, all three choices of loader strategy are available on a per-query basis, using the :func:`~sqlalchemy.orm.joinedload`, @@ -117,42 +120,38 @@ query options: # set children to load eagerly with a second statement session.query(Parent).options(subqueryload('children')).all() -To reference a relationship that is deeper than one level, separate the names by periods: +Loading Along Paths +------------------- -.. sourcecode:: python+sql +To reference a relationship that is deeper than one level, method chaining +may be used. The object returned by all loader options is an instance of +the :class:`.Load` class, which provides a so-called "generative" interface:: - session.query(Parent).options(joinedload('foo.bar.bat')).all() + session.query(Parent).options( + joinedload('foo'). + joinedload('bar'). + joinedload('bat') + ).all() -When using dot-separated names with :func:`~sqlalchemy.orm.joinedload` or -:func:`~sqlalchemy.orm.subqueryload`, the option applies **only** to the actual -attribute named, and **not** its ancestors. For example, suppose a mapping -from ``A`` to ``B`` to ``C``, where the relationships, named ``atob`` and -``btoc``, are both lazy-loading. A statement like the following: +Using method chaining, the loader style of each link in the path is explicitly +stated. To navigate along a path without changing the existing loader style +of a particular attribute, the :func:`.defaultload` method/function may be used:: -.. sourcecode:: python+sql - - session.query(A).options(joinedload('atob.btoc')).all() - -will load only ``A`` objects to start. When the ``atob`` attribute on each -``A`` is accessed, the returned ``B`` objects will *eagerly* load their ``C`` -objects. + session.query(A).options( + defaultload("atob").joinedload("btoc") + ).all() -Therefore, to modify the eager load to load both ``atob`` as well as ``btoc``, -place joinedloads for both: - -.. sourcecode:: python+sql +.. versionchanged:: 0.9.0 - session.query(A).options(joinedload('atob'), joinedload('atob.btoc')).all() - -or more succinctly just use :func:`~sqlalchemy.orm.joinedload_all` or -:func:`~sqlalchemy.orm.subqueryload_all`: - -.. sourcecode:: python+sql - - session.query(A).options(joinedload_all('atob.btoc')).all() - -There are two other loader strategies available, **dynamic loading** and **no -loading**; these are described in :ref:`largecollections`. + The previous approach of specifying dot-separated paths within loader + options has been superseded by the less ambiguous approach of the + :class:`.Load` object and related methods. With this system, the user + specifies the style of loading for each link along the chain explicitly, + rather than guessing between options like ``joinedload()`` vs. ``joinedload_all()``. + The :func:`.orm.defaultload` is provided to allow path navigation without + modification of existing loader options. The dot-separated path system + as well as the ``_all()`` functions will remain available for backwards- + compatibility indefinitely. Default Loading Strategies -------------------------- @@ -191,6 +190,22 @@ for the ``widget`` relationship:: If multiple ``'*'`` options are passed, the last one overrides those previously passed. +Per-Entity Default Loading Strategies +------------------------------------- + +.. versionadded:: 0.9.0 + Per-entity default loader strategies. + +A variant of the default loader strategy is the ability to set the strategy +on a per-entity basis. For example, if querying for ``User`` and ``Address``, +we can instruct all relationships on ``Address`` only to use lazy loading +by first applying the :class:`.Load` object, then specifying the ``*`` as a +chained option:: + + session.query(User, Address).options(Load(Address).lazyload('*')) + +Above, all relationships on ``Address`` will be set to a lazy load. + .. _zen_of_eager_loading: The Zen of Eager Loading @@ -402,31 +417,27 @@ For this SQLAlchemy supplies the :func:`~sqlalchemy.orm.contains_eager()` option. This option is used in the same manner as the :func:`~sqlalchemy.orm.joinedload()` option except it is assumed that the :class:`~sqlalchemy.orm.query.Query` will specify the appropriate joins -explicitly. Below it's used with a ``from_statement`` load:: +explicitly. Below, we specify a join between ``User`` and ``Address`` +and addtionally establish this as the basis for eager loading of ``User.addresses``:: - # mapping is the users->addresses mapping - mapper(User, users_table, properties={ - 'addresses': relationship(Address, addresses_table) - }) + class User(Base): + __tablename__ = 'user' + id = Column(Integer, primary_key=True) + addresses = relationship("Address") - # define a query on USERS with an outer join to ADDRESSES - statement = users_table.outerjoin(addresses_table).select().apply_labels() + class Address(Base): + __tablename__ = 'address' - # construct a Query object which expects the "addresses" results - query = session.query(User).options(contains_eager('addresses')) - - # get results normally - r = query.from_statement(statement) + # ... -It works just as well with an inline :meth:`.Query.join` or -:meth:`.Query.outerjoin`:: + q = session.query(User).join(User.addresses).\ + options(contains_eager(User.addresses)) - session.query(User).outerjoin(User.addresses).options(contains_eager(User.addresses)).all() If the "eager" portion of the statement is "aliased", the ``alias`` keyword argument to :func:`~sqlalchemy.orm.contains_eager` may be used to indicate it. -This is a string alias name or reference to an actual -:class:`~sqlalchemy.sql.expression.Alias` (or other selectable) object: +This is sent as a reference to an :func:`.aliased` or :class:`.Alias` +construct: .. sourcecode:: python+sql @@ -444,10 +455,23 @@ This is a string alias name or reference to an actual adalias.user_id AS adalias_user_id, adalias.email_address AS adalias_email_address, (...other columns...) FROM users LEFT OUTER JOIN email_addresses AS email_addresses_1 ON users.user_id = email_addresses_1.user_id -The ``alias`` argument is used only as a source of columns to match up to the -result set. You can use it to match up the result to arbitrary label -names in a string SQL statement, by passing a :func:`.select` which links those -labels to the mapped :class:`.Table`:: +The path given as the argument to :func:`.contains_eager` needs +to be a full path from the starting entity. For example if we were loading +``Users->orders->Order->items->Item``, the string version would look like:: + + query(User).options(contains_eager('orders').contains_eager('items')) + +Or using the class-bound descriptor:: + + query(User).options(contains_eager(User.orders).contains_eager(Order.items)) + +Advanced Usage with Arbitrary Statements +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The ``alias`` argument can be more creatively used, in that it can be made +to represent any set of arbitrary names to match up into a statement. +Below it is linked to a :func:`.select` which links a set of column objects +to a string SQL statement:: # label the columns of the addresses table eager_columns = select([ @@ -463,24 +487,17 @@ labels to the mapped :class:`.Table`:: "from users left outer join addresses on users.user_id=addresses.user_id").\ options(contains_eager(User.addresses, alias=eager_columns)) -The path given as the argument to :func:`.contains_eager` needs -to be a full path from the starting entity. For example if we were loading -``Users->orders->Order->items->Item``, the string version would look like:: - - query(User).options(contains_eager('orders', 'items')) - -Or using the class-bound descriptor:: - query(User).options(contains_eager(User.orders, Order.items)) - -Relation Loader API --------------------- +Relationship Loader API +------------------------ .. autofunction:: contains_alias .. autofunction:: contains_eager +.. autofunction:: defaultload + .. autofunction:: eagerload .. autofunction:: eagerload_all diff --git a/doc/build/orm/mapper_config.rst b/doc/build/orm/mapper_config.rst index 420ab3a32..409015200 100644 --- a/doc/build/orm/mapper_config.rst +++ b/doc/build/orm/mapper_config.rst @@ -310,23 +310,68 @@ separately when it is accessed:: photo3 = deferred(Column(Binary), group='photos') You can defer or undefer columns at the :class:`~sqlalchemy.orm.query.Query` -level using the :func:`.orm.defer` and :func:`.orm.undefer` query options:: +level using options, including :func:`.orm.defer` and :func:`.orm.undefer`:: from sqlalchemy.orm import defer, undefer query = session.query(Book) - query.options(defer('summary')).all() - query.options(undefer('excerpt')).all() + query = query.options(defer('summary')) + query = query.options(undefer('excerpt')) + query.all() -And an entire "deferred group", i.e. which uses the ``group`` keyword argument -to :func:`.orm.deferred`, can be undeferred using -:func:`.orm.undefer_group`, sending in the group name:: +An arbitrary set of columns can be selected as "load only" columns, which will +be loaded while deferring all other columns on a given entity, using :func:`.orm.load_only`:: + + from sqlalchemy.orm import load_only + + session.query(Book).options(load_only("summary", "excerpt")) + +:func:`.orm.deferred` attributes which are marked with a "group" can be undeferred +using :func:`.orm.undefer_group`, sending in the group name:: from sqlalchemy.orm import undefer_group query = session.query(Book) query.options(undefer_group('photos')).all() +Deferred Loading with Multiple Entities +--------------------------------------- + +To specify column deferral options within a :class:`.Query` that loads multiple types +of entity, the :class:`.Load` object can specify which parent entity to start with:: + + from sqlalchemy.orm import Load + + query = session.query(Book, Author).join(Book.author) + query = query.options( + Load(Book).load_only("summary", "excerpt"), + Load(Author).defer("bio") + ) + +To specify column deferral options along the path of various relationships, +the options support chaining, where the loading style of each relationship +is specified first, then is chained to the deferral options. Such as, to load +``Book`` instances, then joined-eager-load the ``Author``, then apply deferral +options to the ``Author`` entity:: + + from sqlalchemy.orm import joinedload + + query = session.query(Book) + query = query.options( + joinedload(Book.author).load_only("summary", "excerpt"), + ) + +In the case where the loading style of parent relationships should be left +unchanged, use :func:`.orm.defaultload`:: + + from sqlalchemy.orm import defaultload + + query = session.query(Book) + query = query.options( + defaultload(Book.author).load_only("summary", "excerpt"), + ) + + Column Deferral API ------------------- @@ -334,6 +379,8 @@ Column Deferral API .. autofunction:: defer +.. autofunction:: load_only + .. autofunction:: undefer .. autofunction:: undefer_group diff --git a/doc/build/orm/query.rst b/doc/build/orm/query.rst index 344c4e013..d83bdb6ae 100644 --- a/doc/build/orm/query.rst +++ b/doc/build/orm/query.rst @@ -37,6 +37,9 @@ ORM-Specific Query Constructs .. autoclass:: sqlalchemy.util.KeyedTuple :members: keys, _fields, _asdict +.. autoclass:: sqlalchemy.orm.strategy_options.Load + :members: + .. autofunction:: join .. autofunction:: outerjoin diff --git a/lib/sqlalchemy/orm/__init__.py b/lib/sqlalchemy/orm/__init__.py index 5cd0f2854..225311db6 100644 --- a/lib/sqlalchemy/orm/__init__.py +++ b/lib/sqlalchemy/orm/__init__.py @@ -148,19 +148,22 @@ def backref(name, **kwargs): return (name, kwargs) -def deferred(*columns, **kwargs): - """Return a :class:`.DeferredColumnProperty`, which indicates this - object attributes should only be loaded from its corresponding - table column when first accessed. +def deferred(*columns, **kw): + """Indicate a column-based mapped attribute that by default will + not load unless accessed. - Used with the "properties" dictionary sent to :func:`mapper`. + :param \*columns: columns to be mapped. This is typically a single + :class:`.Column` object, however a collection is supported in order + to support multiple columns mapped under the same attribute. - See also: + :param \**kw: additional keyword arguments passed to :class:`.ColumnProperty`. - :ref:`deferred` + .. seealso:: + + :ref:`deferred` """ - return ColumnProperty(deferred=True, *columns, **kwargs) + return ColumnProperty(deferred=True, *columns, **kw) mapper = public_factory(Mapper, ".orm.mapper") @@ -213,107 +216,24 @@ def clear_mappers(): finally: mapperlib._CONFIGURE_MUTEX.release() - -def joinedload(*keys, **kw): - """Return a ``MapperOption`` that will convert the property of the given - name or series of mapped attributes into an joined eager load. - - .. versionchanged:: 0.6beta3 - This function is known as :func:`eagerload` in all versions - of SQLAlchemy prior to version 0.6beta3, including the 0.5 and 0.4 - series. :func:`eagerload` will remain available for the foreseeable - future in order to enable cross-compatibility. - - Used with :meth:`~sqlalchemy.orm.query.Query.options`. - - examples:: - - # joined-load the "orders" collection on "User" - query(User).options(joinedload(User.orders)) - - # joined-load the "keywords" collection on each "Item", - # but not the "items" collection on "Order" - those - # remain lazily loaded. - query(Order).options(joinedload(Order.items, Item.keywords)) - - # to joined-load across both, use joinedload_all() - query(Order).options(joinedload_all(Order.items, Item.keywords)) - - # set the default strategy to be 'joined' - query(Order).options(joinedload('*')) - - :func:`joinedload` also accepts a keyword argument `innerjoin=True` which - indicates using an inner join instead of an outer:: - - query(Order).options(joinedload(Order.user, innerjoin=True)) - - .. note:: - - The join created by :func:`joinedload` is anonymously aliased such that - it **does not affect the query results**. An :meth:`.Query.order_by` - or :meth:`.Query.filter` call **cannot** reference these aliased - tables - so-called "user space" joins are constructed using - :meth:`.Query.join`. The rationale for this is that - :func:`joinedload` is only applied in order to affect how related - objects or collections are loaded as an optimizing detail - it can be - added or removed with no impact on actual results. See the section - :ref:`zen_of_eager_loading` for a detailed description of how this is - used, including how to use a single explicit JOIN for - filtering/ordering and eager loading simultaneously. - - See also: :func:`subqueryload`, :func:`lazyload` - - """ - innerjoin = kw.pop('innerjoin', None) - if innerjoin is not None: - return ( - _strategies.EagerLazyOption(keys, lazy='joined'), - _strategies.EagerJoinOption(keys, innerjoin) - ) - else: - return _strategies.EagerLazyOption(keys, lazy='joined') - - -def joinedload_all(*keys, **kw): - """Return a ``MapperOption`` that will convert all properties along the - given dot-separated path or series of mapped attributes - into an joined eager load. - - .. versionchanged:: 0.6beta3 - This function is known as :func:`eagerload_all` in all versions - of SQLAlchemy prior to version 0.6beta3, including the 0.5 and 0.4 - series. :func:`eagerload_all` will remain available for the - foreseeable future in order to enable cross-compatibility. - - Used with :meth:`~sqlalchemy.orm.query.Query.options`. - - For example:: - - query.options(joinedload_all('orders.items.keywords'))... - - will set all of ``orders``, ``orders.items``, and - ``orders.items.keywords`` to load in one joined eager load. - - Individual descriptors are accepted as arguments as well:: - - query.options(joinedload_all(User.orders, Order.items, Item.keywords)) - - The keyword arguments accept a flag `innerjoin=True|False` which will - override the value of the `innerjoin` flag specified on the - relationship(). - - See also: :func:`subqueryload_all`, :func:`lazyload` - - """ - innerjoin = kw.pop('innerjoin', None) - if innerjoin is not None: - return ( - _strategies.EagerLazyOption(keys, lazy='joined', chained=True), - _strategies.EagerJoinOption(keys, innerjoin, chained=True) - ) - else: - return _strategies.EagerLazyOption(keys, lazy='joined', chained=True) - +from . import strategy_options + +joinedload = strategy_options.joinedload._unbound_fn +joinedload_all = strategy_options.joinedload._unbound_all_fn +contains_eager = strategy_options.contains_eager._unbound_fn +defer = strategy_options.defer._unbound_fn +undefer = strategy_options.undefer._unbound_fn +undefer_group = strategy_options.undefer_group._unbound_fn +load_only = strategy_options.load_only._unbound_fn +lazyload = strategy_options.lazyload._unbound_fn +lazyload_all = strategy_options.lazyload_all._unbound_all_fn +subqueryload = strategy_options.subqueryload._unbound_fn +subqueryload_all = strategy_options.subqueryload_all._unbound_all_fn +immediateload = strategy_options.immediateload._unbound_fn +noload = strategy_options.noload._unbound_fn +defaultload = strategy_options.defaultload._unbound_fn + +from .strategy_options import Load def eagerload(*args, **kwargs): """A synonym for :func:`joinedload()`.""" @@ -325,285 +245,11 @@ def eagerload_all(*args, **kwargs): return joinedload_all(*args, **kwargs) -def subqueryload(*keys): - """Return a ``MapperOption`` that will convert the property - of the given name or series of mapped attributes - into an subquery eager load. - - Used with :meth:`~sqlalchemy.orm.query.Query.options`. - - examples:: - - # subquery-load the "orders" collection on "User" - query(User).options(subqueryload(User.orders)) - - # subquery-load the "keywords" collection on each "Item", - # but not the "items" collection on "Order" - those - # remain lazily loaded. - query(Order).options(subqueryload(Order.items, Item.keywords)) - # to subquery-load across both, use subqueryload_all() - query(Order).options(subqueryload_all(Order.items, Item.keywords)) - - # set the default strategy to be 'subquery' - query(Order).options(subqueryload('*')) - - See also: :func:`joinedload`, :func:`lazyload` - - """ - return _strategies.EagerLazyOption(keys, lazy="subquery") - - -def subqueryload_all(*keys): - """Return a ``MapperOption`` that will convert all properties along the - given dot-separated path or series of mapped attributes - into a subquery eager load. - - Used with :meth:`~sqlalchemy.orm.query.Query.options`. - - For example:: - - query.options(subqueryload_all('orders.items.keywords'))... - - will set all of ``orders``, ``orders.items``, and - ``orders.items.keywords`` to load in one subquery eager load. - - Individual descriptors are accepted as arguments as well:: - - query.options(subqueryload_all(User.orders, Order.items, - Item.keywords)) - - See also: :func:`joinedload_all`, :func:`lazyload`, :func:`immediateload` - - """ - return _strategies.EagerLazyOption(keys, lazy="subquery", chained=True) - - -def lazyload(*keys): - """Return a ``MapperOption`` that will convert the property of the given - name or series of mapped attributes into a lazy load. - - Used with :meth:`~sqlalchemy.orm.query.Query.options`. - - See also: :func:`eagerload`, :func:`subqueryload`, :func:`immediateload` - - """ - return _strategies.EagerLazyOption(keys, lazy=True) - - -def lazyload_all(*keys): - """Return a ``MapperOption`` that will convert all the properties - along the given dot-separated path or series of mapped attributes - into a lazy load. - - Used with :meth:`~sqlalchemy.orm.query.Query.options`. - - See also: :func:`eagerload`, :func:`subqueryload`, :func:`immediateload` - - """ - return _strategies.EagerLazyOption(keys, lazy=True, chained=True) - - -def noload(*keys): - """Return a ``MapperOption`` that will convert the property of the - given name or series of mapped attributes into a non-load. - - Used with :meth:`~sqlalchemy.orm.query.Query.options`. - - See also: :func:`lazyload`, :func:`eagerload`, - :func:`subqueryload`, :func:`immediateload` - - """ - return _strategies.EagerLazyOption(keys, lazy=None) - - -def immediateload(*keys): - """Return a ``MapperOption`` that will convert the property of the given - name or series of mapped attributes into an immediate load. - - The "immediate" load means the attribute will be fetched - with a separate SELECT statement per parent in the - same way as lazy loading - except the loader is guaranteed - to be called at load time before the parent object - is returned in the result. - - The normal behavior of lazy loading applies - if - the relationship is a simple many-to-one, and the child - object is already present in the :class:`.Session`, - no SELECT statement will be emitted. - - Used with :meth:`~sqlalchemy.orm.query.Query.options`. - - See also: :func:`lazyload`, :func:`eagerload`, :func:`subqueryload` - - .. versionadded:: 0.6.5 - - """ - return _strategies.EagerLazyOption(keys, lazy='immediate') contains_alias = public_factory(AliasOption, ".orm.contains_alias") -def contains_eager(*keys, **kwargs): - """Return a ``MapperOption`` that will indicate to the query that - the given attribute should be eagerly loaded from columns currently - in the query. - - Used with :meth:`~sqlalchemy.orm.query.Query.options`. - - The option is used in conjunction with an explicit join that loads - the desired rows, i.e.:: - - sess.query(Order).\\ - join(Order.user).\\ - options(contains_eager(Order.user)) - - The above query would join from the ``Order`` entity to its related - ``User`` entity, and the returned ``Order`` objects would have the - ``Order.user`` attribute pre-populated. - - :func:`contains_eager` also accepts an `alias` argument, which is the - string name of an alias, an :func:`~sqlalchemy.sql.expression.alias` - construct, or an :func:`~sqlalchemy.orm.aliased` construct. Use this when - the eagerly-loaded rows are to come from an aliased table:: - - user_alias = aliased(User) - sess.query(Order).\\ - join((user_alias, Order.user)).\\ - options(contains_eager(Order.user, alias=user_alias)) - - See also :func:`eagerload` for the "automatic" version of this - functionality. - - For additional examples of :func:`contains_eager` see - :ref:`contains_eager`. - - """ - alias = kwargs.pop('alias', None) - if kwargs: - raise exc.ArgumentError( - 'Invalid kwargs for contains_eager: %r' % list(kwargs.keys())) - return _strategies.EagerLazyOption(keys, lazy='joined', - propagate_to_loaders=False, chained=True), \ - _strategies.LoadEagerFromAliasOption(keys, alias=alias, chained=True) - - -def defer(*key): - """Return a :class:`.MapperOption` that will convert the column property - of the given name into a deferred load. - - Used with :meth:`.Query.options`. - - e.g.:: - - from sqlalchemy.orm import defer - - query(MyClass).options(defer("attribute_one"), - defer("attribute_two")) - - A class bound descriptor is also accepted:: - - query(MyClass).options( - defer(MyClass.attribute_one), - defer(MyClass.attribute_two)) - - A "path" can be specified onto a related or collection object using a - dotted name. The :func:`.orm.defer` option will be applied to that object - when loaded:: - - query(MyClass).options( - defer("related.attribute_one"), - defer("related.attribute_two")) - - To specify a path via class, send multiple arguments:: - - query(MyClass).options( - defer(MyClass.related, MyOtherClass.attribute_one), - defer(MyClass.related, MyOtherClass.attribute_two)) - - See also: - - :ref:`deferred` - - :param \*key: A key representing an individual path. Multiple entries - are accepted to allow a multiple-token path for a single target, not - multiple targets. - - """ - return _strategies.DeferredOption(key, defer=True) - - -def undefer(*key): - """Return a :class:`.MapperOption` that will convert the column property - of the given name into a non-deferred (regular column) load. - - Used with :meth:`.Query.options`. - - e.g.:: - - from sqlalchemy.orm import undefer - - query(MyClass).options( - undefer("attribute_one"), - undefer("attribute_two")) - - A class bound descriptor is also accepted:: - - query(MyClass).options( - undefer(MyClass.attribute_one), - undefer(MyClass.attribute_two)) - - A "path" can be specified onto a related or collection object using a - dotted name. The :func:`.orm.undefer` option will be applied to that - object when loaded:: - - query(MyClass).options( - undefer("related.attribute_one"), - undefer("related.attribute_two")) - - To specify a path via class, send multiple arguments:: - - query(MyClass).options( - undefer(MyClass.related, MyOtherClass.attribute_one), - undefer(MyClass.related, MyOtherClass.attribute_two)) - - See also: - - :func:`.orm.undefer_group` as a means to "undefer" a group - of attributes at once. - - :ref:`deferred` - - :param \*key: A key representing an individual path. Multiple entries - are accepted to allow a multiple-token path for a single target, not - multiple targets. - - """ - return _strategies.DeferredOption(key, defer=False) - - -def undefer_group(name): - """Return a :class:`.MapperOption` that will convert the given group of - deferred column properties into a non-deferred (regular column) load. - - Used with :meth:`.Query.options`. - - e.g.:: - - query(MyClass).options(undefer("group_one")) - - See also: - - :ref:`deferred` - - :param name: String name of the deferred group. This name is - established using the "group" name to the :func:`.orm.deferred` - configurational function. - - """ - return _strategies.UndeferGroupOption(name) - - def __go(lcls): global __all__ diff --git a/lib/sqlalchemy/orm/base.py b/lib/sqlalchemy/orm/base.py index f7d9dd4fe..47d8796b8 100644 --- a/lib/sqlalchemy/orm/base.py +++ b/lib/sqlalchemy/orm/base.py @@ -129,6 +129,19 @@ NOT_EXTENSION = util.symbol('NOT_EXTENSION') _none_set = frozenset([None]) +def _generative(*assertions): + """Mark a method as generative, e.g. method-chained.""" + + @util.decorator + def generate(fn, *args, **kw): + self = args[0]._clone() + for assertion in assertions: + assertion(self, fn.__name__) + fn(self, *args[1:], **kw) + return self + return generate + + # these can be replaced by sqlalchemy.ext.instrumentation # if augmented class instrumentation is enabled. def manager_of_class(cls): diff --git a/lib/sqlalchemy/orm/descriptor_props.py b/lib/sqlalchemy/orm/descriptor_props.py index bbfe602d0..daf125ea2 100644 --- a/lib/sqlalchemy/orm/descriptor_props.py +++ b/lib/sqlalchemy/orm/descriptor_props.py @@ -261,7 +261,8 @@ class CompositeProperty(DescriptorProperty): if self.deferred: prop.deferred = self.deferred prop.strategy_class = prop._strategy_lookup( - deferred=True, instrument=True) + ("deferred", True), + ("instrument", True)) prop.group = self.group def _setup_event_handlers(self): diff --git a/lib/sqlalchemy/orm/dynamic.py b/lib/sqlalchemy/orm/dynamic.py index 4631e806f..b419d2a07 100644 --- a/lib/sqlalchemy/orm/dynamic.py +++ b/lib/sqlalchemy/orm/dynamic.py @@ -20,7 +20,7 @@ from . import ( from .query import Query @log.class_logger -@properties.RelationshipProperty._strategy_for(dict(lazy="dynamic")) +@properties.RelationshipProperty.strategy_for(lazy="dynamic") class DynaLoader(strategies.AbstractRelationshipLoader): def init_class_attribute(self, mapper): self.is_class_level = True diff --git a/lib/sqlalchemy/orm/interfaces.py b/lib/sqlalchemy/orm/interfaces.py index 2f4aa5208..18723e4f6 100644 --- a/lib/sqlalchemy/orm/interfaces.py +++ b/lib/sqlalchemy/orm/interfaces.py @@ -21,7 +21,6 @@ from __future__ import absolute_import from .. import exc as sa_exc, util, inspect from ..sql import operators from collections import deque -from .base import _is_aliased_class, _class_to_mapper from .base import ONETOMANY, MANYTOONE, MANYTOMANY, EXT_CONTINUE, EXT_STOP, NOT_EXTENSION from .base import _InspectionAttr, _MappedAttribute from .path_registry import PathRegistry @@ -424,51 +423,57 @@ class StrategizedProperty(MapperProperty): strategy_wildcard_key = None - @util.memoized_property - def _wildcard_path(self): - if self.strategy_wildcard_key: - return ('loaderstrategy', (self.strategy_wildcard_key,)) - else: - return None + def _get_context_loader(self, context, path): + load = None - def _get_context_strategy(self, context, path): - strategy_cls = path._inlined_get_for(self, context, 'loaderstrategy') + # use EntityRegistry.__getitem__()->PropRegistry here so + # that the path is stated in terms of our base + search_path = dict.__getitem__(path, self) - if not strategy_cls: - wc_key = self._wildcard_path - if wc_key and wc_key in context.attributes: - strategy_cls = context.attributes[wc_key] + # search among: exact match, "attr.*", "default" strategy + # if any. + for path_key in ( + search_path._loader_key, + search_path._wildcard_path_loader_key, + search_path._default_path_loader_key + ): + if path_key in context.attributes: + load = context.attributes[path_key] + break - if strategy_cls: - try: - return self._strategies[strategy_cls] - except KeyError: - return self.__init_strategy(strategy_cls) - return self.strategy + return load - def _get_strategy(self, cls): + def _get_strategy(self, key): try: - return self._strategies[cls] + return self._strategies[key] except KeyError: - return self.__init_strategy(cls) + cls = self._strategy_lookup(*key) + self._strategies[key] = self._strategies[cls] = strategy = cls(self) + return strategy - def __init_strategy(self, cls): - self._strategies[cls] = strategy = cls(self) - return strategy + def _get_strategy_by_cls(self, cls): + return self._get_strategy(cls._strategy_keys[0]) def setup(self, context, entity, path, adapter, **kwargs): - self._get_context_strategy(context, path).\ - setup_query(context, entity, path, - adapter, **kwargs) + loader = self._get_context_loader(context, path) + if loader and loader.strategy: + strat = self._get_strategy(loader.strategy) + else: + strat = self.strategy + strat.setup_query(context, entity, path, loader, adapter, **kwargs) def create_row_processor(self, context, path, mapper, row, adapter): - return self._get_context_strategy(context, path).\ - create_row_processor(context, path, + loader = self._get_context_loader(context, path) + if loader and loader.strategy: + strat = self._get_strategy(loader.strategy) + else: + strat = self.strategy + return strat.create_row_processor(context, path, loader, mapper, row, adapter) def do_init(self): self._strategies = {} - self.strategy = self.__init_strategy(self.strategy_class) + self.strategy = self._get_strategy_by_cls(self.strategy_class) def post_instrument_class(self, mapper): if self.is_primary() and \ @@ -479,17 +484,17 @@ class StrategizedProperty(MapperProperty): _strategies = collections.defaultdict(dict) @classmethod - def _strategy_for(cls, *keys): + def strategy_for(cls, **kw): def decorate(dec_cls): - for key in keys: - key = tuple(sorted(key.items())) - cls._strategies[cls][key] = dec_cls + dec_cls._strategy_keys = [] + key = tuple(sorted(kw.items())) + cls._strategies[cls][key] = dec_cls + dec_cls._strategy_keys.append(key) return dec_cls return decorate @classmethod - def _strategy_lookup(cls, **kw): - key = tuple(sorted(kw.items())) + def _strategy_lookup(cls, *key): for prop_cls in cls.__mro__: if prop_cls in cls._strategies: strategies = cls._strategies[prop_cls] @@ -497,7 +502,7 @@ class StrategizedProperty(MapperProperty): return strategies[key] except KeyError: pass - raise Exception("can't locate strategy for %s %s" % (cls, kw)) + raise Exception("can't locate strategy for %s %s" % (cls, key)) class MapperOption(object): @@ -521,242 +526,6 @@ class MapperOption(object): self.process_query(query) -class PropertyOption(MapperOption): - """A MapperOption that is applied to a property off the mapper or - one of its child mappers, identified by a dot-separated key - or list of class-bound attributes. """ - - def __init__(self, key, mapper=None): - self.key = key - self.mapper = mapper - - def process_query(self, query): - self._process(query, True) - - def process_query_conditionally(self, query): - self._process(query, False) - - def _process(self, query, raiseerr): - paths = self._process_paths(query, raiseerr) - if paths: - self.process_query_property(query, paths) - - def process_query_property(self, query, paths): - pass - - def __getstate__(self): - d = self.__dict__.copy() - d['key'] = ret = [] - for token in util.to_list(self.key): - if isinstance(token, PropComparator): - ret.append((token._parentmapper.class_, token.key)) - else: - ret.append(token) - return d - - def __setstate__(self, state): - ret = [] - for key in state['key']: - if isinstance(key, tuple): - cls, propkey = key - ret.append(getattr(cls, propkey)) - else: - ret.append(key) - state['key'] = tuple(ret) - self.__dict__ = state - - def _find_entity_prop_comparator(self, query, token, mapper, raiseerr): - if _is_aliased_class(mapper): - searchfor = mapper - else: - searchfor = _class_to_mapper(mapper) - for ent in query._mapper_entities: - if ent.corresponds_to(searchfor): - return ent - else: - if raiseerr: - if not list(query._mapper_entities): - raise sa_exc.ArgumentError( - "Query has only expression-based entities - " - "can't find property named '%s'." - % (token, ) - ) - else: - raise sa_exc.ArgumentError( - "Can't find property '%s' on any entity " - "specified in this Query. Note the full path " - "from root (%s) to target entity must be specified." - % (token, ",".join(str(x) for - x in query._mapper_entities)) - ) - else: - return None - - def _find_entity_basestring(self, query, token, raiseerr): - for ent in query._mapper_entities: - # return only the first _MapperEntity when searching - # based on string prop name. Ideally object - # attributes are used to specify more exactly. - return ent - else: - if raiseerr: - raise sa_exc.ArgumentError( - "Query has only expression-based entities - " - "can't find property named '%s'." - % (token, ) - ) - else: - return None - - @util.dependencies("sqlalchemy.orm.util") - def _process_paths(self, orm_util, query, raiseerr): - """reconcile the 'key' for this PropertyOption with - the current path and entities of the query. - - Return a list of affected paths. - - """ - path = PathRegistry.root - entity = None - paths = [] - no_result = [] - - # _current_path implies we're in a - # secondary load with an existing path - current_path = list(query._current_path.path) - - tokens = deque(self.key) - while tokens: - token = tokens.popleft() - if isinstance(token, str): - # wildcard token - if token.endswith(':*'): - return [path.token(token)] - sub_tokens = token.split(".", 1) - token = sub_tokens[0] - tokens.extendleft(sub_tokens[1:]) - - # exhaust current_path before - # matching tokens to entities - if current_path: - if current_path[1].key == token: - current_path = current_path[2:] - continue - else: - return no_result - - if not entity: - entity = self._find_entity_basestring( - query, - token, - raiseerr) - if entity is None: - return no_result - path_element = entity.entity_zero - mapper = entity.mapper - - if hasattr(mapper.class_, token): - prop = getattr(mapper.class_, token).property - else: - if raiseerr: - raise sa_exc.ArgumentError( - "Can't find property named '%s' on the " - "mapped entity %s in this Query. " % ( - token, mapper) - ) - else: - return no_result - elif isinstance(token, PropComparator): - prop = token.property - - # exhaust current_path before - # matching tokens to entities - if current_path: - if current_path[0:2] == \ - [token._parententity, prop]: - current_path = current_path[2:] - continue - else: - return no_result - - if not entity: - entity = self._find_entity_prop_comparator( - query, - prop.key, - token._parententity, - raiseerr) - if not entity: - return no_result - - path_element = entity.entity_zero - mapper = entity.mapper - else: - raise sa_exc.ArgumentError( - "mapper option expects " - "string key or list of attributes") - assert prop is not None - if raiseerr and not prop.parent.common_parent(mapper): - raise sa_exc.ArgumentError("Attribute '%s' does not " - "link from element '%s'" % (token, path_element)) - - path = path[path_element][prop] - - paths.append(path) - - if getattr(token, '_of_type', None): - ac = token._of_type - ext_info = inspect(ac) - path_element = mapper = ext_info.mapper - if not ext_info.is_aliased_class: - ac = orm_util.with_polymorphic( - ext_info.mapper.base_mapper, - ext_info.mapper, aliased=True, - _use_mapper_path=True) - ext_info = inspect(ac) - path.set(query._attributes, "path_with_polymorphic", ext_info) - else: - path_element = mapper = getattr(prop, 'mapper', None) - if mapper is None and tokens: - raise sa_exc.ArgumentError( - "Attribute '%s' of entity '%s' does not " - "refer to a mapped entity" % - (token, entity) - ) - - if current_path: - # ran out of tokens before - # current_path was exhausted. - assert not tokens - return no_result - - return paths - - -class StrategizedOption(PropertyOption): - """A MapperOption that affects which LoaderStrategy will be used - for an operation by a StrategizedProperty. - """ - - chained = False - - def process_query_property(self, query, paths): - strategy = self.get_strategy_class() - if self.chained: - for path in paths: - path.set( - query._attributes, - "loaderstrategy", - strategy - ) - else: - paths[-1].set( - query._attributes, - "loaderstrategy", - strategy - ) - - def get_strategy_class(self): - raise NotImplementedError() class LoaderStrategy(object): @@ -791,10 +560,10 @@ class LoaderStrategy(object): def init_class_attribute(self, mapper): pass - def setup_query(self, context, entity, path, adapter, **kwargs): + def setup_query(self, context, entity, path, loadopt, adapter, **kwargs): pass - def create_row_processor(self, context, path, mapper, + def create_row_processor(self, context, path, loadopt, mapper, row, adapter): """Return row processing functions which fulfill the contract specified by MapperProperty.create_row_processor. diff --git a/lib/sqlalchemy/orm/path_registry.py b/lib/sqlalchemy/orm/path_registry.py index c9c91f905..fdc4f5654 100644 --- a/lib/sqlalchemy/orm/path_registry.py +++ b/lib/sqlalchemy/orm/path_registry.py @@ -9,12 +9,17 @@ from .. import inspection from .. import util +from .. import exc from itertools import chain from .base import class_mapper def _unreduce_path(path): return PathRegistry.deserialize(path) + +_WILDCARD_TOKEN = "*" +_DEFAULT_TOKEN = "_sa_default" + class PathRegistry(object): """Represent query load paths and registry functions. @@ -116,9 +121,13 @@ class PathRegistry(object): def coerce(cls, raw): return util.reduce(lambda prev, next: prev[next], raw, cls.root) - @classmethod - def token(cls, token): - return TokenRegistry(cls.root, token) + def token(self, token): + if token.endswith(':' + _WILDCARD_TOKEN): + return TokenRegistry(self, token) + elif token.endswith(":" + _DEFAULT_TOKEN): + return TokenRegistry(self.root, token) + else: + raise exc.ArgumentError("invalid token: %s" % token) def __add__(self, other): return util.reduce( @@ -135,9 +144,10 @@ class RootRegistry(PathRegistry): """ path = () - + has_entity = False def __getitem__(self, entity): return entity._path_registry + PathRegistry.root = RootRegistry() class TokenRegistry(PathRegistry): @@ -146,6 +156,8 @@ class TokenRegistry(PathRegistry): self.parent = parent self.path = parent.path + (token,) + has_entity = False + def __getitem__(self, entity): raise NotImplementedError() @@ -166,6 +178,47 @@ class PropRegistry(PathRegistry): self.parent = parent self.path = parent.path + (prop,) + @util.memoized_property + def has_entity(self): + return hasattr(self.prop, "mapper") + + @util.memoized_property + def entity(self): + return self.prop.mapper + + @util.memoized_property + def _wildcard_path_loader_key(self): + """Given a path (mapper A, prop X), replace the prop with the wildcard, + e.g. (mapper A, 'relationship:.*') or (mapper A, 'column:.*'), then + return within the ("loader", path) structure. + + """ + return ("loader", + self.parent.token( + "%s:%s" % (self.prop.strategy_wildcard_key, _WILDCARD_TOKEN) + ).path + ) + + @util.memoized_property + def _default_path_loader_key(self): + return ("loader", + self.parent.token( + "%s:%s" % (self.prop.strategy_wildcard_key, _DEFAULT_TOKEN) + ).path + ) + + @util.memoized_property + def _loader_key(self): + return ("loader", self.path) + + @property + def mapper(self): + return self.entity + + @property + def entity_path(self): + return self[self.entity] + def __getitem__(self, entity): if isinstance(entity, (int, slice)): return self.path[entity] @@ -174,16 +227,21 @@ class PropRegistry(PathRegistry): self, entity ) - class EntityRegistry(PathRegistry, dict): is_aliased_class = False + has_entity = True def __init__(self, parent, entity): self.key = entity self.parent = parent self.is_aliased_class = entity.is_aliased_class - + self.entity = entity self.path = parent.path + (entity,) + self.entity_path = self + + @property + def mapper(self): + return inspection.inspect(self.entity).mapper def __bool__(self): return True @@ -195,26 +253,9 @@ class EntityRegistry(PathRegistry, dict): else: return dict.__getitem__(self, entity) - def _inlined_get_for(self, prop, context, key): - """an inlined version of: - - cls = path[mapperproperty].get(context, key) - - Skips the isinstance() check in __getitem__ - and the extra method call for get(). - Used by StrategizedProperty for its - very frequent lookup. - - """ - path = dict.__getitem__(self, prop) - path_key = (key, path.path) - if path_key in context.attributes: - return context.attributes[path_key] - else: - return None - def __missing__(self, key): self[key] = item = PropRegistry(self, key) return item + diff --git a/lib/sqlalchemy/orm/properties.py b/lib/sqlalchemy/orm/properties.py index ef71d663c..c6eccf944 100644 --- a/lib/sqlalchemy/orm/properties.py +++ b/lib/sqlalchemy/orm/properties.py @@ -31,6 +31,8 @@ class ColumnProperty(StrategizedProperty): """ + strategy_wildcard_key = 'column' + def __init__(self, *columns, **kwargs): """Provide a column-level property for use with a Mapper. @@ -142,8 +144,9 @@ class ColumnProperty(StrategizedProperty): util.set_creation_order(self) self.strategy_class = self._strategy_lookup( - deferred=self.deferred, - instrument=self.instrument) + ("deferred", self.deferred), + ("instrument", self.instrument) + ) @property def expression(self): diff --git a/lib/sqlalchemy/orm/query.py b/lib/sqlalchemy/orm/query.py index beabc5811..ebfcf1087 100644 --- a/lib/sqlalchemy/orm/query.py +++ b/lib/sqlalchemy/orm/query.py @@ -24,7 +24,8 @@ from . import ( attributes, interfaces, object_mapper, persistence, exc as orm_exc, loading ) -from .base import _entity_descriptor, _is_aliased_class, _is_mapped_class, _orm_columns +from .base import _entity_descriptor, _is_aliased_class, \ + _is_mapped_class, _orm_columns, _generative from .path_registry import PathRegistry from .util import ( AliasedClass, ORMAdapter, join as orm_join, with_parent, aliased @@ -42,18 +43,6 @@ from . import properties __all__ = ['Query', 'QueryContext', 'aliased'] -def _generative(*assertions): - """Mark a method as generative.""" - - @util.decorator - def generate(fn, *args, **kw): - self = args[0]._clone() - for assertion in assertions: - assertion(self, fn.__name__) - fn(self, *args[1:], **kw) - return self - return generate - _path_registry = PathRegistry.root @inspection._self_inspects @@ -3438,28 +3427,29 @@ class QueryContext(object): class AliasOption(interfaces.MapperOption): def __init__(self, alias): - """Return a :class:`.MapperOption` that will indicate to the query that - the main table has been aliased. + """Return a :class:`.MapperOption` that will indicate to the :class:`.Query` + that the main table has been aliased. - This is used in the very rare case that :func:`.contains_eager` + This is a seldom-used option to suit the + very rare case that :func:`.contains_eager` is being used in conjunction with a user-defined SELECT statement that aliases the parent table. E.g.:: # define an aliased UNION called 'ulist' - statement = users.select(users.c.user_id==7).\\ + ulist = users.select(users.c.user_id==7).\\ union(users.select(users.c.user_id>7)).\\ alias('ulist') # add on an eager load of "addresses" - statement = statement.outerjoin(addresses).\\ + statement = ulist.outerjoin(addresses).\\ select().apply_labels() # create query, indicating "ulist" will be an # alias for the main table, "addresses" # property should be eager loaded query = session.query(User).options( - contains_alias('ulist'), - contains_eager('addresses')) + contains_alias(ulist), + contains_eager(User.addresses)) # then get results via the statement results = query.from_statement(statement).all() diff --git a/lib/sqlalchemy/orm/relationships.py b/lib/sqlalchemy/orm/relationships.py index f37bb8a4d..2393df26b 100644 --- a/lib/sqlalchemy/orm/relationships.py +++ b/lib/sqlalchemy/orm/relationships.py @@ -83,7 +83,7 @@ class RelationshipProperty(StrategizedProperty): """ - strategy_wildcard_key = 'relationship:*' + strategy_wildcard_key = 'relationship' _dependency_processor = None @@ -638,8 +638,7 @@ class RelationshipProperty(StrategizedProperty): if strategy_class: self.strategy_class = strategy_class else: - self.strategy_class = self._strategy_lookup(lazy=self.lazy) - self._lazy_strategy = self._strategy_lookup(lazy="select") + self.strategy_class = self._strategy_lookup(("lazy", self.lazy)) self._reverse_property = set() @@ -1149,7 +1148,7 @@ class RelationshipProperty(StrategizedProperty): alias_secondary=True): if value is not None: value = attributes.instance_state(value) - return self._get_strategy(self._lazy_strategy).lazy_clause(value, + return self._lazy_strategy.lazy_clause(value, reverse_direction=not value_is_parent, alias_secondary=alias_secondary, adapt_source=adapt_source) @@ -1361,6 +1360,8 @@ class RelationshipProperty(StrategizedProperty): self._post_init() self._generate_backref() super(RelationshipProperty, self).do_init() + self._lazy_strategy = self._get_strategy((("lazy", "select"),)) + def _process_dependent_arguments(self): """Convert incoming configuration arguments to their @@ -1602,7 +1603,7 @@ class RelationshipProperty(StrategizedProperty): """memoize the 'use_get' attribute of this RelationshipLoader's lazyloader.""" - strategy = self._get_strategy(self._lazy_strategy) + strategy = self._lazy_strategy return strategy.use_get @util.memoized_property diff --git a/lib/sqlalchemy/orm/strategies.py b/lib/sqlalchemy/orm/strategies.py index 761e6b999..6ca737c64 100644 --- a/lib/sqlalchemy/orm/strategies.py +++ b/lib/sqlalchemy/orm/strategies.py @@ -18,8 +18,7 @@ from .state import InstanceState from .util import _none_set from . import properties from .interfaces import ( - LoaderStrategy, StrategizedOption, MapperOption, PropertyOption, - StrategizedProperty + LoaderStrategy, StrategizedProperty ) from .session import _state_session import itertools @@ -88,7 +87,7 @@ def _register_attribute(strategy, mapper, useobject, for hook in listen_hooks: hook(desc, prop) -@properties.ColumnProperty._strategy_for(dict(instrument=False, deferred=False)) +@properties.ColumnProperty.strategy_for(instrument=False, deferred=False) class UninstrumentedColumnLoader(LoaderStrategy): """Represent the a non-instrumented MapperProperty. @@ -100,19 +99,19 @@ class UninstrumentedColumnLoader(LoaderStrategy): super(UninstrumentedColumnLoader, self).__init__(parent) self.columns = self.parent_property.columns - def setup_query(self, context, entity, path, adapter, + def setup_query(self, context, entity, path, loadopt, adapter, column_collection=None, **kwargs): for c in self.columns: if adapter: c = adapter.columns[c] column_collection.append(c) - def create_row_processor(self, context, path, mapper, row, adapter): + def create_row_processor(self, context, path, loadopt, mapper, row, adapter): return None, None, None @log.class_logger -@properties.ColumnProperty._strategy_for(dict(instrument=True, deferred=False)) +@properties.ColumnProperty.strategy_for(instrument=True, deferred=False) class ColumnLoader(LoaderStrategy): """Provide loading behavior for a :class:`.ColumnProperty`.""" @@ -121,7 +120,7 @@ class ColumnLoader(LoaderStrategy): self.columns = self.parent_property.columns self.is_composite = hasattr(self.parent_property, 'composite_class') - def setup_query(self, context, entity, path, + def setup_query(self, context, entity, path, loadopt, adapter, column_collection, **kwargs): for c in self.columns: if adapter: @@ -142,7 +141,7 @@ class ColumnLoader(LoaderStrategy): ) def create_row_processor(self, context, path, - mapper, row, adapter): + loadopt, mapper, row, adapter): key = self.key # look through list of columns represented here # to see which, if any, is present in the row. @@ -161,7 +160,7 @@ class ColumnLoader(LoaderStrategy): @log.class_logger -@properties.ColumnProperty._strategy_for(dict(deferred=True, instrument=True)) +@properties.ColumnProperty.strategy_for(deferred=True, instrument=True) class DeferredColumnLoader(LoaderStrategy): """Provide loading behavior for a deferred :class:`.ColumnProperty`.""" @@ -173,16 +172,16 @@ class DeferredColumnLoader(LoaderStrategy): self.columns = self.parent_property.columns self.group = self.parent_property.group - def create_row_processor(self, context, path, mapper, row, adapter): + def create_row_processor(self, context, path, loadopt, mapper, row, adapter): col = self.columns[0] if adapter: col = adapter.columns[col] key = self.key if col in row: - return self.parent_property._get_strategy(ColumnLoader).\ + return self.parent_property._get_strategy_by_cls(ColumnLoader).\ create_row_processor( - context, path, mapper, row, adapter) + context, path, loadopt, mapper, row, adapter) elif not self.is_class_level: set_deferred_for_local_state = InstanceState._row_processor( @@ -205,15 +204,15 @@ class DeferredColumnLoader(LoaderStrategy): expire_missing=False ) - def setup_query(self, context, entity, path, adapter, + def setup_query(self, context, entity, path, loadopt, adapter, only_load_props=None, **kwargs): if ( - self.group is not None and - context.attributes.get(('undefer', self.group), False) + loadopt and self.group and + loadopt.local_opts.get('undefer_group', False) == self.group ) or (only_load_props and self.key in only_load_props): - self.parent_property._get_strategy(ColumnLoader).\ + self.parent_property._get_strategy_by_cls(ColumnLoader).\ setup_query(context, entity, - path, adapter, **kwargs) + path, loadopt, adapter, **kwargs) def _load_for_state(self, state, passive): if not state.key: @@ -270,29 +269,6 @@ class LoadDeferredColumns(object): return strategy._load_for_state(state, passive) -class DeferredOption(StrategizedOption): - propagate_to_loaders = True - - def __init__(self, key, defer=False): - super(DeferredOption, self).__init__(key) - self.defer = defer - - def get_strategy_class(self): - if self.defer: - return DeferredColumnLoader - else: - return ColumnLoader - - -class UndeferGroupOption(MapperOption): - propagate_to_loaders = True - - def __init__(self, group): - self.group = group - - def process_query(self, query): - query._attributes[("undefer", self.group)] = True - class AbstractRelationshipLoader(LoaderStrategy): """LoaderStratgies which deal with related objects.""" @@ -306,7 +282,8 @@ class AbstractRelationshipLoader(LoaderStrategy): @log.class_logger -@properties.RelationshipProperty._strategy_for(dict(lazy=None), dict(lazy="noload")) +@properties.RelationshipProperty.strategy_for(lazy="noload") +@properties.RelationshipProperty.strategy_for(lazy=None) class NoLoader(AbstractRelationshipLoader): """Provide loading behavior for a :class:`.RelationshipProperty` with "lazy=None". @@ -322,7 +299,7 @@ class NoLoader(AbstractRelationshipLoader): typecallable=self.parent_property.collection_class, ) - def create_row_processor(self, context, path, mapper, row, adapter): + def create_row_processor(self, context, path, loadopt, mapper, row, adapter): def invoke_no_load(state, dict_, row): state._initialize(self.key) return invoke_no_load, None, None @@ -330,7 +307,8 @@ class NoLoader(AbstractRelationshipLoader): @log.class_logger -@properties.RelationshipProperty._strategy_for(dict(lazy=True), dict(lazy="select")) +@properties.RelationshipProperty.strategy_for(lazy=True) +@properties.RelationshipProperty.strategy_for(lazy="select") class LazyLoader(AbstractRelationshipLoader): """Provide loading behavior for a :class:`.RelationshipProperty` with "lazy=True", that is loads when first accessed. @@ -544,7 +522,8 @@ class LazyLoader(AbstractRelationshipLoader): for pk in self.mapper.primary_key ] - def _emit_lazyload(self, session, state, ident_key, passive): + @util.dependencies("sqlalchemy.orm.strategy_options") + def _emit_lazyload(self, strategy_options, session, state, ident_key, passive): q = session.query(self.mapper)._adapt_all_clauses() q = q._with_invoke_all_eagers(False) @@ -573,7 +552,7 @@ class LazyLoader(AbstractRelationshipLoader): if rev.direction is interfaces.MANYTOONE and \ rev._use_get and \ not isinstance(rev.strategy, LazyLoader): - q = q.options(EagerLazyOption((rev.key,), lazy='select')) + q = q.options(strategy_options.Load(rev.parent).lazyload(rev.key)) lazy_clause = self.lazy_clause(state, passive=passive) @@ -600,7 +579,7 @@ class LazyLoader(AbstractRelationshipLoader): else: return None - def create_row_processor(self, context, path, + def create_row_processor(self, context, path, loadopt, mapper, row, adapter): key = self.key if not self.is_class_level: @@ -648,19 +627,19 @@ class LoadLazyAttribute(object): return strategy._load_for_state(state, passive) -@properties.RelationshipProperty._strategy_for(dict(lazy="immediate")) +@properties.RelationshipProperty.strategy_for(lazy="immediate") class ImmediateLoader(AbstractRelationshipLoader): def init_class_attribute(self, mapper): self.parent_property.\ - _get_strategy(LazyLoader).\ + _get_strategy_by_cls(LazyLoader).\ init_class_attribute(mapper) def setup_query(self, context, entity, - path, adapter, column_collection=None, + path, loadopt, adapter, column_collection=None, parentmapper=None, **kwargs): pass - def create_row_processor(self, context, path, + def create_row_processor(self, context, path, loadopt, mapper, row, adapter): def load_immediate(state, dict_, row): state.get_impl(self.key).get(state, dict_) @@ -669,7 +648,7 @@ class ImmediateLoader(AbstractRelationshipLoader): @log.class_logger -@properties.RelationshipProperty._strategy_for(dict(lazy="subquery")) +@properties.RelationshipProperty.strategy_for(lazy="subquery") class SubqueryLoader(AbstractRelationshipLoader): def __init__(self, parent): super(SubqueryLoader, self).__init__(parent) @@ -677,11 +656,11 @@ class SubqueryLoader(AbstractRelationshipLoader): def init_class_attribute(self, mapper): self.parent_property.\ - _get_strategy(LazyLoader).\ + _get_strategy_by_cls(LazyLoader).\ init_class_attribute(mapper) def setup_query(self, context, entity, - path, adapter, + path, loadopt, adapter, column_collection=None, parentmapper=None, **kwargs): @@ -706,7 +685,7 @@ class SubqueryLoader(AbstractRelationshipLoader): # if not via query option, check for # a cycle - if not path.contains(context.attributes, "loaderstrategy"): + if not path.contains(context.attributes, "loader"): if self.join_depth: if path.length / 2 > self.join_depth: return @@ -919,7 +898,7 @@ class SubqueryLoader(AbstractRelationshipLoader): q = q.order_by(*eager_order_by) return q - def create_row_processor(self, context, path, + def create_row_processor(self, context, path, loadopt, mapper, row, adapter): if not self.parent.class_manager[self.key].impl.supports_population: raise sa_exc.InvalidRequestError( @@ -989,7 +968,8 @@ class SubqueryLoader(AbstractRelationshipLoader): @log.class_logger -@properties.RelationshipProperty._strategy_for(dict(lazy=False), dict(lazy="joined")) +@properties.RelationshipProperty.strategy_for(lazy="joined") +@properties.RelationshipProperty.strategy_for(lazy=False) class JoinedLoader(AbstractRelationshipLoader): """Provide loading behavior for a :class:`.RelationshipProperty` using joined eager loading. @@ -1001,9 +981,9 @@ class JoinedLoader(AbstractRelationshipLoader): def init_class_attribute(self, mapper): self.parent_property.\ - _get_strategy(LazyLoader).init_class_attribute(mapper) + _get_strategy_by_cls(LazyLoader).init_class_attribute(mapper) - def setup_query(self, context, entity, path, adapter, \ + def setup_query(self, context, entity, path, loadopt, adapter, \ column_collection=None, parentmapper=None, allow_innerjoin=True, **kwargs): @@ -1016,19 +996,19 @@ class JoinedLoader(AbstractRelationshipLoader): with_polymorphic = None - user_defined_adapter = path.get(context.attributes, - "user_defined_eager_row_processor", - False) + user_defined_adapter = self._init_user_defined_eager_proc( + loadopt, context) if loadopt else False + if user_defined_adapter is not False: clauses, adapter, add_to_collection = \ - self._get_user_defined_adapter( + self._setup_query_on_user_defined_adapter( context, entity, path, adapter, user_defined_adapter ) else: # if not via query option, check for # a cycle - if not path.contains(context.attributes, "loaderstrategy"): + if not path.contains(context.attributes, "loader"): if self.join_depth: if path.length / 2 > self.join_depth: return @@ -1037,7 +1017,7 @@ class JoinedLoader(AbstractRelationshipLoader): clauses, adapter, add_to_collection, \ allow_innerjoin = self._generate_row_adapter( - context, entity, path, adapter, + context, entity, path, loadopt, adapter, column_collection, parentmapper, allow_innerjoin ) @@ -1072,24 +1052,74 @@ class JoinedLoader(AbstractRelationshipLoader): "when using joined loading with with_polymorphic()." ) - def _get_user_defined_adapter(self, context, entity, + def _init_user_defined_eager_proc(self, loadopt, context): + + # check if the opt applies at all + if "eager_from_alias" not in loadopt.local_opts: + # nope + return False + + path = loadopt.path.parent + + # the option applies. check if the "user_defined_eager_row_processor" + # has been built up. + adapter = path.get(context.attributes, + "user_defined_eager_row_processor", False) + if adapter is not False: + # just return it + return adapter + + # otherwise figure it out. + alias = loadopt.local_opts["eager_from_alias"] + + root_mapper, prop = path[-2:] + + #from .mapper import Mapper + #from .interfaces import MapperProperty + #assert isinstance(root_mapper, Mapper) + #assert isinstance(prop, MapperProperty) + + if alias is not None: + if isinstance(alias, str): + alias = prop.target.alias(alias) + adapter = sql_util.ColumnAdapter(alias, + equivalents=prop.mapper._equivalent_columns) + else: + if path.contains(context.attributes, "path_with_polymorphic"): + with_poly_info = path.get(context.attributes, + "path_with_polymorphic") + adapter = orm_util.ORMAdapter( + with_poly_info.entity, + equivalents=prop.mapper._equivalent_columns) + else: + adapter = context.query._polymorphic_adapters.get(prop.mapper, None) + path.set(context.attributes, + "user_defined_eager_row_processor", + adapter) + + return adapter + + def _setup_query_on_user_defined_adapter(self, context, entity, path, adapter, user_defined_adapter): - adapter = entity._get_entity_clauses(context.query, context) - if adapter and user_defined_adapter: - user_defined_adapter = user_defined_adapter.wrap(adapter) - path.set(context.attributes, "user_defined_eager_row_processor", - user_defined_adapter) - elif adapter: - user_defined_adapter = adapter - path.set(context.attributes, "user_defined_eager_row_processor", - user_defined_adapter) + # apply some more wrapping to the "user defined adapter" + # if we are setting up the query for SQL render. + adapter = entity._get_entity_clauses(context.query, context) + + if adapter and user_defined_adapter: + user_defined_adapter = user_defined_adapter.wrap(adapter) + path.set(context.attributes, "user_defined_eager_row_processor", + user_defined_adapter) + elif adapter: + user_defined_adapter = adapter + path.set(context.attributes, "user_defined_eager_row_processor", + user_defined_adapter) - add_to_collection = context.primary_columns - return user_defined_adapter, adapter, add_to_collection + add_to_collection = context.primary_columns + return user_defined_adapter, adapter, add_to_collection def _generate_row_adapter(self, - context, entity, path, adapter, + context, entity, path, loadopt, adapter, column_collection, parentmapper, allow_innerjoin ): with_poly_info = path.get( @@ -1112,9 +1142,12 @@ class JoinedLoader(AbstractRelationshipLoader): if self.parent_property.direction != interfaces.MANYTOONE: context.multi_row_eager_loaders = True - innerjoin = allow_innerjoin and path.get(context.attributes, - "eager_join_type", - self.parent_property.innerjoin) + innerjoin = allow_innerjoin and ( + loadopt.local_opts.get( + 'innerjoin', self.parent_property.innerjoin) + if loadopt is not None + else self.parent_property.innerjoin + ) if not innerjoin: # if this is an outer join, all eager joins from # here must also be outer joins @@ -1221,10 +1254,10 @@ class JoinedLoader(AbstractRelationshipLoader): ) ) - def _create_eager_adapter(self, context, row, adapter, path): - user_defined_adapter = path.get(context.attributes, - "user_defined_eager_row_processor", - False) + def _create_eager_adapter(self, context, row, adapter, path, loadopt): + user_defined_adapter = self._init_user_defined_eager_proc( + loadopt, context) if loadopt else False + if user_defined_adapter is not False: decorator = user_defined_adapter # user defined eagerloads are part of the "primary" @@ -1247,7 +1280,7 @@ class JoinedLoader(AbstractRelationshipLoader): # processor, will cause a degrade to lazy return False - def create_row_processor(self, context, path, mapper, row, adapter): + def create_row_processor(self, context, path, loadopt, mapper, row, adapter): if not self.parent.class_manager[self.key].impl.supports_population: raise sa_exc.InvalidRequestError( "'%s' does not support object " @@ -1259,7 +1292,7 @@ class JoinedLoader(AbstractRelationshipLoader): eager_adapter = self._create_eager_adapter( context, row, - adapter, our_path) + adapter, our_path, loadopt) if eager_adapter is not False: key = self.key @@ -1276,9 +1309,9 @@ class JoinedLoader(AbstractRelationshipLoader): return self._create_collection_loader(context, key, _instance) else: return self.parent_property.\ - _get_strategy(LazyLoader).\ + _get_strategy_by_cls(LazyLoader).\ create_row_processor( - context, path, + context, path, loadopt, mapper, row, adapter) def _create_collection_loader(self, context, key, _instance): @@ -1339,84 +1372,6 @@ class JoinedLoader(AbstractRelationshipLoader): None, load_scalar_from_joined_exec -class EagerLazyOption(StrategizedOption): - def __init__(self, key, lazy=True, chained=False, - propagate_to_loaders=True - ): - if isinstance(key[0], str) and key[0] == '*': - if len(key) != 1: - raise sa_exc.ArgumentError( - "Wildcard identifier '*' must " - "be specified alone.") - key = ("relationship:*",) - propagate_to_loaders = False - super(EagerLazyOption, self).__init__(key) - self.lazy = lazy - self.chained = chained - self.propagate_to_loaders = propagate_to_loaders - self.strategy_cls = properties.RelationshipProperty._strategy_lookup(lazy=lazy) - - def get_strategy_class(self): - return self.strategy_cls - - -class EagerJoinOption(PropertyOption): - - def __init__(self, key, innerjoin, chained=False): - super(EagerJoinOption, self).__init__(key) - self.innerjoin = innerjoin - self.chained = chained - - def process_query_property(self, query, paths): - if self.chained: - for path in paths: - path.set(query._attributes, "eager_join_type", self.innerjoin) - else: - paths[-1].set(query._attributes, "eager_join_type", self.innerjoin) - - -class LoadEagerFromAliasOption(PropertyOption): - - def __init__(self, key, alias=None, chained=False): - super(LoadEagerFromAliasOption, self).__init__(key) - if alias is not None: - if not isinstance(alias, str): - info = inspect(alias) - alias = info.selectable - self.alias = alias - self.chained = chained - - def process_query_property(self, query, paths): - if self.chained: - for path in paths[0:-1]: - (root_mapper, prop) = path.path[-2:] - adapter = query._polymorphic_adapters.get(prop.mapper, None) - path.setdefault(query._attributes, - "user_defined_eager_row_processor", - adapter) - - root_mapper, prop = paths[-1].path[-2:] - if self.alias is not None: - if isinstance(self.alias, str): - self.alias = prop.target.alias(self.alias) - paths[-1].set(query._attributes, - "user_defined_eager_row_processor", - sql_util.ColumnAdapter(self.alias, - equivalents=prop.mapper._equivalent_columns) - ) - else: - if paths[-1].contains(query._attributes, "path_with_polymorphic"): - with_poly_info = paths[-1].get(query._attributes, - "path_with_polymorphic") - adapter = orm_util.ORMAdapter( - with_poly_info.entity, - equivalents=prop.mapper._equivalent_columns) - else: - adapter = query._polymorphic_adapters.get(prop.mapper, None) - paths[-1].set(query._attributes, - "user_defined_eager_row_processor", - adapter) - def single_parent_validator(desc, prop): def _do_check(state, value, oldvalue, initiator): diff --git a/lib/sqlalchemy/orm/strategy_options.py b/lib/sqlalchemy/orm/strategy_options.py new file mode 100644 index 000000000..5f7eb2c25 --- /dev/null +++ b/lib/sqlalchemy/orm/strategy_options.py @@ -0,0 +1,893 @@ +# orm/strategy_options.py +# Copyright (C) 2005-2013 the SQLAlchemy authors and contributors <see AUTHORS file> +# +# This module is part of SQLAlchemy and is released under +# the MIT License: http://www.opensource.org/licenses/mit-license.php + +""" + +""" + +from .interfaces import MapperOption, PropComparator +from .. import util +from ..sql.base import _generative, Generative +from .. import exc as sa_exc, inspect +from .base import _is_aliased_class, _class_to_mapper +from . import util as orm_util +from .path_registry import PathRegistry, TokenRegistry, \ + _WILDCARD_TOKEN, _DEFAULT_TOKEN + +class Load(Generative, MapperOption): + """Represents loader options which modify the state of a + :class:`.Query` in order to affect how various mapped attributes are loaded. + + .. versionadded:: 0.9.0 The :meth:`.Load` system is a new foundation for + the existing system of loader options, including options such as + :func:`.orm.joinedload`, :func:`.orm.defer`, and others. In particular, + it introduces a new method-chained system that replaces the need for + dot-separated paths as well as "_all()" options such as :func:`.orm.joinedload_all`. + + A :class:`.Load` object can be used directly or indirectly. To use one + directly, instantiate given the parent class. This style of usage is + useful when dealing with a :class:`.Query` that has multiple entities, + or when producing a loader option that can be applied generically to + any style of query:: + + myopt = Load(MyClass).joinedload("widgets") + + The above ``myopt`` can now be used with :meth:`.Query.options`:: + + session.query(MyClass).options(myopt) + + The :class:`.Load` construct is invoked indirectly whenever one makes use + of the various loader options that are present in ``sqlalchemy.orm``, including + options such as :func:`.orm.joinedload`, :func:`.orm.defer`, :func:`.orm.subqueryload`, + and all the rest. These constructs produce an "anonymous" form of the + :class:`.Load` object which tracks attributes and options, but is not linked + to a parent class until it is associated with a parent :class:`.Query`:: + + # produce "unbound" Load object + myopt = joinedload("widgets") + + # when applied using options(), the option is "bound" to the + # class observed in the given query, e.g. MyClass + session.query(MyClass).options(myopt) + + Whether the direct or indirect style is used, the :class:`.Load` object + returned now represents a specific "path" along the entities of a :class:`.Query`. + This path can be traversed using a standard method-chaining approach. + Supposing a class hierarchy such as ``User``, ``User.addresses -> Address``, + ``User.orders -> Order`` and ``Order.items -> Item``, we can specify a variety + of loader options along each element in the "path":: + + session.query(User).options( + joinedload("addresses"), + subqueryload("orders").joinedload("items") + ) + + Where above, the ``addresses`` collection will be joined-loaded, the + ``orders`` collection will be subquery-loaded, and within that subquery load + the ``items`` collection will be joined-loaded. + + + """ + def __init__(self, entity): + insp = inspect(entity) + self.path = insp._path_registry + self.context = {} + self.local_opts = {} + + def _generate(self): + cloned = super(Load, self)._generate() + cloned.local_opts = {} + return cloned + + strategy = None + propagate_to_loaders = False + + def process_query(self, query): + self._process(query, True) + + def process_query_conditionally(self, query): + self._process(query, False) + + def _process(self, query, raiseerr): + query._attributes.update(self.context) + + def _generate_path(self, path, attr, wildcard_key, raiseerr=True): + if raiseerr and not path.has_entity: + if isinstance(path, TokenRegistry): + raise sa_exc.ArgumentError( + "Wildcard token cannot be followed by another entity") + else: + raise sa_exc.ArgumentError( + "Attribute '%s' of entity '%s' does not " + "refer to a mapped entity" % + (path.prop.key, path.parent.entity) + ) + + if isinstance(attr, util.string_types): + if attr.endswith(_WILDCARD_TOKEN) or attr.endswith(_DEFAULT_TOKEN): + if wildcard_key: + attr = "%s:%s" % (wildcard_key, attr) + self.propagate_to_loaders = False + return path.token(attr) + + try: + # use getattr on the class to work around + # synonyms, hybrids, etc. + attr = getattr(path.entity.class_, attr) + except AttributeError: + if raiseerr: + raise sa_exc.ArgumentError( + "Can't find property named '%s' on the " + "mapped entity %s in this Query. " % ( + attr, path.entity) + ) + else: + return None + else: + attr = attr.property + + path = path[attr] + else: + prop = attr.property + + if not prop.parent.common_parent(path.mapper): + if raiseerr: + raise sa_exc.ArgumentError("Attribute '%s' does not " + "link from element '%s'" % (attr, path.entity)) + else: + return None + + if getattr(attr, '_of_type', None): + ac = attr._of_type + ext_info = inspect(ac) + + path_element = ext_info.mapper + if not ext_info.is_aliased_class: + ac = orm_util.with_polymorphic( + ext_info.mapper.base_mapper, + ext_info.mapper, aliased=True, + _use_mapper_path=True) + path.entity_path[prop].set(self.context, + "path_with_polymorphic", inspect(ac)) + path = path[prop][path_element] + else: + path = path[prop] + + if path.has_entity: + path = path.entity_path + return path + + def _coerce_strat(self, strategy): + if strategy is not None: + strategy = tuple(strategy.items()) + return strategy + + @_generative + def set_relationship_strategy(self, attr, strategy, propagate_to_loaders=True): + strategy = self._coerce_strat(strategy) + + self.propagate_to_loaders = propagate_to_loaders + # if the path is a wildcard, this will set propagate_to_loaders=False + self.path = self._generate_path(self.path, attr, "relationship") + self.strategy = strategy + if strategy is not None: + self._set_path_strategy() + + @_generative + def set_column_strategy(self, attrs, strategy, opts=None): + strategy = self._coerce_strat(strategy) + + for attr in attrs: + path = self._generate_path(self.path, attr, "column") + cloned = self._generate() + cloned.strategy = strategy + cloned.path = path + cloned.propagate_to_loaders = True + if opts: + cloned.local_opts.update(opts) + cloned._set_path_strategy() + + def _set_path_strategy(self): + if self.path.has_entity: + self.path.parent.set(self.context, "loader", self) + else: + self.path.set(self.context, "loader", self) + + def __getstate__(self): + d = self.__dict__.copy() + d["path"] = self.path.serialize() + return d + + def __setstate__(self, state): + self.__dict__.update(state) + self.path = PathRegistry.deserialize(self.path) + + +class _UnboundLoad(Load): + """Represent a loader option that isn't tied to a root entity. + + The loader option will produce an entity-linked :class:`.Load` + object when it is passed :meth:`.Query.options`. + + This provides compatibility with the traditional system + of freestanding options, e.g. ``joinedload('x.y.z')``. + + """ + def __init__(self): + self.path = () + self._to_bind = set() + self.local_opts = {} + + _is_chain_link = False + + def _set_path_strategy(self): + self._to_bind.add(self) + + def _generate_path(self, path, attr, wildcard_key): + if wildcard_key and isinstance(attr, util.string_types) and \ + attr in (_WILDCARD_TOKEN, _DEFAULT_TOKEN): + attr = "%s:%s" % (wildcard_key, attr) + self.propagate_to_loaders = False + + return path + (attr, ) + + def __getstate__(self): + d = self.__dict__.copy() + d['path'] = ret = [] + for token in util.to_list(self.path): + if isinstance(token, PropComparator): + ret.append((token._parentmapper.class_, token.key)) + else: + ret.append(token) + return d + + def __setstate__(self, state): + ret = [] + for key in state['path']: + if isinstance(key, tuple): + cls, propkey = key + ret.append(getattr(cls, propkey)) + else: + ret.append(key) + state['path'] = tuple(ret) + self.__dict__ = state + + def _process(self, query, raiseerr): + for val in self._to_bind: + val._bind_loader(query, query._attributes, raiseerr) + + @classmethod + def _from_keys(self, meth, keys, chained, kw): + opt = _UnboundLoad() + + def _split_key(key): + if isinstance(key, util.string_types): + # coerce fooload('*') into "default loader strategy" + if key == _WILDCARD_TOKEN: + return (_DEFAULT_TOKEN, ) + # coerce fooload(".*") into "wildcard on default entity" + elif key.startswith("." + _WILDCARD_TOKEN): + key = key[1:] + return key.split(".") + else: + return (key,) + all_tokens = [token for key in keys for token in _split_key(key)] + + for token in all_tokens[0:-1]: + if chained: + opt = meth(opt, token, **kw) + else: + opt = opt.defaultload(token) + opt._is_chain_link = True + + opt = meth(opt, all_tokens[-1], **kw) + opt._is_chain_link = False + + return opt + + + def _bind_loader(self, query, context, raiseerr): + start_path = self.path + # _current_path implies we're in a + # secondary load with an existing path + + current_path = query._current_path + if current_path: + start_path = self._chop_path(start_path, current_path) + if not start_path: + return None + + token = start_path[0] + if isinstance(token, util.string_types): + entity = self._find_entity_basestring(query, token, raiseerr) + elif isinstance(token, PropComparator): + prop = token.property + entity = self._find_entity_prop_comparator( + query, + prop.key, + token._parententity, + raiseerr) + + else: + raise sa_exc.ArgumentError( + "mapper option expects " + "string key or list of attributes") + + if not entity: + return + + path_element = entity.entity_zero + + # transfer our entity-less state into a Load() object + # with a real entity path. + loader = Load(path_element) + loader.context = context + loader.strategy = self.strategy + + path = loader.path + for token in start_path: + loader.path = path = loader._generate_path( + loader.path, token, None, raiseerr) + if path is None: + return + + loader.local_opts.update(self.local_opts) + + if loader.path.has_entity: + effective_path = loader.path.parent + else: + effective_path = loader.path + + # prioritize "first class" options over those + # that were "links in the chain", e.g. "x" and "y" in someload("x.y.z") + # versus someload("x") / someload("x.y") + if self._is_chain_link: + effective_path.setdefault(context, "loader", loader) + else: + effective_path.set(context, "loader", loader) + + def _chop_path(self, to_chop, path): + i = -1 + for i, (c_token, (p_mapper, p_prop)) in enumerate(zip(to_chop, path.pairs())): + if isinstance(c_token, util.string_types): + if i == 0 and c_token.endswith(':' + _DEFAULT_TOKEN): + return to_chop + elif c_token != 'relationship:%s' % (_WILDCARD_TOKEN,) and c_token != p_prop.key: + return None + elif isinstance(c_token, PropComparator): + if c_token.property is not p_prop: + return None + else: + i += 1 + + return to_chop[i:] + + def _find_entity_prop_comparator(self, query, token, mapper, raiseerr): + if _is_aliased_class(mapper): + searchfor = mapper + else: + searchfor = _class_to_mapper(mapper) + for ent in query._mapper_entities: + if ent.corresponds_to(searchfor): + return ent + else: + if raiseerr: + if not list(query._mapper_entities): + raise sa_exc.ArgumentError( + "Query has only expression-based entities - " + "can't find property named '%s'." + % (token, ) + ) + else: + raise sa_exc.ArgumentError( + "Can't find property '%s' on any entity " + "specified in this Query. Note the full path " + "from root (%s) to target entity must be specified." + % (token, ",".join(str(x) for + x in query._mapper_entities)) + ) + else: + return None + + def _find_entity_basestring(self, query, token, raiseerr): + if token.endswith(':' + _WILDCARD_TOKEN): + if len(list(query._mapper_entities)) != 1: + if raiseerr: + raise sa_exc.ArgumentError( + "Wildcard loader can only be used with exactly " + "one entity. Use Load(ent) to specify " + "specific entities.") + + for ent in query._mapper_entities: + # return only the first _MapperEntity when searching + # based on string prop name. Ideally object + # attributes are used to specify more exactly. + return ent + else: + if raiseerr: + raise sa_exc.ArgumentError( + "Query has only expression-based entities - " + "can't find property named '%s'." + % (token, ) + ) + else: + return None + + + +class loader_option(object): + def __init__(self): + pass + + def __call__(self, fn): + self.name = name = fn.__name__ + self.fn = fn + if hasattr(Load, name): + raise TypeError("Load class already has a %s method." % (name)) + setattr(Load, name, fn) + + return self + + def _add_unbound_fn(self, fn): + self._unbound_fn = fn + fn_doc = self.fn.__doc__ + self.fn.__doc__ = """Produce a new :class:`.Load` object with the +:func:`.orm.%(name)s` option applied. + +See :func:`.orm.%(name)s` for usage examples. + +""" % {"name": self.name} + + fn.__doc__ = fn_doc + return self + + def _add_unbound_all_fn(self, fn): + self._unbound_all_fn = fn + fn.__doc__ = """Produce a standalone "all" option for :func:`.orm.%(name)s`. + +.. deprecated:: 0.9.0 + + The "_all()" style is replaced by method chaining, e.g.:: + + session.query(MyClass).options( + %(name)s("someattribute").%(name)s("anotherattribute") + ) + +""" % {"name": self.name} + return self + +@loader_option() +def contains_eager(loadopt, attr, alias=None): + """Indicate that the given attribute should be eagerly loaded from + columns stated manually in the query. + + This function is part of the :class:`.Load` interface and supports + both method-chained and standalone operation. + + The option is used in conjunction with an explicit join that loads + the desired rows, i.e.:: + + sess.query(Order).\\ + join(Order.user).\\ + options(contains_eager(Order.user)) + + The above query would join from the ``Order`` entity to its related + ``User`` entity, and the returned ``Order`` objects would have the + ``Order.user`` attribute pre-populated. + + :func:`contains_eager` also accepts an `alias` argument, which is the + string name of an alias, an :func:`~sqlalchemy.sql.expression.alias` + construct, or an :func:`~sqlalchemy.orm.aliased` construct. Use this when + the eagerly-loaded rows are to come from an aliased table:: + + user_alias = aliased(User) + sess.query(Order).\\ + join((user_alias, Order.user)).\\ + options(contains_eager(Order.user, alias=user_alias)) + + .. seealso:: + + :ref:`contains_eager` + + """ + if alias is not None: + if not isinstance(alias, str): + info = inspect(alias) + alias = info.selectable + + cloned = loadopt.set_relationship_strategy( + attr, + {"lazy": "joined"}, + propagate_to_loaders=False + ) + cloned.local_opts['eager_from_alias'] = alias + return cloned + +@contains_eager._add_unbound_fn +def contains_eager(*keys, **kw): + return _UnboundLoad()._from_keys(_UnboundLoad.contains_eager, keys, True, kw) + +@loader_option() +def load_only(loadopt, *attrs): + """Indicate that for a particular entity, only the given list + of column-based attribute names should be loaded; all others will be + deferred. + + This function is part of the :class:`.Load` interface and supports + both method-chained and standalone operation. + + Example - given a class ``User``, load only the ``name`` and ``fullname`` + attributes:: + + session.query(User).options(load_only("name", "fullname")) + + Example - given a relationship ``User.addresses -> Address``, specify + subquery loading for the ``User.addresses`` collection, but on each ``Address`` + object load only the ``email_address`` attribute:: + + session.query(User).options( + subqueryload("addreses").load_only("email_address") + ) + + For a :class:`.Query` that has multiple entities, the lead entity can be + specifically referred to using the :class:`.Load` constructor:: + + session.query(User, Address).join(User.addresses).options( + Load(User).load_only("name", "fullname"), + Load(Address).load_only("email_addres") + ) + + + .. versionadded:: 0.9.0 + + """ + cloned = loadopt.set_column_strategy( + attrs, + {"deferred": False, "instrument": True} + ) + cloned.set_column_strategy("*", + {"deferred": True, "instrument": True}) + return cloned + +@load_only._add_unbound_fn +def load_only(*attrs): + return _UnboundLoad().load_only(*attrs) + +@loader_option() +def joinedload(loadopt, attr, innerjoin=None): + """Indicate that the given attribute should be loaded using joined + eager loading. + + This function is part of the :class:`.Load` interface and supports + both method-chained and standalone operation. + + examples:: + + # joined-load the "orders" collection on "User" + query(User).options(joinedload(User.orders)) + + # joined-load Order.items and then Item.keywords + query(Order).options(joinedload(Order.items).joinedload(Item.keywords)) + + # lazily load Order.items, but when Items are loaded, + # joined-load the keywords collection + query(Order).options(lazyload(Order.items).joinedload(Item.keywords)) + + :func:`.orm.joinedload` also accepts a keyword argument `innerjoin=True` which + indicates using an inner join instead of an outer:: + + query(Order).options(joinedload(Order.user, innerjoin=True)) + + .. note:: + + The joins produced by :func:`.orm.joinedload` are **anonymously aliased**. + The criteria by which the join proceeds cannot be modified, nor can the + :class:`.Query` refer to these joins in any way, including ordering. + + To produce a specific SQL JOIN which is explicitly available, use + :class:`.Query.join`. To combine explicit JOINs with eager loading + of collections, use :func:`.orm.contains_eager`; see :ref:`contains_eager`. + + .. seealso:: + + :ref:`loading_toplevel` + + :ref:`contains_eager` + + :func:`.orm.subqueryload` + + :func:`.orm.lazyload` + + """ + loader = loadopt.set_relationship_strategy(attr, {"lazy": "joined"}) + if innerjoin is not None: + loader.local_opts['innerjoin'] = innerjoin + return loader + +@joinedload._add_unbound_fn +def joinedload(*keys, **kw): + return _UnboundLoad._from_keys( + _UnboundLoad.joinedload, keys, False, kw) + +@joinedload._add_unbound_all_fn +def joinedload_all(*keys, **kw): + return _UnboundLoad._from_keys( + _UnboundLoad.joinedload, keys, True, kw) + + +@loader_option() +def subqueryload(loadopt, attr): + """Indicate that the given attribute should be loaded using + subquery eager loading. + + This function is part of the :class:`.Load` interface and supports + both method-chained and standalone operation. + + examples:: + + # subquery-load the "orders" collection on "User" + query(User).options(subqueryload(User.orders)) + + # subquery-load Order.items and then Item.keywords + query(Order).options(subqueryload(Order.items).subqueryload(Item.keywords)) + + # lazily load Order.items, but when Items are loaded, + # subquery-load the keywords collection + query(Order).options(lazyload(Order.items).subqueryload(Item.keywords)) + + + .. seealso:: + + :ref:`loading_toplevel` + + :func:`.orm.joinedload` + + :func:`.orm.lazyload` + + """ + return loadopt.set_relationship_strategy(attr, {"lazy": "subquery"}) + +@subqueryload._add_unbound_fn +def subqueryload(*keys): + return _UnboundLoad._from_keys(_UnboundLoad.subqueryload, keys, False, {}) + +@subqueryload._add_unbound_all_fn +def subqueryload_all(*keys): + return _UnboundLoad._from_keys(_UnboundLoad.subqueryload, keys, True, {}) + +@loader_option() +def lazyload(loadopt, attr): + """Indicate that the given attribute should be loaded using "lazy" + loading. + + This function is part of the :class:`.Load` interface and supports + both method-chained and standalone operation. + + """ + return loadopt.set_relationship_strategy(attr, {"lazy": "select"}) + +@lazyload._add_unbound_fn +def lazyload(*keys): + return _UnboundLoad._from_keys(_UnboundLoad.lazyload, keys, False, {}) + +@lazyload._add_unbound_all_fn +def lazyload_all(*keys): + return _UnboundLoad._from_keys(_UnboundLoad.lazyload, keys, True, {}) + +@loader_option() +def immediateload(loadopt, attr): + """Indicate that the given attribute should be loaded using + an immediate load with a per-attribute SELECT statement. + + This function is part of the :class:`.Load` interface and supports + both method-chained and standalone operation. + + .. seealso:: + + :ref:`loading_toplevel` + + :func:`.orm.joinedload` + + :func:`.orm.lazyload` + + """ + loader = loadopt.set_relationship_strategy(attr, {"lazy": "immediate"}) + return loader + +@immediateload._add_unbound_fn +def immediateload(*keys): + return _UnboundLoad._from_keys(_UnboundLoad.immediateload, keys, False, {}) + + +@loader_option() +def noload(loadopt, attr): + """Indicate that the given relationship attribute should remain unloaded. + + This function is part of the :class:`.Load` interface and supports + both method-chained and standalone operation. + + :func:`.orm.noload` applies to :func:`.relationship` attributes; for + column-based attributes, see :func:`.orm.defer`. + + """ + + return loadopt.set_relationship_strategy(attr, {"lazy": "noload"}) + +@noload._add_unbound_fn +def noload(*keys): + return _UnboundLoad._from_keys(_UnboundLoad.noload, keys, False, {}) + +@loader_option() +def defaultload(loadopt, attr): + """Indicate an attribute should load using its default loader style. + + This method is used to link to other loader options, such as + to set the :func:`.orm.defer` option on a class that is linked to + a relationship of the parent class being loaded, :func:`.orm.defaultload` + can be used to navigate this path without changing the loading style + of the relationship:: + + session.query(MyClass).options(defaultload("someattr").defer("some_column")) + + .. seealso:: + + :func:`.orm.defer` + + :func:`.orm.undefer` + + """ + return loadopt.set_relationship_strategy( + attr, + None + ) + +@defaultload._add_unbound_fn +def defaultload(*keys): + return _UnboundLoad._from_keys(_UnboundLoad.defaultload, keys, False, {}) + +@loader_option() +def defer(loadopt, key, *addl_attrs): + """Indicate that the given column-oriented attribute should be deferred, e.g. + not loaded until accessed. + + This function is part of the :class:`.Load` interface and supports + both method-chained and standalone operation. + + e.g.:: + + from sqlalchemy.orm import defer + + session.query(MyClass).options( + defer("attribute_one"), + defer("attribute_two")) + + session.query(MyClass).options( + defer(MyClass.attribute_one), + defer(MyClass.attribute_two)) + + To specify a deferred load of an attribute on a related class, + the path can be specified one token at a time, specifying the loading + style for each link along the chain. To leave the loading style + for a link unchanged, use :func:`.orm.defaultload`:: + + session.query(MyClass).options(defaultload("someattr").defer("some_column")) + + A :class:`.Load` object that is present on a certain path can have + :meth:`.Load.defer` called multiple times, each will operate on the same + parent entity:: + + + session.query(MyClass).options( + defaultload("someattr"). + defer("some_column"). + defer("some_other_column"). + defer("another_column") + ) + + :param key: Attribute to be deferred. + + :param \*addl_attrs: Deprecated; this option supports the old 0.8 style + of specifying a path as a series of attributes, which is now superseded + by the method-chained style. + + .. seealso:: + + :ref:`deferred` + + :func:`.orm.undefer` + + """ + return loadopt.set_column_strategy( + (key, ) + addl_attrs, + {"deferred": True, "instrument": True} + ) + + +@defer._add_unbound_fn +def defer(*key): + return _UnboundLoad._from_keys(_UnboundLoad.defer, key, False, {}) + +@loader_option() +def undefer(loadopt, key, *addl_attrs): + """Indicate that the given column-oriented attribute should be undeferred, e.g. + specified within the SELECT statement of the entity as a whole. + + The column being undeferred is typically set up on the mapping as a + :func:`.deferred` attribute. + + This function is part of the :class:`.Load` interface and supports + both method-chained and standalone operation. + + Examples:: + + # undefer two columns + session.query(MyClass).options(undefer("col1"), undefer("col2")) + + # undefer all columns specific to a single class using Load + * + session.query(MyClass, MyOtherClass).options(Load(MyClass).undefer("*")) + + :param key: Attribute to be undeferred. + + :param \*addl_attrs: Deprecated; this option supports the old 0.8 style + of specifying a path as a series of attributes, which is now superseded + by the method-chained style. + + .. seealso:: + + :ref:`deferred` + + :func:`.orm.defer` + + :func:`.orm.undefer_group` + + """ + return loadopt.set_column_strategy( + (key, ) + addl_attrs, + {"deferred": False, "instrument": True} + ) + +@undefer._add_unbound_fn +def undefer(*key): + return _UnboundLoad._from_keys(_UnboundLoad.undefer, key, False, {}) + +@loader_option() +def undefer_group(loadopt, name): + """Indicate that columns within the given deferred group name should be undeferred. + + The columns being undeferred are set up on the mapping as + :func:`.deferred` attributes and include a "group" name. + + E.g:: + + session.query(MyClass).options(undefer_group("large_attrs")) + + To undefer a group of attributes on a related entity, the path can be + spelled out using relationship loader options, such as :func:`.orm.defaultload`:: + + session.query(MyClass).options(defaultload("someattr").undefer_group("large_attrs")) + + .. versionchanged:: 0.9.0 :func:`.orm.undefer_group` is now specific to a + particiular entity load path. + + .. seealso:: + + :ref:`deferred` + + :func:`.orm.defer` + + :func:`.orm.undefer` + + """ + return loadopt.set_column_strategy( + "*", + None, + {"undefer_group": name} + ) + +@undefer_group._add_unbound_fn +def undefer_group(name): + return _UnboundLoad().undefer_group(name) + diff --git a/test/orm/test_cascade.py b/test/orm/test_cascade.py index 12196b4e7..d0318b079 100644 --- a/test/orm/test_cascade.py +++ b/test/orm/test_cascade.py @@ -1715,7 +1715,7 @@ class M2MCascadeTest(fixtures.MappedTest): a1.bs.remove(b1) sess.flush() - assert atob.count().scalar() ==0 + assert atob.count().scalar() == 0 assert b.count().scalar() == 0 assert a.count().scalar() == 1 diff --git a/test/orm/test_default_strategies.py b/test/orm/test_default_strategies.py index c1668cdd4..b1175fc51 100644 --- a/test/orm/test_default_strategies.py +++ b/test/orm/test_default_strategies.py @@ -149,11 +149,13 @@ class DefaultStrategyOptionsTest(_fixtures.FixtureTest): def test_star_must_be_alone(self): sess = self._downgrade_fixture() User = self.classes.User + opt = sa.orm.subqueryload('*', User.addresses) assert_raises_message( sa.exc.ArgumentError, - "Wildcard identifier '\*' must be specified alone.", - sa.orm.subqueryload, '*', User.addresses + "Wildcard token cannot be followed by another entity", + sess.query(User).options, opt ) + def test_select_with_joinedload(self): """Mapper load strategy defaults can be downgraded with lazyload('*') option, while explicit joinedload() option @@ -283,6 +285,23 @@ class DefaultStrategyOptionsTest(_fixtures.FixtureTest): # verify everything loaded, with no additional sql needed self._assert_fully_loaded(users) + def test_joined_path_wildcards(self): + sess = self._upgrade_fixture() + users = [] + + # test upgrade all to joined: 1 sql + def go(): + users[:] = sess.query(self.classes.User)\ + .options(sa.orm.joinedload('.*'))\ + .options(sa.orm.joinedload("addresses.*"))\ + .options(sa.orm.joinedload("orders.*"))\ + .options(sa.orm.joinedload("orders.items.*"))\ + .order_by(self.classes.User.id)\ + .all() + + self.assert_sql_count(testing.db, go, 1) + self._assert_fully_loaded(users) + def test_joined_with_lazyload(self): """Mapper load strategy defaults can be upgraded with joinedload('*') option, while explicit lazyload() option @@ -350,6 +369,24 @@ class DefaultStrategyOptionsTest(_fixtures.FixtureTest): # verify everything loaded, with no additional sql needed self._assert_fully_loaded(users) + def test_subquery_path_wildcards(self): + sess = self._upgrade_fixture() + users = [] + + # test upgrade all to subquery: 1 sql + 4 relationships = 5 + def go(): + users[:] = sess.query(self.classes.User)\ + .options(sa.orm.subqueryload('.*'))\ + .options(sa.orm.subqueryload('addresses.*'))\ + .options(sa.orm.subqueryload('orders.*'))\ + .options(sa.orm.subqueryload('orders.items.*'))\ + .order_by(self.classes.User.id)\ + .all() + self.assert_sql_count(testing.db, go, 5) + + # verify everything loaded, with no additional sql needed + self._assert_fully_loaded(users) + def test_subquery_with_lazyload(self): """Mapper load strategy defaults can be upgraded with subqueryload('*') option, while explicit lazyload() option diff --git a/test/orm/test_deferred.py b/test/orm/test_deferred.py new file mode 100644 index 000000000..2dcd821dc --- /dev/null +++ b/test/orm/test_deferred.py @@ -0,0 +1,486 @@ +import sqlalchemy as sa +from sqlalchemy import testing, util +from sqlalchemy.orm import mapper, deferred, defer, undefer, Load, \ + load_only, undefer_group, create_session, synonym, relationship, Session,\ + joinedload, defaultload +from sqlalchemy.testing import eq_, AssertsCompiledSQL +from test.orm import _fixtures + + +class DeferredTest(AssertsCompiledSQL, _fixtures.FixtureTest): + + def test_basic(self): + """A basic deferred load.""" + + Order, orders = self.classes.Order, self.tables.orders + + + mapper(Order, orders, order_by=orders.c.id, properties={ + 'description': deferred(orders.c.description)}) + + o = Order() + self.assert_(o.description is None) + + q = create_session().query(Order) + def go(): + l = q.all() + o2 = l[2] + x = o2.description + + self.sql_eq_(go, [ + ("SELECT orders.id AS orders_id, " + "orders.user_id AS orders_user_id, " + "orders.address_id AS orders_address_id, " + "orders.isopen AS orders_isopen " + "FROM orders ORDER BY orders.id", {}), + ("SELECT orders.description AS orders_description " + "FROM orders WHERE orders.id = :param_1", + {'param_1':3})]) + + def test_unsaved(self): + """Deferred loading does not kick in when just PK cols are set.""" + + Order, orders = self.classes.Order, self.tables.orders + + + mapper(Order, orders, properties={ + 'description': deferred(orders.c.description)}) + + sess = create_session() + o = Order() + sess.add(o) + o.id = 7 + def go(): + o.description = "some description" + self.sql_count_(0, go) + + def test_synonym_group_bug(self): + orders, Order = self.tables.orders, self.classes.Order + + mapper(Order, orders, properties={ + 'isopen':synonym('_isopen', map_column=True), + 'description':deferred(orders.c.description, group='foo') + }) + + sess = create_session() + o1 = sess.query(Order).get(1) + eq_(o1.description, "order 1") + + def test_unsaved_2(self): + Order, orders = self.classes.Order, self.tables.orders + + mapper(Order, orders, properties={ + 'description': deferred(orders.c.description)}) + + sess = create_session() + o = Order() + sess.add(o) + def go(): + o.description = "some description" + self.sql_count_(0, go) + + def test_unsaved_group(self): + """Deferred loading doesnt kick in when just PK cols are set""" + + orders, Order = self.tables.orders, self.classes.Order + + + mapper(Order, orders, order_by=orders.c.id, properties=dict( + description=deferred(orders.c.description, group='primary'), + opened=deferred(orders.c.isopen, group='primary'))) + + sess = create_session() + o = Order() + sess.add(o) + o.id = 7 + def go(): + o.description = "some description" + self.sql_count_(0, go) + + def test_unsaved_group_2(self): + orders, Order = self.tables.orders, self.classes.Order + + mapper(Order, orders, order_by=orders.c.id, properties=dict( + description=deferred(orders.c.description, group='primary'), + opened=deferred(orders.c.isopen, group='primary'))) + + sess = create_session() + o = Order() + sess.add(o) + def go(): + o.description = "some description" + self.sql_count_(0, go) + + def test_save(self): + Order, orders = self.classes.Order, self.tables.orders + + m = mapper(Order, orders, properties={ + 'description': deferred(orders.c.description)}) + + sess = create_session() + o2 = sess.query(Order).get(2) + o2.isopen = 1 + sess.flush() + + def test_group(self): + """Deferred load with a group""" + + orders, Order = self.tables.orders, self.classes.Order + + mapper(Order, orders, properties=util.OrderedDict([ + ('userident', deferred(orders.c.user_id, group='primary')), + ('addrident', deferred(orders.c.address_id, group='primary')), + ('description', deferred(orders.c.description, group='primary')), + ('opened', deferred(orders.c.isopen, group='primary')) + ])) + + sess = create_session() + q = sess.query(Order).order_by(Order.id) + def go(): + l = q.all() + o2 = l[2] + eq_(o2.opened, 1) + eq_(o2.userident, 7) + eq_(o2.description, 'order 3') + + self.sql_eq_(go, [ + ("SELECT orders.id AS orders_id " + "FROM orders ORDER BY orders.id", {}), + ("SELECT orders.user_id AS orders_user_id, " + "orders.address_id AS orders_address_id, " + "orders.description AS orders_description, " + "orders.isopen AS orders_isopen " + "FROM orders WHERE orders.id = :param_1", + {'param_1':3})]) + + o2 = q.all()[2] + eq_(o2.description, 'order 3') + assert o2 not in sess.dirty + o2.description = 'order 3' + def go(): + sess.flush() + self.sql_count_(0, go) + + def test_preserve_changes(self): + """A deferred load operation doesn't revert modifications on attributes""" + + orders, Order = self.tables.orders, self.classes.Order + + mapper(Order, orders, properties = { + 'userident': deferred(orders.c.user_id, group='primary'), + 'description': deferred(orders.c.description, group='primary'), + 'opened': deferred(orders.c.isopen, group='primary') + }) + sess = create_session() + o = sess.query(Order).get(3) + assert 'userident' not in o.__dict__ + o.description = 'somenewdescription' + eq_(o.description, 'somenewdescription') + def go(): + eq_(o.opened, 1) + self.assert_sql_count(testing.db, go, 1) + eq_(o.description, 'somenewdescription') + assert o in sess.dirty + + def test_commits_state(self): + """ + When deferred elements are loaded via a group, they get the proper + CommittedState and don't result in changes being committed + + """ + + orders, Order = self.tables.orders, self.classes.Order + + mapper(Order, orders, properties = { + 'userident': deferred(orders.c.user_id, group='primary'), + 'description': deferred(orders.c.description, group='primary'), + 'opened': deferred(orders.c.isopen, group='primary')}) + + sess = create_session() + o2 = sess.query(Order).get(3) + + # this will load the group of attributes + eq_(o2.description, 'order 3') + assert o2 not in sess.dirty + # this will mark it as 'dirty', but nothing actually changed + o2.description = 'order 3' + # therefore the flush() shouldnt actually issue any SQL + self.assert_sql_count(testing.db, sess.flush, 0) + + def test_map_selectable_wo_deferred(self): + """test mapping to a selectable with deferred cols, + the selectable doesn't include the deferred col. + + """ + + Order, orders = self.classes.Order, self.tables.orders + + + order_select = sa.select([ + orders.c.id, + orders.c.user_id, + orders.c.address_id, + orders.c.description, + orders.c.isopen]).alias() + mapper(Order, order_select, properties={ + 'description':deferred(order_select.c.description) + }) + + sess = Session() + o1 = sess.query(Order).order_by(Order.id).first() + assert 'description' not in o1.__dict__ + eq_(o1.description, 'order 1') + + +class DeferredOptionsTest(AssertsCompiledSQL, _fixtures.FixtureTest): + __dialect__ = 'default' + + def test_options(self): + """Options on a mapper to create deferred and undeferred columns""" + + orders, Order = self.tables.orders, self.classes.Order + + + mapper(Order, orders) + + sess = create_session() + q = sess.query(Order).order_by(Order.id).options(defer('user_id')) + + def go(): + q.all()[0].user_id + + self.sql_eq_(go, [ + ("SELECT orders.id AS orders_id, " + "orders.address_id AS orders_address_id, " + "orders.description AS orders_description, " + "orders.isopen AS orders_isopen " + "FROM orders ORDER BY orders.id", {}), + ("SELECT orders.user_id AS orders_user_id " + "FROM orders WHERE orders.id = :param_1", + {'param_1':1})]) + sess.expunge_all() + + q2 = q.options(undefer('user_id')) + self.sql_eq_(q2.all, [ + ("SELECT orders.id AS orders_id, " + "orders.user_id AS orders_user_id, " + "orders.address_id AS orders_address_id, " + "orders.description AS orders_description, " + "orders.isopen AS orders_isopen " + "FROM orders ORDER BY orders.id", + {})]) + + def test_undefer_group(self): + orders, Order = self.tables.orders, self.classes.Order + + mapper(Order, orders, properties=util.OrderedDict([ + ('userident', deferred(orders.c.user_id, group='primary')), + ('description', deferred(orders.c.description, group='primary')), + ('opened', deferred(orders.c.isopen, group='primary')) + ] + )) + + sess = create_session() + q = sess.query(Order).order_by(Order.id) + def go(): + l = q.options(undefer_group('primary')).all() + o2 = l[2] + eq_(o2.opened, 1) + eq_(o2.userident, 7) + eq_(o2.description, 'order 3') + + self.sql_eq_(go, [ + ("SELECT orders.user_id AS orders_user_id, " + "orders.description AS orders_description, " + "orders.isopen AS orders_isopen, " + "orders.id AS orders_id, " + "orders.address_id AS orders_address_id " + "FROM orders ORDER BY orders.id", + {})]) + + def test_undefer_star(self): + orders, Order = self.tables.orders, self.classes.Order + + mapper(Order, orders, properties=util.OrderedDict([ + ('userident', deferred(orders.c.user_id)), + ('description', deferred(orders.c.description)), + ('opened', deferred(orders.c.isopen)) + ] + )) + + sess = create_session() + q = sess.query(Order).options(Load(Order).undefer('*')) + self.assert_compile(q, + "SELECT orders.user_id AS orders_user_id, " + "orders.description AS orders_description, " + "orders.isopen AS orders_isopen, " + "orders.id AS orders_id, " + "orders.address_id AS orders_address_id FROM orders" + ) + + def test_locates_col(self): + """Manually adding a column to the result undefers the column.""" + + orders, Order = self.tables.orders, self.classes.Order + + + mapper(Order, orders, properties={ + 'description': deferred(orders.c.description)}) + + sess = create_session() + o1 = sess.query(Order).order_by(Order.id).first() + def go(): + eq_(o1.description, 'order 1') + self.sql_count_(1, go) + + sess = create_session() + o1 = (sess.query(Order). + order_by(Order.id). + add_column(orders.c.description).first())[0] + def go(): + eq_(o1.description, 'order 1') + self.sql_count_(0, go) + + def test_deep_options(self): + users, items, order_items, Order, Item, User, orders = (self.tables.users, + self.tables.items, + self.tables.order_items, + self.classes.Order, + self.classes.Item, + self.classes.User, + self.tables.orders) + + mapper(Item, items, properties=dict( + description=deferred(items.c.description))) + mapper(Order, orders, properties=dict( + items=relationship(Item, secondary=order_items))) + mapper(User, users, properties=dict( + orders=relationship(Order, order_by=orders.c.id))) + + sess = create_session() + q = sess.query(User).order_by(User.id) + l = q.all() + item = l[0].orders[1].items[1] + def go(): + eq_(item.description, 'item 4') + self.sql_count_(1, go) + eq_(item.description, 'item 4') + + sess.expunge_all() + l = q.options(undefer('orders.items.description')).all() + item = l[0].orders[1].items[1] + def go(): + eq_(item.description, 'item 4') + self.sql_count_(0, go) + eq_(item.description, 'item 4') + + def test_chained_multi_col_options(self): + users, User = self.tables.users, self.classes.User + orders, Order = self.tables.orders, self.classes.Order + + mapper(User, users, properties={ + "orders": relationship(Order) + }) + mapper(Order, orders) + + sess = create_session() + q = sess.query(User).options( + joinedload(User.orders).defer("description").defer("isopen") + ) + self.assert_compile(q, + "SELECT users.id AS users_id, users.name AS users_name, " + "orders_1.id AS orders_1_id, orders_1.user_id AS orders_1_user_id, " + "orders_1.address_id AS orders_1_address_id FROM users " + "LEFT OUTER JOIN orders AS orders_1 ON users.id = orders_1.user_id" + ) + + def test_load_only(self): + orders, Order = self.tables.orders, self.classes.Order + + mapper(Order, orders) + + sess = create_session() + q = sess.query(Order).options(load_only("isopen", "description")) + self.assert_compile(q, + "SELECT orders.description AS orders_description, " + "orders.isopen AS orders_isopen FROM orders") + + def test_load_only_w_deferred(self): + orders, Order = self.tables.orders, self.classes.Order + + mapper(Order, orders, properties={ + "description": deferred(orders.c.description) + }) + + sess = create_session() + q = sess.query(Order).options( + load_only("isopen", "description"), + undefer("user_id") + ) + self.assert_compile(q, + "SELECT orders.description AS orders_description, " + "orders.user_id AS orders_user_id, " + "orders.isopen AS orders_isopen FROM orders") + + def test_load_only_parent_specific(self): + User = self.classes.User + Address = self.classes.Address + Order = self.classes.Order + + users = self.tables.users + addresses = self.tables.addresses + orders = self.tables.orders + + mapper(User, users) + mapper(Address, addresses) + mapper(Order, orders) + + sess = create_session() + q = sess.query(User, Order, Address).options( + Load(User).load_only("name"), + Load(Order).load_only("id"), + Load(Address).load_only("id", "email_address") + ) + + self.assert_compile(q, + "SELECT users.name AS users_name, orders.id AS orders_id, " + "addresses.id AS addresses_id, addresses.email_address " + "AS addresses_email_address FROM users, orders, addresses" + ) + + def test_load_only_path_specific(self): + User = self.classes.User + Address = self.classes.Address + Order = self.classes.Order + + users = self.tables.users + addresses = self.tables.addresses + orders = self.tables.orders + + mapper(User, users, properties={ + "addresses": relationship(Address, lazy="joined"), + "orders": relationship(Order, lazy="joined") + }) + mapper(Address, addresses) + mapper(Order, orders) + + sess = create_session() + + q = sess.query(User).options( + load_only("name").defaultload("addresses").load_only("id", "email_address"), + defaultload("orders").load_only("id") + ) + + # hmmmm joinedload seems to be forcing users.id into here... + self.assert_compile( + q, + "SELECT users.name AS users_name, users.id AS users_id, " + "addresses_1.id AS addresses_1_id, " + "addresses_1.email_address AS addresses_1_email_address, " + "orders_1.id AS orders_1_id FROM users " + "LEFT OUTER JOIN addresses AS addresses_1 " + "ON users.id = addresses_1.user_id " + "LEFT OUTER JOIN orders AS orders_1 ON users.id = orders_1.user_id" + ) + + diff --git a/test/orm/test_eager_relations.py b/test/orm/test_eager_relations.py index e53ff6669..ee671d04f 100644 --- a/test/orm/test_eager_relations.py +++ b/test/orm/test_eager_relations.py @@ -2166,7 +2166,8 @@ class MixedSelfReferentialEagerTest(fixtures.MappedTest): options( joinedload('parent_b1'), joinedload('parent_b2'), - joinedload('parent_z')). + joinedload('parent_z') + ). filter(B.id.in_([2, 8, 11])).order_by(B.id).all(), [ B(id=2, parent_z=A(id=1), parent_b1=B(id=1), parent_b2=None), @@ -2804,7 +2805,7 @@ class CyclicalInheritingEagerTestThree(fixtures.DeclarativeMappedTest, Director = self.classes.Director sess = create_session() self.assert_compile( - sess.query(PersistentObject).options(joinedload(Director.other, join_depth=1)), + sess.query(PersistentObject).options(joinedload(Director.other)), "SELECT persistent.id AS persistent_id, director.id AS director_id, " "director.other_id AS director_other_id, " "director.name AS director_name, persistent_1.id AS " diff --git a/test/orm/test_froms.py b/test/orm/test_froms.py index 2403f4aae..8dc06a630 100644 --- a/test/orm/test_froms.py +++ b/test/orm/test_froms.py @@ -675,19 +675,18 @@ class AddEntityEquivalenceTest(fixtures.MappedTest, AssertsCompiledSQL): class InstancesTest(QueryTest, AssertsCompiledSQL): - def test_from_alias(self): + def test_from_alias_one(self): User, addresses, users = (self.classes.User, self.tables.addresses, self.tables.users) - - query = users.select(users.c.id==7).\ - union(users.select(users.c.id>7)).\ + query = users.select(users.c.id == 7).\ + union(users.select(users.c.id > 7)).\ alias('ulist').\ outerjoin(addresses).\ select(use_labels=True, order_by=['ulist.id', addresses.c.id]) - sess =create_session() + sess = create_session() q = sess.query(User) def go(): @@ -697,7 +696,19 @@ class InstancesTest(QueryTest, AssertsCompiledSQL): assert self.static.user_address_result == l self.assert_sql_count(testing.db, go, 1) - sess.expunge_all() + def test_from_alias_two(self): + User, addresses, users = (self.classes.User, + self.tables.addresses, + self.tables.users) + + query = users.select(users.c.id == 7).\ + union(users.select(users.c.id > 7)).\ + alias('ulist').\ + outerjoin(addresses).\ + select(use_labels=True, + order_by=['ulist.id', addresses.c.id]) + sess = create_session() + q = sess.query(User) def go(): l = q.options(contains_alias('ulist'), @@ -706,6 +717,19 @@ class InstancesTest(QueryTest, AssertsCompiledSQL): assert self.static.user_address_result == l self.assert_sql_count(testing.db, go, 1) + def test_from_alias_three(self): + User, addresses, users = (self.classes.User, + self.tables.addresses, + self.tables.users) + + query = users.select(users.c.id == 7).\ + union(users.select(users.c.id > 7)).\ + alias('ulist').\ + outerjoin(addresses).\ + select(use_labels=True, + order_by=['ulist.id', addresses.c.id]) + sess = create_session() + # better way. use select_entity_from() def go(): l = sess.query(User).select_entity_from(query).\ @@ -713,12 +737,19 @@ class InstancesTest(QueryTest, AssertsCompiledSQL): assert self.static.user_address_result == l self.assert_sql_count(testing.db, go, 1) + def test_from_alias_four(self): + User, addresses, users = (self.classes.User, + self.tables.addresses, + self.tables.users) + + sess = create_session() + # same thing, but alias addresses, so that the adapter # generated by select_entity_from() is wrapped within # the adapter created by contains_eager() adalias = addresses.alias() - query = users.select(users.c.id==7).\ - union(users.select(users.c.id>7)).\ + query = users.select(users.c.id == 7).\ + union(users.select(users.c.id > 7)).\ alias('ulist').\ outerjoin(adalias).\ select(use_labels=True, @@ -902,6 +933,11 @@ class InstancesTest(QueryTest, AssertsCompiledSQL): order_by(users.c.id, oalias.c.id, ialias.c.id) # test using Alias with more than one level deep + + # new way: + #from sqlalchemy.orm.strategy_options import Load + #opt = Load(User).contains_eager('orders', alias=oalias).contains_eager('items', alias=ialias) + def go(): l = list(q.options( contains_eager('orders', alias=oalias), diff --git a/test/orm/test_mapper.py b/test/orm/test_mapper.py index e073093fa..5255e4fe2 100644 --- a/test/orm/test_mapper.py +++ b/test/orm/test_mapper.py @@ -1707,7 +1707,6 @@ class ORMLoggingTest(_fixtures.FixtureTest): class OptionsTest(_fixtures.FixtureTest): - @testing.fails_on('maxdb', 'FIXME: unknown') def test_synonym_options(self): Address, addresses, users, User = (self.classes.Address, self.tables.addresses, @@ -1925,12 +1924,11 @@ class OptionsTest(_fixtures.FixtureTest): oalias = aliased(Order) opt1 = sa.orm.joinedload(User.orders, Order.items) - opt2a, opt2b = sa.orm.contains_eager(User.orders, Order.items, alias=oalias) - u1 = sess.query(User).join(oalias, User.orders).options(opt1, opt2a, opt2b).first() + opt2 = sa.orm.contains_eager(User.orders, Order.items, alias=oalias) + u1 = sess.query(User).join(oalias, User.orders).options(opt1, opt2).first() ustate = attributes.instance_state(u1) assert opt1 in ustate.load_options - assert opt2a not in ustate.load_options - assert opt2b not in ustate.load_options + assert opt2 not in ustate.load_options class DeepOptionsTest(_fixtures.FixtureTest): @@ -2286,349 +2284,6 @@ class ComparatorFactoryTest(_fixtures.FixtureTest, AssertsCompiledSQL): dialect=default.DefaultDialect()) -class DeferredTest(_fixtures.FixtureTest): - - def test_basic(self): - """A basic deferred load.""" - - Order, orders = self.classes.Order, self.tables.orders - - - mapper(Order, orders, order_by=orders.c.id, properties={ - 'description': deferred(orders.c.description)}) - - o = Order() - self.assert_(o.description is None) - - q = create_session().query(Order) - def go(): - l = q.all() - o2 = l[2] - x = o2.description - - self.sql_eq_(go, [ - ("SELECT orders.id AS orders_id, " - "orders.user_id AS orders_user_id, " - "orders.address_id AS orders_address_id, " - "orders.isopen AS orders_isopen " - "FROM orders ORDER BY orders.id", {}), - ("SELECT orders.description AS orders_description " - "FROM orders WHERE orders.id = :param_1", - {'param_1':3})]) - - def test_unsaved(self): - """Deferred loading does not kick in when just PK cols are set.""" - - Order, orders = self.classes.Order, self.tables.orders - - - mapper(Order, orders, properties={ - 'description': deferred(orders.c.description)}) - - sess = create_session() - o = Order() - sess.add(o) - o.id = 7 - def go(): - o.description = "some description" - self.sql_count_(0, go) - - def test_synonym_group_bug(self): - orders, Order = self.tables.orders, self.classes.Order - - mapper(Order, orders, properties={ - 'isopen':synonym('_isopen', map_column=True), - 'description':deferred(orders.c.description, group='foo') - }) - - sess = create_session() - o1 = sess.query(Order).get(1) - eq_(o1.description, "order 1") - - def test_unsaved_2(self): - Order, orders = self.classes.Order, self.tables.orders - - mapper(Order, orders, properties={ - 'description': deferred(orders.c.description)}) - - sess = create_session() - o = Order() - sess.add(o) - def go(): - o.description = "some description" - self.sql_count_(0, go) - - def test_unsaved_group(self): - """Deferred loading doesnt kick in when just PK cols are set""" - - orders, Order = self.tables.orders, self.classes.Order - - - mapper(Order, orders, order_by=orders.c.id, properties=dict( - description=deferred(orders.c.description, group='primary'), - opened=deferred(orders.c.isopen, group='primary'))) - - sess = create_session() - o = Order() - sess.add(o) - o.id = 7 - def go(): - o.description = "some description" - self.sql_count_(0, go) - - def test_unsaved_group_2(self): - orders, Order = self.tables.orders, self.classes.Order - - mapper(Order, orders, order_by=orders.c.id, properties=dict( - description=deferred(orders.c.description, group='primary'), - opened=deferred(orders.c.isopen, group='primary'))) - - sess = create_session() - o = Order() - sess.add(o) - def go(): - o.description = "some description" - self.sql_count_(0, go) - - def test_save(self): - Order, orders = self.classes.Order, self.tables.orders - - m = mapper(Order, orders, properties={ - 'description': deferred(orders.c.description)}) - - sess = create_session() - o2 = sess.query(Order).get(2) - o2.isopen = 1 - sess.flush() - - def test_group(self): - """Deferred load with a group""" - - orders, Order = self.tables.orders, self.classes.Order - - mapper(Order, orders, properties=util.OrderedDict([ - ('userident', deferred(orders.c.user_id, group='primary')), - ('addrident', deferred(orders.c.address_id, group='primary')), - ('description', deferred(orders.c.description, group='primary')), - ('opened', deferred(orders.c.isopen, group='primary')) - ])) - - sess = create_session() - q = sess.query(Order).order_by(Order.id) - def go(): - l = q.all() - o2 = l[2] - eq_(o2.opened, 1) - eq_(o2.userident, 7) - eq_(o2.description, 'order 3') - - self.sql_eq_(go, [ - ("SELECT orders.id AS orders_id " - "FROM orders ORDER BY orders.id", {}), - ("SELECT orders.user_id AS orders_user_id, " - "orders.address_id AS orders_address_id, " - "orders.description AS orders_description, " - "orders.isopen AS orders_isopen " - "FROM orders WHERE orders.id = :param_1", - {'param_1':3})]) - - o2 = q.all()[2] - eq_(o2.description, 'order 3') - assert o2 not in sess.dirty - o2.description = 'order 3' - def go(): - sess.flush() - self.sql_count_(0, go) - - def test_preserve_changes(self): - """A deferred load operation doesn't revert modifications on attributes""" - - orders, Order = self.tables.orders, self.classes.Order - - mapper(Order, orders, properties = { - 'userident': deferred(orders.c.user_id, group='primary'), - 'description': deferred(orders.c.description, group='primary'), - 'opened': deferred(orders.c.isopen, group='primary') - }) - sess = create_session() - o = sess.query(Order).get(3) - assert 'userident' not in o.__dict__ - o.description = 'somenewdescription' - eq_(o.description, 'somenewdescription') - def go(): - eq_(o.opened, 1) - self.assert_sql_count(testing.db, go, 1) - eq_(o.description, 'somenewdescription') - assert o in sess.dirty - - def test_commits_state(self): - """ - When deferred elements are loaded via a group, they get the proper - CommittedState and don't result in changes being committed - - """ - - orders, Order = self.tables.orders, self.classes.Order - - mapper(Order, orders, properties = { - 'userident':deferred(orders.c.user_id, group='primary'), - 'description':deferred(orders.c.description, group='primary'), - 'opened':deferred(orders.c.isopen, group='primary')}) - - sess = create_session() - o2 = sess.query(Order).get(3) - - # this will load the group of attributes - eq_(o2.description, 'order 3') - assert o2 not in sess.dirty - # this will mark it as 'dirty', but nothing actually changed - o2.description = 'order 3' - # therefore the flush() shouldnt actually issue any SQL - self.assert_sql_count(testing.db, sess.flush, 0) - - def test_options(self): - """Options on a mapper to create deferred and undeferred columns""" - - orders, Order = self.tables.orders, self.classes.Order - - - mapper(Order, orders) - - sess = create_session() - q = sess.query(Order).order_by(Order.id).options(defer('user_id')) - - def go(): - q.all()[0].user_id - - self.sql_eq_(go, [ - ("SELECT orders.id AS orders_id, " - "orders.address_id AS orders_address_id, " - "orders.description AS orders_description, " - "orders.isopen AS orders_isopen " - "FROM orders ORDER BY orders.id", {}), - ("SELECT orders.user_id AS orders_user_id " - "FROM orders WHERE orders.id = :param_1", - {'param_1':1})]) - sess.expunge_all() - - q2 = q.options(sa.orm.undefer('user_id')) - self.sql_eq_(q2.all, [ - ("SELECT orders.id AS orders_id, " - "orders.user_id AS orders_user_id, " - "orders.address_id AS orders_address_id, " - "orders.description AS orders_description, " - "orders.isopen AS orders_isopen " - "FROM orders ORDER BY orders.id", - {})]) - - def test_undefer_group(self): - orders, Order = self.tables.orders, self.classes.Order - - mapper(Order, orders, properties=util.OrderedDict([ - ('userident',deferred(orders.c.user_id, group='primary')), - ('description',deferred(orders.c.description, group='primary')), - ('opened',deferred(orders.c.isopen, group='primary')) - ] - )) - - sess = create_session() - q = sess.query(Order).order_by(Order.id) - def go(): - l = q.options(sa.orm.undefer_group('primary')).all() - o2 = l[2] - eq_(o2.opened, 1) - eq_(o2.userident, 7) - eq_(o2.description, 'order 3') - - self.sql_eq_(go, [ - ("SELECT orders.user_id AS orders_user_id, " - "orders.description AS orders_description, " - "orders.isopen AS orders_isopen, " - "orders.id AS orders_id, " - "orders.address_id AS orders_address_id " - "FROM orders ORDER BY orders.id", - {})]) - - def test_locates_col(self): - """Manually adding a column to the result undefers the column.""" - - orders, Order = self.tables.orders, self.classes.Order - - - mapper(Order, orders, properties={ - 'description':deferred(orders.c.description)}) - - sess = create_session() - o1 = sess.query(Order).order_by(Order.id).first() - def go(): - eq_(o1.description, 'order 1') - self.sql_count_(1, go) - - sess = create_session() - o1 = (sess.query(Order). - order_by(Order.id). - add_column(orders.c.description).first())[0] - def go(): - eq_(o1.description, 'order 1') - self.sql_count_(0, go) - - def test_map_selectable_wo_deferred(self): - """test mapping to a selectable with deferred cols, - the selectable doesn't include the deferred col. - - """ - - Order, orders = self.classes.Order, self.tables.orders - - - order_select = sa.select([ - orders.c.id, - orders.c.user_id, - orders.c.address_id, - orders.c.description, - orders.c.isopen]).alias() - mapper(Order, order_select, properties={ - 'description':deferred(order_select.c.description) - }) - - sess = Session() - o1 = sess.query(Order).order_by(Order.id).first() - assert 'description' not in o1.__dict__ - eq_(o1.description, 'order 1') - - def test_deep_options(self): - users, items, order_items, Order, Item, User, orders = (self.tables.users, - self.tables.items, - self.tables.order_items, - self.classes.Order, - self.classes.Item, - self.classes.User, - self.tables.orders) - - mapper(Item, items, properties=dict( - description=deferred(items.c.description))) - mapper(Order, orders, properties=dict( - items=relationship(Item, secondary=order_items))) - mapper(User, users, properties=dict( - orders=relationship(Order, order_by=orders.c.id))) - - sess = create_session() - q = sess.query(User).order_by(User.id) - l = q.all() - item = l[0].orders[1].items[1] - def go(): - eq_(item.description, 'item 4') - self.sql_count_(1, go) - eq_(item.description, 'item 4') - - sess.expunge_all() - l = q.options(sa.orm.undefer('orders.items.description')).all() - item = l[0].orders[1].items[1] - def go(): - eq_(item.description, 'item 4') - self.sql_count_(0, go) - eq_(item.description, 'item 4') - class SecondaryOptionsTest(fixtures.MappedTest): """test that the contains_eager() option doesn't bleed into a secondary load.""" diff --git a/test/orm/test_options.py b/test/orm/test_options.py new file mode 100644 index 000000000..29c2c698e --- /dev/null +++ b/test/orm/test_options.py @@ -0,0 +1,760 @@ +from sqlalchemy import inspect +from sqlalchemy.orm import attributes, mapper, relationship, backref, \ + configure_mappers, create_session, synonym, Session, class_mapper, \ + aliased, column_property, joinedload_all, joinedload, Query,\ + util as orm_util, Load +import sqlalchemy as sa +from sqlalchemy import testing +from sqlalchemy.testing.assertions import eq_, assert_raises, assert_raises_message +from test.orm import _fixtures + +class QueryTest(_fixtures.FixtureTest): + run_setup_mappers = 'once' + run_inserts = 'once' + run_deletes = None + + @classmethod + def setup_mappers(cls): + cls._setup_stock_mapping() + +class PathTest(object): + def _make_path(self, path): + r = [] + for i, item in enumerate(path): + if i % 2 == 0: + if isinstance(item, type): + item = class_mapper(item) + else: + if isinstance(item, str): + item = inspect(r[-1]).mapper.attrs[item] + r.append(item) + return tuple(r) + + def _make_path_registry(self, path): + return orm_util.PathRegistry.coerce(self._make_path(path)) + + def _assert_path_result(self, opt, q, paths): + q._attributes = q._attributes.copy() + attr = {} + + for val in opt._to_bind: + val._bind_loader(q, attr, False) + + assert_paths = [k[1] for k in attr] + eq_( + set([p for p in assert_paths]), + set([self._make_path(p) for p in paths]) + ) + +class LoadTest(PathTest, QueryTest): + + def test_gen_path_attr_entity(self): + User = self.classes.User + Address = self.classes.Address + + l = Load(User) + eq_( + l._generate_path(inspect(User)._path_registry, User.addresses, "relationship"), + self._make_path_registry([User, "addresses", Address]) + ) + + def test_gen_path_attr_column(self): + User = self.classes.User + + l = Load(User) + eq_( + l._generate_path(inspect(User)._path_registry, User.name, "column"), + self._make_path_registry([User, "name"]) + ) + + def test_gen_path_string_entity(self): + User = self.classes.User + Address = self.classes.Address + + l = Load(User) + eq_( + l._generate_path(inspect(User)._path_registry, "addresses", "relationship"), + self._make_path_registry([User, "addresses", Address]) + ) + + def test_gen_path_string_column(self): + User = self.classes.User + + l = Load(User) + eq_( + l._generate_path(inspect(User)._path_registry, "name", "column"), + self._make_path_registry([User, "name"]) + ) + + def test_gen_path_invalid_from_col(self): + User = self.classes.User + + l = Load(User) + l.path = self._make_path_registry([User, "name"]) + assert_raises_message( + sa.exc.ArgumentError, + "Attribute 'name' of entity 'Mapper|User|users' does " + "not refer to a mapped entity", + l._generate_path, l.path, User.addresses, "relationship" + + ) + def test_gen_path_attr_entity_invalid_raiseerr(self): + User = self.classes.User + Order = self.classes.Order + + l = Load(User) + + assert_raises_message( + sa.exc.ArgumentError, + "Attribute 'Order.items' does not link from element 'Mapper|User|users'", + l._generate_path, + inspect(User)._path_registry, Order.items, "relationship", + ) + + def test_gen_path_attr_entity_invalid_noraiseerr(self): + User = self.classes.User + Order = self.classes.Order + + l = Load(User) + + eq_( + l._generate_path( + inspect(User)._path_registry, Order.items, "relationship", False + ), + None + ) + + def test_set_strat_ent(self): + User = self.classes.User + + l1 = Load(User) + l2 = l1.joinedload("addresses") + eq_( + l1.context, + { + ('loader', self._make_path([User, "addresses"])): l2 + } + ) + + def test_set_strat_col(self): + User = self.classes.User + + l1 = Load(User) + l2 = l1.defer("name") + l3 = l2.context.values()[0] + eq_( + l1.context, + { + ('loader', self._make_path([User, "name"])): l3 + } + ) + + +class OptionsTest(PathTest, QueryTest): + + def _option_fixture(self, *arg): + from sqlalchemy.orm import strategy_options + + return strategy_options._UnboundLoad._from_keys( + strategy_options._UnboundLoad.joinedload, arg, True, {}) + + + + def test_get_path_one_level_string(self): + User = self.classes.User + + sess = Session() + q = sess.query(User) + + opt = self._option_fixture("addresses") + self._assert_path_result(opt, q, [(User, 'addresses')]) + + def test_get_path_one_level_attribute(self): + User = self.classes.User + + sess = Session() + q = sess.query(User) + + opt = self._option_fixture(User.addresses) + self._assert_path_result(opt, q, [(User, 'addresses')]) + + def test_path_on_entity_but_doesnt_match_currentpath(self): + User, Address = self.classes.User, self.classes.Address + + # ensure "current path" is fully consumed before + # matching against current entities. + # see [ticket:2098] + sess = Session() + q = sess.query(User) + opt = self._option_fixture('email_address', 'id') + q = sess.query(Address)._with_current_path( + orm_util.PathRegistry.coerce([inspect(User), + inspect(User).attrs.addresses]) + ) + self._assert_path_result(opt, q, []) + + def test_get_path_one_level_with_unrelated(self): + Order = self.classes.Order + + sess = Session() + q = sess.query(Order) + opt = self._option_fixture("addresses") + self._assert_path_result(opt, q, []) + + def test_path_multilevel_string(self): + Item, User, Order = (self.classes.Item, + self.classes.User, + self.classes.Order) + + sess = Session() + q = sess.query(User) + + opt = self._option_fixture("orders.items.keywords") + self._assert_path_result(opt, q, [ + (User, 'orders'), + (User, 'orders', Order, 'items'), + (User, 'orders', Order, 'items', Item, 'keywords') + ]) + + def test_path_multilevel_attribute(self): + Item, User, Order = (self.classes.Item, + self.classes.User, + self.classes.Order) + + sess = Session() + q = sess.query(User) + + opt = self._option_fixture(User.orders, Order.items, Item.keywords) + self._assert_path_result(opt, q, [ + (User, 'orders'), + (User, 'orders', Order, 'items'), + (User, 'orders', Order, 'items', Item, 'keywords') + ]) + + def test_with_current_matching_string(self): + Item, User, Order = (self.classes.Item, + self.classes.User, + self.classes.Order) + + sess = Session() + q = sess.query(Item)._with_current_path( + self._make_path_registry([User, 'orders', Order, 'items']) + ) + + opt = self._option_fixture("orders.items.keywords") + self._assert_path_result(opt, q, [ + (Item, 'keywords') + ]) + + def test_with_current_matching_attribute(self): + Item, User, Order = (self.classes.Item, + self.classes.User, + self.classes.Order) + + sess = Session() + q = sess.query(Item)._with_current_path( + self._make_path_registry([User, 'orders', Order, 'items']) + ) + + opt = self._option_fixture(User.orders, Order.items, Item.keywords) + self._assert_path_result(opt, q, [ + (Item, 'keywords') + ]) + + def test_with_current_nonmatching_string(self): + Item, User, Order = (self.classes.Item, + self.classes.User, + self.classes.Order) + + sess = Session() + q = sess.query(Item)._with_current_path( + self._make_path_registry([User, 'orders', Order, 'items']) + ) + + opt = self._option_fixture("keywords") + self._assert_path_result(opt, q, []) + + opt = self._option_fixture("items.keywords") + self._assert_path_result(opt, q, []) + + def test_with_current_nonmatching_attribute(self): + Item, User, Order = (self.classes.Item, + self.classes.User, + self.classes.Order) + + sess = Session() + q = sess.query(Item)._with_current_path( + self._make_path_registry([User, 'orders', Order, 'items']) + ) + + opt = self._option_fixture(Item.keywords) + self._assert_path_result(opt, q, []) + + opt = self._option_fixture(Order.items, Item.keywords) + self._assert_path_result(opt, q, []) + + def test_from_base_to_subclass_attr(self): + Dingaling, Address = self.classes.Dingaling, self.classes.Address + + sess = Session() + class SubAddr(Address): + pass + mapper(SubAddr, inherits=Address, properties={ + 'flub': relationship(Dingaling) + }) + + q = sess.query(Address) + opt = self._option_fixture(SubAddr.flub) + + self._assert_path_result(opt, q, [(SubAddr, 'flub')]) + + def test_from_subclass_to_subclass_attr(self): + Dingaling, Address = self.classes.Dingaling, self.classes.Address + + sess = Session() + class SubAddr(Address): + pass + mapper(SubAddr, inherits=Address, properties={ + 'flub': relationship(Dingaling) + }) + + q = sess.query(SubAddr) + opt = self._option_fixture(SubAddr.flub) + + self._assert_path_result(opt, q, [(SubAddr, 'flub')]) + + def test_from_base_to_base_attr_via_subclass(self): + Dingaling, Address = self.classes.Dingaling, self.classes.Address + + sess = Session() + class SubAddr(Address): + pass + mapper(SubAddr, inherits=Address, properties={ + 'flub': relationship(Dingaling) + }) + + q = sess.query(Address) + opt = self._option_fixture(SubAddr.user) + + self._assert_path_result(opt, q, + [(Address, inspect(Address).attrs.user)]) + + def test_of_type(self): + User, Address = self.classes.User, self.classes.Address + + sess = Session() + class SubAddr(Address): + pass + mapper(SubAddr, inherits=Address) + + q = sess.query(User) + opt = self._option_fixture(User.addresses.of_type(SubAddr), SubAddr.user) + + u_mapper = inspect(User) + a_mapper = inspect(Address) + self._assert_path_result(opt, q, [ + (u_mapper, u_mapper.attrs.addresses), + (u_mapper, u_mapper.attrs.addresses, a_mapper, a_mapper.attrs.user) + ]) + + def test_of_type_plus_level(self): + Dingaling, User, Address = (self.classes.Dingaling, + self.classes.User, + self.classes.Address) + + sess = Session() + class SubAddr(Address): + pass + mapper(SubAddr, inherits=Address, properties={ + 'flub': relationship(Dingaling) + }) + + q = sess.query(User) + opt = self._option_fixture(User.addresses.of_type(SubAddr), SubAddr.flub) + + u_mapper = inspect(User) + sa_mapper = inspect(SubAddr) + self._assert_path_result(opt, q, [ + (u_mapper, u_mapper.attrs.addresses), + (u_mapper, u_mapper.attrs.addresses, sa_mapper, sa_mapper.attrs.flub) + ]) + + def test_aliased_single(self): + User = self.classes.User + + sess = Session() + ualias = aliased(User) + q = sess.query(ualias) + opt = self._option_fixture(ualias.addresses) + self._assert_path_result(opt, q, [(inspect(ualias), 'addresses')]) + + def test_with_current_aliased_single(self): + User, Address = self.classes.User, self.classes.Address + + sess = Session() + ualias = aliased(User) + q = sess.query(ualias)._with_current_path( + self._make_path_registry([Address, 'user']) + ) + opt = self._option_fixture(Address.user, ualias.addresses) + self._assert_path_result(opt, q, [(inspect(ualias), 'addresses')]) + + def test_with_current_aliased_single_nonmatching_option(self): + User, Address = self.classes.User, self.classes.Address + + sess = Session() + ualias = aliased(User) + q = sess.query(User)._with_current_path( + self._make_path_registry([Address, 'user']) + ) + opt = self._option_fixture(Address.user, ualias.addresses) + self._assert_path_result(opt, q, []) + + def test_with_current_aliased_single_nonmatching_entity(self): + User, Address = self.classes.User, self.classes.Address + + sess = Session() + ualias = aliased(User) + q = sess.query(ualias)._with_current_path( + self._make_path_registry([Address, 'user']) + ) + opt = self._option_fixture(Address.user, User.addresses) + self._assert_path_result(opt, q, []) + + def test_multi_entity_opt_on_second(self): + Item = self.classes.Item + Order = self.classes.Order + opt = self._option_fixture(Order.items) + sess = Session() + q = sess.query(Item, Order) + self._assert_path_result(opt, q, [(Order, "items")]) + + def test_multi_entity_opt_on_string(self): + Item = self.classes.Item + Order = self.classes.Order + opt = self._option_fixture("items") + sess = Session() + q = sess.query(Item, Order) + self._assert_path_result(opt, q, []) + + def test_multi_entity_no_mapped_entities(self): + Item = self.classes.Item + Order = self.classes.Order + opt = self._option_fixture("items") + sess = Session() + q = sess.query(Item.id, Order.id) + self._assert_path_result(opt, q, []) + + def test_path_exhausted(self): + User = self.classes.User + Item = self.classes.Item + Order = self.classes.Order + opt = self._option_fixture(User.orders) + sess = Session() + q = sess.query(Item)._with_current_path( + self._make_path_registry([User, 'orders', Order, 'items']) + ) + self._assert_path_result(opt, q, []) + + def test_chained(self): + User = self.classes.User + Order = self.classes.Order + Item = self.classes.Item + sess = Session() + q = sess.query(User) + opt = self._option_fixture(User.orders).joinedload("items") + self._assert_path_result(opt, q, [ + (User, 'orders'), + (User, 'orders', Order, "items") + ]) + + def test_chained_plus_dotted(self): + User = self.classes.User + Order = self.classes.Order + Item = self.classes.Item + sess = Session() + q = sess.query(User) + opt = self._option_fixture("orders.items").joinedload("keywords") + self._assert_path_result(opt, q, [ + (User, 'orders'), + (User, 'orders', Order, "items"), + (User, 'orders', Order, "items", Item, "keywords") + ]) + + def test_chained_plus_multi(self): + User = self.classes.User + Order = self.classes.Order + Item = self.classes.Item + sess = Session() + q = sess.query(User) + opt = self._option_fixture(User.orders, Order.items).joinedload("keywords") + self._assert_path_result(opt, q, [ + (User, 'orders'), + (User, 'orders', Order, "items"), + (User, 'orders', Order, "items", Item, "keywords") + ]) + + +class OptionsNoPropTest(_fixtures.FixtureTest): + """test the error messages emitted when using property + options in conjunection with column-only entities, or + for not existing options + + """ + + run_create_tables = False + run_inserts = None + run_deletes = None + + def test_option_with_mapper_basestring(self): + Item = self.classes.Item + + self._assert_option([Item], 'keywords') + + def test_option_with_mapper_PropCompatator(self): + Item = self.classes.Item + + self._assert_option([Item], Item.keywords) + + def test_option_with_mapper_then_column_basestring(self): + Item = self.classes.Item + + self._assert_option([Item, Item.id], 'keywords') + + def test_option_with_mapper_then_column_PropComparator(self): + Item = self.classes.Item + + self._assert_option([Item, Item.id], Item.keywords) + + def test_option_with_column_then_mapper_basestring(self): + Item = self.classes.Item + + self._assert_option([Item.id, Item], 'keywords') + + def test_option_with_column_then_mapper_PropComparator(self): + Item = self.classes.Item + + self._assert_option([Item.id, Item], Item.keywords) + + def test_option_with_column_basestring(self): + Item = self.classes.Item + + message = \ + "Query has only expression-based entities - "\ + "can't find property named 'keywords'." + self._assert_eager_with_just_column_exception(Item.id, + 'keywords', message) + + def test_option_with_column_PropComparator(self): + Item = self.classes.Item + + self._assert_eager_with_just_column_exception(Item.id, + Item.keywords, + "Query has only expression-based entities " + "- can't find property named 'keywords'." + ) + + def test_option_against_nonexistent_PropComparator(self): + Item = self.classes.Item + Keyword = self.classes.Keyword + self._assert_eager_with_entity_exception( + [Keyword], + (joinedload(Item.keywords), ), + r"Can't find property 'keywords' on any entity specified " + r"in this Query. Note the full path from root " + r"\(Mapper\|Keyword\|keywords\) to target entity must be specified." + ) + + def test_option_against_nonexistent_basestring(self): + Item = self.classes.Item + self._assert_eager_with_entity_exception( + [Item], + (joinedload("foo"), ), + r"Can't find property named 'foo' on the mapped " + r"entity Mapper\|Item\|items in this Query." + ) + + def test_option_against_nonexistent_twolevel_basestring(self): + Item = self.classes.Item + self._assert_eager_with_entity_exception( + [Item], + (joinedload("keywords.foo"), ), + r"Can't find property named 'foo' on the mapped entity " + r"Mapper\|Keyword\|keywords in this Query." + ) + + def test_option_against_nonexistent_twolevel_all(self): + Item = self.classes.Item + self._assert_eager_with_entity_exception( + [Item], + (joinedload_all("keywords.foo"), ), + r"Can't find property named 'foo' on the mapped entity " + r"Mapper\|Keyword\|keywords in this Query." + ) + + @testing.fails_if(lambda: True, + "PropertyOption doesn't yet check for relation/column on end result") + def test_option_against_non_relation_basestring(self): + Item = self.classes.Item + Keyword = self.classes.Keyword + self._assert_eager_with_entity_exception( + [Keyword, Item], + (joinedload_all("keywords"), ), + r"Attribute 'keywords' of entity 'Mapper\|Keyword\|keywords' " + "does not refer to a mapped entity" + ) + + @testing.fails_if(lambda: True, + "PropertyOption doesn't yet check for relation/column on end result") + def test_option_against_multi_non_relation_basestring(self): + Item = self.classes.Item + Keyword = self.classes.Keyword + self._assert_eager_with_entity_exception( + [Keyword, Item], + (joinedload_all("keywords"), ), + r"Attribute 'keywords' of entity 'Mapper\|Keyword\|keywords' " + "does not refer to a mapped entity" + ) + + def test_option_against_wrong_entity_type_basestring(self): + Item = self.classes.Item + self._assert_eager_with_entity_exception( + [Item], + (joinedload_all("id", "keywords"), ), + r"Attribute 'id' of entity 'Mapper\|Item\|items' does not " + r"refer to a mapped entity" + ) + + def test_option_against_multi_non_relation_twolevel_basestring(self): + Item = self.classes.Item + Keyword = self.classes.Keyword + self._assert_eager_with_entity_exception( + [Keyword, Item], + (joinedload_all("id", "keywords"), ), + r"Attribute 'id' of entity 'Mapper\|Keyword\|keywords' " + "does not refer to a mapped entity" + ) + + def test_option_against_multi_nonexistent_basestring(self): + Item = self.classes.Item + Keyword = self.classes.Keyword + self._assert_eager_with_entity_exception( + [Keyword, Item], + (joinedload_all("description"), ), + r"Can't find property named 'description' on the mapped " + r"entity Mapper\|Keyword\|keywords in this Query." + ) + + def test_option_against_multi_no_entities_basestring(self): + Item = self.classes.Item + Keyword = self.classes.Keyword + self._assert_eager_with_entity_exception( + [Keyword.id, Item.id], + (joinedload_all("keywords"), ), + r"Query has only expression-based entities - can't find property " + "named 'keywords'." + ) + + def test_option_against_wrong_multi_entity_type_attr_one(self): + Item = self.classes.Item + Keyword = self.classes.Keyword + self._assert_eager_with_entity_exception( + [Keyword, Item], + (joinedload_all(Keyword.id, Item.keywords), ), + r"Attribute 'id' of entity 'Mapper\|Keyword\|keywords' " + "does not refer to a mapped entity" + ) + + def test_option_against_wrong_multi_entity_type_attr_two(self): + Item = self.classes.Item + Keyword = self.classes.Keyword + self._assert_eager_with_entity_exception( + [Keyword, Item], + (joinedload_all(Keyword.keywords, Item.keywords), ), + r"Attribute 'keywords' of entity 'Mapper\|Keyword\|keywords' " + "does not refer to a mapped entity" + ) + + def test_option_against_wrong_multi_entity_type_attr_three(self): + Item = self.classes.Item + Keyword = self.classes.Keyword + self._assert_eager_with_entity_exception( + [Keyword.id, Item.id], + (joinedload_all(Keyword.keywords, Item.keywords), ), + r"Query has only expression-based entities - " + "can't find property named 'keywords'." + ) + + def test_wrong_type_in_option(self): + Item = self.classes.Item + Keyword = self.classes.Keyword + self._assert_eager_with_entity_exception( + [Item], + (joinedload_all(Keyword), ), + r"mapper option expects string key or list of attributes" + ) + + def test_non_contiguous_all_option(self): + User = self.classes.User + self._assert_eager_with_entity_exception( + [User], + (joinedload_all(User.addresses, User.orders), ), + r"Attribute 'User.orders' does not link " + "from element 'Mapper|Address|addresses'" + ) + + def test_non_contiguous_all_option_of_type(self): + User = self.classes.User + Order = self.classes.Order + self._assert_eager_with_entity_exception( + [User], + (joinedload_all(User.addresses, User.orders.of_type(Order)), ), + r"Attribute 'User.orders' does not link " + "from element 'Mapper|Address|addresses'" + ) + + @classmethod + def setup_mappers(cls): + users, User, addresses, Address, orders, Order = ( + cls.tables.users, cls.classes.User, + cls.tables.addresses, cls.classes.Address, + cls.tables.orders, cls.classes.Order) + mapper(User, users, properties={ + 'addresses': relationship(Address), + 'orders': relationship(Order) + }) + mapper(Address, addresses) + mapper(Order, orders) + keywords, items, item_keywords, Keyword, Item = (cls.tables.keywords, + cls.tables.items, + cls.tables.item_keywords, + cls.classes.Keyword, + cls.classes.Item) + mapper(Keyword, keywords, properties={ + "keywords": column_property(keywords.c.name + "some keyword") + }) + mapper(Item, items, + properties=dict(keywords=relationship(Keyword, + secondary=item_keywords))) + + def _assert_option(self, entity_list, option): + Item = self.classes.Item + + q = create_session().query(*entity_list).\ + options(joinedload(option)) + key = ('loader', (inspect(Item), inspect(Item).attrs.keywords)) + assert key in q._attributes + + def _assert_eager_with_entity_exception(self, entity_list, options, + message): + assert_raises_message(sa.exc.ArgumentError, + message, + create_session().query(*entity_list).options, + *options) + + def _assert_eager_with_just_column_exception(self, column, + eager_option, message): + assert_raises_message(sa.exc.ArgumentError, message, + create_session().query(column).options, + joinedload(eager_option)) + diff --git a/test/orm/test_pickled.py b/test/orm/test_pickled.py index b54af93f2..753eee244 100644 --- a/test/orm/test_pickled.py +++ b/test/orm/test_pickled.py @@ -267,7 +267,7 @@ class PickleTest(fixtures.MappedTest): sa.orm.joinedload("addresses", Address.dingaling), ]: opt2 = pickle.loads(pickle.dumps(opt)) - eq_(opt.key, opt2.key) + eq_(opt.path, opt2.path) u1 = sess.query(User).options(opt).first() u2 = pickle.loads(pickle.dumps(u1)) diff --git a/test/orm/test_query.py b/test/orm/test_query.py index 7151ef0b6..e9d0f3a7e 100644 --- a/test/orm/test_query.py +++ b/test/orm/test_query.py @@ -2452,584 +2452,3 @@ class ExecutionOptionsTest(QueryTest): q1.all() -class OptionsTest(QueryTest): - """Test the _process_paths() method of PropertyOption.""" - - def _option_fixture(self, *arg): - from sqlalchemy.orm import interfaces - class Opt(interfaces.PropertyOption): - pass - return Opt(arg) - - def _make_path(self, path): - r = [] - for i, item in enumerate(path): - if i % 2 == 0: - if isinstance(item, type): - item = class_mapper(item) - else: - if isinstance(item, str): - item = inspect(r[-1]).mapper.attrs[item] - r.append(item) - return tuple(r) - - def _make_path_registry(self, path): - return orm_util.PathRegistry.coerce(self._make_path(path)) - - def _assert_path_result(self, opt, q, paths): - q._attributes = q._attributes.copy() - assert_paths = opt._process_paths(q, False) - eq_( - [p.path for p in assert_paths], - [self._make_path(p) for p in paths] - ) - - def test_get_path_one_level_string(self): - User = self.classes.User - - sess = Session() - q = sess.query(User) - - opt = self._option_fixture("addresses") - self._assert_path_result(opt, q, [(User, 'addresses')]) - - def test_get_path_one_level_attribute(self): - User = self.classes.User - - sess = Session() - q = sess.query(User) - - opt = self._option_fixture(User.addresses) - self._assert_path_result(opt, q, [(User, 'addresses')]) - - def test_path_on_entity_but_doesnt_match_currentpath(self): - User, Address = self.classes.User, self.classes.Address - - # ensure "current path" is fully consumed before - # matching against current entities. - # see [ticket:2098] - sess = Session() - q = sess.query(User) - opt = self._option_fixture('email_address', 'id') - q = sess.query(Address)._with_current_path( - orm_util.PathRegistry.coerce([inspect(User), - inspect(User).attrs.addresses]) - ) - self._assert_path_result(opt, q, []) - - def test_get_path_one_level_with_unrelated(self): - Order = self.classes.Order - - sess = Session() - q = sess.query(Order) - opt = self._option_fixture("addresses") - self._assert_path_result(opt, q, []) - - def test_path_multilevel_string(self): - Item, User, Order = (self.classes.Item, - self.classes.User, - self.classes.Order) - - sess = Session() - q = sess.query(User) - - opt = self._option_fixture("orders.items.keywords") - self._assert_path_result(opt, q, [ - (User, 'orders'), - (User, 'orders', Order, 'items'), - (User, 'orders', Order, 'items', Item, 'keywords') - ]) - - def test_path_multilevel_attribute(self): - Item, User, Order = (self.classes.Item, - self.classes.User, - self.classes.Order) - - sess = Session() - q = sess.query(User) - - opt = self._option_fixture(User.orders, Order.items, Item.keywords) - self._assert_path_result(opt, q, [ - (User, 'orders'), - (User, 'orders', Order, 'items'), - (User, 'orders', Order, 'items', Item, 'keywords') - ]) - - def test_with_current_matching_string(self): - Item, User, Order = (self.classes.Item, - self.classes.User, - self.classes.Order) - - sess = Session() - q = sess.query(Item)._with_current_path( - self._make_path_registry([User, 'orders', Order, 'items']) - ) - - opt = self._option_fixture("orders.items.keywords") - self._assert_path_result(opt, q, [ - (Item, 'keywords') - ]) - - def test_with_current_matching_attribute(self): - Item, User, Order = (self.classes.Item, - self.classes.User, - self.classes.Order) - - sess = Session() - q = sess.query(Item)._with_current_path( - self._make_path_registry([User, 'orders', Order, 'items']) - ) - - opt = self._option_fixture(User.orders, Order.items, Item.keywords) - self._assert_path_result(opt, q, [ - (Item, 'keywords') - ]) - - def test_with_current_nonmatching_string(self): - Item, User, Order = (self.classes.Item, - self.classes.User, - self.classes.Order) - - sess = Session() - q = sess.query(Item)._with_current_path( - self._make_path_registry([User, 'orders', Order, 'items']) - ) - - opt = self._option_fixture("keywords") - self._assert_path_result(opt, q, []) - - opt = self._option_fixture("items.keywords") - self._assert_path_result(opt, q, []) - - def test_with_current_nonmatching_attribute(self): - Item, User, Order = (self.classes.Item, - self.classes.User, - self.classes.Order) - - sess = Session() - q = sess.query(Item)._with_current_path( - self._make_path_registry([User, 'orders', Order, 'items']) - ) - - opt = self._option_fixture(Item.keywords) - self._assert_path_result(opt, q, []) - - opt = self._option_fixture(Order.items, Item.keywords) - self._assert_path_result(opt, q, []) - - def test_from_base_to_subclass_attr(self): - Dingaling, Address = self.classes.Dingaling, self.classes.Address - - sess = Session() - class SubAddr(Address): - pass - mapper(SubAddr, inherits=Address, properties={ - 'flub': relationship(Dingaling) - }) - - q = sess.query(Address) - opt = self._option_fixture(SubAddr.flub) - - self._assert_path_result(opt, q, [(SubAddr, 'flub')]) - - def test_from_subclass_to_subclass_attr(self): - Dingaling, Address = self.classes.Dingaling, self.classes.Address - - sess = Session() - class SubAddr(Address): - pass - mapper(SubAddr, inherits=Address, properties={ - 'flub': relationship(Dingaling) - }) - - q = sess.query(SubAddr) - opt = self._option_fixture(SubAddr.flub) - - self._assert_path_result(opt, q, [(SubAddr, 'flub')]) - - def test_from_base_to_base_attr_via_subclass(self): - Dingaling, Address = self.classes.Dingaling, self.classes.Address - - sess = Session() - class SubAddr(Address): - pass - mapper(SubAddr, inherits=Address, properties={ - 'flub': relationship(Dingaling) - }) - - q = sess.query(Address) - opt = self._option_fixture(SubAddr.user) - - self._assert_path_result(opt, q, - [(Address, inspect(Address).attrs.user)]) - - def test_of_type(self): - User, Address = self.classes.User, self.classes.Address - - sess = Session() - class SubAddr(Address): - pass - mapper(SubAddr, inherits=Address) - - q = sess.query(User) - opt = self._option_fixture(User.addresses.of_type(SubAddr), SubAddr.user) - - u_mapper = inspect(User) - a_mapper = inspect(Address) - self._assert_path_result(opt, q, [ - (u_mapper, u_mapper.attrs.addresses), - (u_mapper, u_mapper.attrs.addresses, a_mapper, a_mapper.attrs.user) - ]) - - def test_of_type_plus_level(self): - Dingaling, User, Address = (self.classes.Dingaling, - self.classes.User, - self.classes.Address) - - sess = Session() - class SubAddr(Address): - pass - mapper(SubAddr, inherits=Address, properties={ - 'flub': relationship(Dingaling) - }) - - q = sess.query(User) - opt = self._option_fixture(User.addresses.of_type(SubAddr), SubAddr.flub) - - u_mapper = inspect(User) - sa_mapper = inspect(SubAddr) - self._assert_path_result(opt, q, [ - (u_mapper, u_mapper.attrs.addresses), - (u_mapper, u_mapper.attrs.addresses, sa_mapper, sa_mapper.attrs.flub) - ]) - - def test_aliased_single(self): - User = self.classes.User - - sess = Session() - ualias = aliased(User) - q = sess.query(ualias) - opt = self._option_fixture(ualias.addresses) - self._assert_path_result(opt, q, [(inspect(ualias), 'addresses')]) - - def test_with_current_aliased_single(self): - User, Address = self.classes.User, self.classes.Address - - sess = Session() - ualias = aliased(User) - q = sess.query(ualias)._with_current_path( - self._make_path_registry([Address, 'user']) - ) - opt = self._option_fixture(Address.user, ualias.addresses) - self._assert_path_result(opt, q, [(inspect(ualias), 'addresses')]) - - def test_with_current_aliased_single_nonmatching_option(self): - User, Address = self.classes.User, self.classes.Address - - sess = Session() - ualias = aliased(User) - q = sess.query(User)._with_current_path( - self._make_path_registry([Address, 'user']) - ) - opt = self._option_fixture(Address.user, ualias.addresses) - self._assert_path_result(opt, q, []) - - def test_with_current_aliased_single_nonmatching_entity(self): - User, Address = self.classes.User, self.classes.Address - - sess = Session() - ualias = aliased(User) - q = sess.query(ualias)._with_current_path( - self._make_path_registry([Address, 'user']) - ) - opt = self._option_fixture(Address.user, User.addresses) - self._assert_path_result(opt, q, []) - - def test_multi_entity_opt_on_second(self): - Item = self.classes.Item - Order = self.classes.Order - opt = self._option_fixture(Order.items) - sess = Session() - q = sess.query(Item, Order) - self._assert_path_result(opt, q, [(Order, "items")]) - - def test_multi_entity_opt_on_string(self): - Item = self.classes.Item - Order = self.classes.Order - opt = self._option_fixture("items") - sess = Session() - q = sess.query(Item, Order) - self._assert_path_result(opt, q, []) - - def test_multi_entity_no_mapped_entities(self): - Item = self.classes.Item - Order = self.classes.Order - opt = self._option_fixture("items") - sess = Session() - q = sess.query(Item.id, Order.id) - self._assert_path_result(opt, q, []) - - def test_path_exhausted(self): - User = self.classes.User - Item = self.classes.Item - Order = self.classes.Order - opt = self._option_fixture(User.orders) - sess = Session() - q = sess.query(Item)._with_current_path( - self._make_path_registry([User, 'orders', Order, 'items']) - ) - self._assert_path_result(opt, q, []) - -class OptionsNoPropTest(_fixtures.FixtureTest): - """test the error messages emitted when using property - options in conjunection with column-only entities, or - for not existing options - - """ - - run_create_tables = False - run_inserts = None - run_deletes = None - - def test_option_with_mapper_basestring(self): - Item = self.classes.Item - - self._assert_option([Item], 'keywords') - - def test_option_with_mapper_PropCompatator(self): - Item = self.classes.Item - - self._assert_option([Item], Item.keywords) - - def test_option_with_mapper_then_column_basestring(self): - Item = self.classes.Item - - self._assert_option([Item, Item.id], 'keywords') - - def test_option_with_mapper_then_column_PropComparator(self): - Item = self.classes.Item - - self._assert_option([Item, Item.id], Item.keywords) - - def test_option_with_column_then_mapper_basestring(self): - Item = self.classes.Item - - self._assert_option([Item.id, Item], 'keywords') - - def test_option_with_column_then_mapper_PropComparator(self): - Item = self.classes.Item - - self._assert_option([Item.id, Item], Item.keywords) - - def test_option_with_column_basestring(self): - Item = self.classes.Item - - message = \ - "Query has only expression-based entities - "\ - "can't find property named 'keywords'." - self._assert_eager_with_just_column_exception(Item.id, - 'keywords', message) - - def test_option_with_column_PropComparator(self): - Item = self.classes.Item - - self._assert_eager_with_just_column_exception(Item.id, - Item.keywords, - "Query has only expression-based entities " - "- can't find property named 'keywords'." - ) - - def test_option_against_nonexistent_PropComparator(self): - Item = self.classes.Item - Keyword = self.classes.Keyword - self._assert_eager_with_entity_exception( - [Keyword], - (joinedload(Item.keywords), ), - r"Can't find property 'keywords' on any entity specified " - r"in this Query. Note the full path from root " - r"\(Mapper\|Keyword\|keywords\) to target entity must be specified." - ) - - def test_option_against_nonexistent_basestring(self): - Item = self.classes.Item - self._assert_eager_with_entity_exception( - [Item], - (joinedload("foo"), ), - r"Can't find property named 'foo' on the mapped " - r"entity Mapper\|Item\|items in this Query." - ) - - def test_option_against_nonexistent_twolevel_basestring(self): - Item = self.classes.Item - self._assert_eager_with_entity_exception( - [Item], - (joinedload("keywords.foo"), ), - r"Can't find property named 'foo' on the mapped entity " - r"Mapper\|Keyword\|keywords in this Query." - ) - - def test_option_against_nonexistent_twolevel_all(self): - Item = self.classes.Item - self._assert_eager_with_entity_exception( - [Item], - (joinedload_all("keywords.foo"), ), - r"Can't find property named 'foo' on the mapped entity " - r"Mapper\|Keyword\|keywords in this Query." - ) - - @testing.fails_if(lambda:True, - "PropertyOption doesn't yet check for relation/column on end result") - def test_option_against_non_relation_basestring(self): - Item = self.classes.Item - Keyword = self.classes.Keyword - self._assert_eager_with_entity_exception( - [Keyword, Item], - (joinedload_all("keywords"), ), - r"Attribute 'keywords' of entity 'Mapper\|Keyword\|keywords' " - "does not refer to a mapped entity" - ) - - @testing.fails_if(lambda:True, - "PropertyOption doesn't yet check for relation/column on end result") - def test_option_against_multi_non_relation_basestring(self): - Item = self.classes.Item - Keyword = self.classes.Keyword - self._assert_eager_with_entity_exception( - [Keyword, Item], - (joinedload_all("keywords"), ), - r"Attribute 'keywords' of entity 'Mapper\|Keyword\|keywords' " - "does not refer to a mapped entity" - ) - - def test_option_against_wrong_entity_type_basestring(self): - Item = self.classes.Item - self._assert_eager_with_entity_exception( - [Item], - (joinedload_all("id", "keywords"), ), - r"Attribute 'id' of entity 'Mapper\|Item\|items' does not " - r"refer to a mapped entity" - ) - - def test_option_against_multi_non_relation_twolevel_basestring(self): - Item = self.classes.Item - Keyword = self.classes.Keyword - self._assert_eager_with_entity_exception( - [Keyword, Item], - (joinedload_all("id", "keywords"), ), - r"Attribute 'id' of entity 'Mapper\|Keyword\|keywords' " - "does not refer to a mapped entity" - ) - - def test_option_against_multi_nonexistent_basestring(self): - Item = self.classes.Item - Keyword = self.classes.Keyword - self._assert_eager_with_entity_exception( - [Keyword, Item], - (joinedload_all("description"), ), - r"Can't find property named 'description' on the mapped " - r"entity Mapper\|Keyword\|keywords in this Query." - ) - - def test_option_against_multi_no_entities_basestring(self): - Item = self.classes.Item - Keyword = self.classes.Keyword - self._assert_eager_with_entity_exception( - [Keyword.id, Item.id], - (joinedload_all("keywords"), ), - r"Query has only expression-based entities - can't find property " - "named 'keywords'." - ) - - def test_option_against_wrong_multi_entity_type_attr_one(self): - Item = self.classes.Item - Keyword = self.classes.Keyword - self._assert_eager_with_entity_exception( - [Keyword, Item], - (joinedload_all(Keyword.id, Item.keywords), ), - r"Attribute 'Keyword.id' of entity 'Mapper\|Keyword\|keywords' " - "does not refer to a mapped entity" - ) - - def test_option_against_wrong_multi_entity_type_attr_two(self): - Item = self.classes.Item - Keyword = self.classes.Keyword - self._assert_eager_with_entity_exception( - [Keyword, Item], - (joinedload_all(Keyword.keywords, Item.keywords), ), - r"Attribute 'Keyword.keywords' of entity 'Mapper\|Keyword\|keywords' " - "does not refer to a mapped entity" - ) - - def test_option_against_wrong_multi_entity_type_attr_three(self): - Item = self.classes.Item - Keyword = self.classes.Keyword - self._assert_eager_with_entity_exception( - [Keyword.id, Item.id], - (joinedload_all(Keyword.keywords, Item.keywords), ), - r"Query has only expression-based entities - " - "can't find property named 'keywords'." - ) - - def test_wrong_type_in_option(self): - Item = self.classes.Item - Keyword = self.classes.Keyword - self._assert_eager_with_entity_exception( - [Item], - (joinedload_all(Keyword), ), - r"mapper option expects string key or list of attributes" - ) - - def test_non_contiguous_all_option(self): - User = self.classes.User - self._assert_eager_with_entity_exception( - [User], - (joinedload_all(User.addresses, User.orders), ), - r"Attribute 'User.orders' does not link " - "from element 'Mapper|Address|addresses'" - ) - - @classmethod - def setup_mappers(cls): - users, User, addresses, Address, orders, Order = ( - cls.tables.users, cls.classes.User, - cls.tables.addresses, cls.classes.Address, - cls.tables.orders, cls.classes.Order) - mapper(User, users, properties={ - 'addresses': relationship(Address), - 'orders': relationship(Order) - }) - mapper(Address, addresses) - mapper(Order, orders) - keywords, items, item_keywords, Keyword, Item = (cls.tables.keywords, - cls.tables.items, - cls.tables.item_keywords, - cls.classes.Keyword, - cls.classes.Item) - mapper(Keyword, keywords, properties={ - "keywords": column_property(keywords.c.name + "some keyword") - }) - mapper(Item, items, - properties=dict(keywords=relationship(Keyword, - secondary=item_keywords))) - - def _assert_option(self, entity_list, option): - Item = self.classes.Item - - q = create_session().query(*entity_list).\ - options(joinedload(option)) - key = ('loaderstrategy', (inspect(Item), inspect(Item).attrs.keywords)) - assert key in q._attributes - - def _assert_eager_with_entity_exception(self, entity_list, options, - message): - assert_raises_message(sa.exc.ArgumentError, - message, - create_session().query(*entity_list).options, - *options) - - def _assert_eager_with_just_column_exception(self, column, - eager_option, message): - assert_raises_message(sa.exc.ArgumentError, message, - create_session().query(column).options, - joinedload(eager_option)) - |