diff options
| author | Mike Bayer <mike_mp@zzzcomputing.com> | 2022-06-30 13:33:30 -0400 |
|---|---|---|
| committer | Mike Bayer <mike_mp@zzzcomputing.com> | 2022-06-30 13:33:30 -0400 |
| commit | 59bafe0fbefe16269c72ac39e699e4127d49841f (patch) | |
| tree | 90f84da9241bcd6d6e0c63d83d787ccd4171b7d5 | |
| parent | b35bbdb13be9a359fc3cfae00e2b6fdca63dea29 (diff) | |
| download | sqlalchemy-59bafe0fbefe16269c72ac39e699e4127d49841f.tar.gz | |
add more cross-linking / notes for yield_per, partitions
Change-Id: I0f8db2532827c76a2751186638d22104230db843
references: #8198
| -rw-r--r-- | doc/build/orm/queryguide.rst | 19 | ||||
| -rw-r--r-- | lib/sqlalchemy/engine/result.py | 20 |
2 files changed, 38 insertions, 1 deletions
diff --git a/doc/build/orm/queryguide.rst b/doc/build/orm/queryguide.rst index f4e2a0ac6..ab93fc5ae 100644 --- a/doc/build/orm/queryguide.rst +++ b/doc/build/orm/queryguide.rst @@ -1031,7 +1031,24 @@ often useful to use with a result partitioning method such as (User(id=1, name='spongebob', fullname='Spongebob Squarepants'),) ... -The purpose of this method is when fetching very large result sets +For expediency, the :meth:`_engine.Result.yield_per` method may also be used +with an ORM-enabled result set, which will have the same effect at result +fetching time as if the ``yield_per`` execution option were used. The +:meth:`_engine.Result.partitions` method, if used, automatically uses the +number sent to :meth:`_engine.Result.yield_per` as the number of rows in each +partition:: + + >>> stmt = select(User) + {sql}>>> for partition in session.execute(stmt).yield_per(10).partitions(): + ... for row in partition: + ... print(row) + SELECT user_account.id, user_account.name, user_account.fullname + FROM user_account + [...] (){stop} + (User(id=1, name='spongebob', fullname='Spongebob Squarepants'),) + ... + +The purpose of "yield per" is when fetching very large result sets (> 10K rows), to batch results in sub-collections and yield them out partially, so that the Python interpreter doesn't need to declare very large areas of memory which is both time consuming and leads diff --git a/lib/sqlalchemy/engine/result.py b/lib/sqlalchemy/engine/result.py index b29027eb5..52be2603c 100644 --- a/lib/sqlalchemy/engine/result.py +++ b/lib/sqlalchemy/engine/result.py @@ -965,6 +965,12 @@ class Result(_WithKeys, ResultInternal[Row[_TP]]): :param num: number of rows to fetch each time the buffer is refilled. If set to a value below 1, fetches all rows for the next buffer. + .. seealso:: + + :ref:`orm_queryguide_yield_per` - in the :ref:`queryguide_toplevel` + + :meth:`_engine.Result.partitions` + """ self._yield_per = num return self @@ -1211,6 +1217,13 @@ class Result(_WithKeys, ResultInternal[Row[_TP]]): results, if possible. Not all drivers support this option and the option is silently ignored for those who do not. + When using the ORM, the :meth:`_engine.Result.partitions` method + is typically more effective from a memory perspective when it is + combined with use of the :meth:`_engine.Result.yield_per` method, + which instructs the ORM loading internals to only build a certain + amount of ORM objects from a result at a time before yielding + them out. + .. versionadded:: 1.4 :param size: indicate the maximum number of rows to be present @@ -1221,6 +1234,13 @@ class Result(_WithKeys, ResultInternal[Row[_TP]]): :return: iterator of lists + .. seealso:: + + :paramref:`.Connection.execution_options.stream_results` + + :ref:`orm_queryguide_yield_per` - in the :ref:`queryguide_toplevel` + + """ getter = self._manyrow_getter |
