From 84e0707afa587e7655410561324ac36085db2b95 Mon Sep 17 00:00:00 2001 From: Matti Picus Date: Mon, 25 Oct 2021 21:53:48 +0300 Subject: ENH: Configurable allocator (#17582) Fixes gh-17467. Adds a public struct to hold memory manipulation routines PyDataMem_Handler and two new API functions PyDataMem_SetHandler to replace the current routines with the new ones, and PyDataMem_GetHandlerName to get the string name of the current routines (either globally or for a specific ndarray object). This also changes the size of the ndarray object to hold the PyDataMem_Handler active when it was created so subsequent actions on its data memory will remain consistent. Tests and documentation are included. Along the way, I found some places in the code where the current policy is inconsistent (all data memory handling should have gone through npy_*_cache not PyDataMem_*) so even if this is rejected it might improve the cache handling. The PyDataMem_Handler has fields to override memcpy, these are currently not implemented: memcpy in the code base is untouched. I think this PR is invasive enough as-is, if desired memcpy can be handled in a follow-up PR. * ENH: add and use global configurable memory routines * ENH: add tests and a way to compile c-extensions from tests * fix allocation/free exposed by tests * DOC: document the new APIs (and some old ones too) * BUG: return void from FREE, also some cleanup * MAINT: changes from review * fixes from linter * setting ndarray->descr on 0d or scalars mess with FREE * make scalar allocation more consistent wrt np_alloc_cache * change formatting for sphinx * remove memcpy variants * update to match NEP 49 * ENH: add a python-level get_handler_name * ENH: add core.multiarray.get_handler_name * Allow closure-like definition of the data mem routines * Fix incompatible pointer warnings * Note PyDataMemAllocator and PyMemAllocatorEx differentiation Co-authored-by: Matti Picus * Redefine default allocator handling * Always allocate new arrays using the current_handler * Search for the mem_handler name of the data owner * Sub-comparisons don't need a local mem_handler * Make the default_handler a valid PyDataMem_Handler * Fix PyDataMem_SetHandler description (NEP discussion) * Pass the allocators by reference * Implement allocator context-locality * Fix documentation, make PyDataMem_GetHandler return const * remove import of setuptools==49.1.3, doesn't work on python3.10 * Fix refcount leaks * fix function signatures in test * Return early on PyDataMem_GetHandler error (VOID_compare) * Add context/thread-locality tests, allow testing custom policies * ENH: add and use global configurable memory routines * ENH: add tests and a way to compile c-extensions from tests * fix allocation/free exposed by tests * DOC: document the new APIs (and some old ones too) * BUG: return void from FREE, also some cleanup * MAINT: changes from review * fixes from linter * setting ndarray->descr on 0d or scalars mess with FREE * make scalar allocation more consistent wrt np_alloc_cache * change formatting for sphinx * remove memcpy variants * update to match NEP 49 * ENH: add a python-level get_handler_name * ENH: add core.multiarray.get_handler_name * Allow closure-like definition of the data mem routines * Fix incompatible pointer warnings * Note PyDataMemAllocator and PyMemAllocatorEx differentiation Co-authored-by: Matti Picus * Redefine default allocator handling * Always allocate new arrays using the current_handler * Search for the mem_handler name of the data owner * Sub-comparisons don't need a local mem_handler * Make the default_handler a valid PyDataMem_Handler * Fix PyDataMem_SetHandler description (NEP discussion) * Pass the allocators by reference * remove import of setuptools==49.1.3, doesn't work on python3.10 * fix function signatures in test * try to fix cygwin extension building * YAPF mem_policy test * Less empty lines, more comments (tests) * Apply suggestions from code review (set an exception and) Co-authored-by: Matti Picus * skip test on cygwin * update API hash for changed signature * TST: add gc.collect to make sure cycles are broken * Implement thread-locality for PyPy Co-authored-by: Sebastian Berg * Update numpy/core/tests/test_mem_policy.py Co-authored-by: Sebastian Berg * fixes from review * update circleci config * fix test * make the connection between OWNDATA and having a allocator handle more explicit * improve docstring, fix flake8 for tests * update PyDataMem_GetHandler() from review * Implement allocator lifetime management * update NEP and add best-effort handling of error in PyDataMem_UserFREE * ENH: fix and test for blindly taking ownership of data * Update doc/neps/nep-0049.rst Co-authored-by: Elias Koromilas --- doc/source/reference/c-api/data_memory.rst | 119 +++++++++++++++++++++++++++++ doc/source/reference/c-api/index.rst | 1 + 2 files changed, 120 insertions(+) create mode 100644 doc/source/reference/c-api/data_memory.rst (limited to 'doc/source/reference/c-api') diff --git a/doc/source/reference/c-api/data_memory.rst b/doc/source/reference/c-api/data_memory.rst new file mode 100644 index 000000000..8e2989403 --- /dev/null +++ b/doc/source/reference/c-api/data_memory.rst @@ -0,0 +1,119 @@ +Memory management in NumPy +========================== + +The `numpy.ndarray` is a python class. It requires additional memory allocations +to hold `numpy.ndarray.strides`, `numpy.ndarray.shape` and +`numpy.ndarray.data` attributes. These attributes are specially allocated +after creating the python object in `__new__`. The ``strides`` and +``shape`` are stored in a piece of memory allocated internally. + +The ``data`` allocation used to store the actual array values (which could be +pointers in the case of ``object`` arrays) can be very large, so NumPy has +provided interfaces to manage its allocation and release. This document details +how those interfaces work. + +Historical overview +------------------- + +Since version 1.7.0, NumPy has exposed a set of ``PyDataMem_*`` functions +(:c:func:`PyDataMem_NEW`, :c:func:`PyDataMem_FREE`, :c:func:`PyDataMem_RENEW`) +which are backed by `alloc`, `free`, `realloc` respectively. In that version +NumPy also exposed the `PyDataMem_EventHook` function described below, which +wrap the OS-level calls. + +Since those early days, Python also improved its memory management +capabilities, and began providing +various :ref:`management policies ` beginning in version +3.4. These routines are divided into a set of domains, each domain has a +:c:type:`PyMemAllocatorEx` structure of routines for memory management. Python also +added a `tracemalloc` module to trace calls to the various routines. These +tracking hooks were added to the NumPy ``PyDataMem_*`` routines. + +NumPy added a small cache of allocated memory in its internal +``npy_alloc_cache``, ``npy_alloc_cache_zero``, and ``npy_free_cache`` +functions. These wrap ``alloc``, ``alloc-and-memset(0)`` and ``free`` +respectively, but when ``npy_free_cache`` is called, it adds the pointer to a +short list of available blocks marked by size. These blocks can be re-used by +subsequent calls to ``npy_alloc*``, avoiding memory thrashing. + +Configurable memory routines in NumPy (NEP 49) +---------------------------------------------- + +Users may wish to override the internal data memory routines with ones of their +own. Since NumPy does not use the Python domain strategy to manage data memory, +it provides an alternative set of C-APIs to change memory routines. There are +no Python domain-wide strategies for large chunks of object data, so those are +less suited to NumPy's needs. User who wish to change the NumPy data memory +management routines can use :c:func:`PyDataMem_SetHandler`, which uses a +:c:type:`PyDataMem_Handler` structure to hold pointers to functions used to +manage the data memory. The calls are still wrapped by internal routines to +call :c:func:`PyTraceMalloc_Track`, :c:func:`PyTraceMalloc_Untrack`, and will +use the :c:func:`PyDataMem_EventHookFunc` mechanism. Since the functions may +change during the lifetime of the process, each ``ndarray`` carries with it the +functions used at the time of its instantiation, and these will be used to +reallocate or free the data memory of the instance. + +.. c:type:: PyDataMem_Handler + + A struct to hold function pointers used to manipulate memory + + .. code-block:: c + + typedef struct { + char name[128]; /* multiple of 64 to keep the struct aligned */ + PyDataMemAllocator allocator; + } PyDataMem_Handler; + + where the allocator structure is + + .. code-block:: c + + /* The declaration of free differs from PyMemAllocatorEx */ + typedef struct { + void *ctx; + void* (*malloc) (void *ctx, size_t size); + void* (*calloc) (void *ctx, size_t nelem, size_t elsize); + void* (*realloc) (void *ctx, void *ptr, size_t new_size); + void (*free) (void *ctx, void *ptr, size_t size); + } PyDataMemAllocator; + +.. c:function:: PyObject * PyDataMem_SetHandler(PyObject *handler) + + Set a new allocation policy. If the input value is ``NULL``, will reset the + policy to the default. Return the previous policy, or + return ``NULL`` if an error has occurred. We wrap the user-provided functions + so they will still call the python and numpy memory management callback + hooks. + +.. c:function:: PyObject * PyDataMem_GetHandler() + + Return the current policy that will be used to allocate data for the + next ``PyArrayObject``. On failure, return ``NULL``. + +For an example of setting up and using the PyDataMem_Handler, see the test in +:file:`numpy/core/tests/test_mem_policy.py` + +.. c:function:: void PyDataMem_EventHookFunc(void *inp, void *outp, size_t size, void *user_data); + + This function will be called during data memory manipulation + +.. c:function:: PyDataMem_EventHookFunc * PyDataMem_SetEventHook(PyDataMem_EventHookFunc *newhook, void *user_data, void **old_data) + + Sets the allocation event hook for numpy array data. + + Returns a pointer to the previous hook or ``NULL``. If old_data is + non-``NULL``, the previous user_data pointer will be copied to it. + + If not ``NULL``, hook will be called at the end of each ``PyDataMem_NEW/FREE/RENEW``: + + .. code-block:: c + + result = PyDataMem_NEW(size) -> (*hook)(NULL, result, size, user_data) + PyDataMem_FREE(ptr) -> (*hook)(ptr, NULL, 0, user_data) + result = PyDataMem_RENEW(ptr, size) -> (*hook)(ptr, result, size, user_data) + + When the hook is called, the GIL will be held by the calling + thread. The hook should be written to be reentrant, if it performs + operations that might cause new allocation events (such as the + creation/destruction numpy objects, or creating/destroying Python + objects which might cause a gc) diff --git a/doc/source/reference/c-api/index.rst b/doc/source/reference/c-api/index.rst index bb1ed154e..6288ff33b 100644 --- a/doc/source/reference/c-api/index.rst +++ b/doc/source/reference/c-api/index.rst @@ -49,3 +49,4 @@ code. generalized-ufuncs coremath deprecations + data_memory -- cgit v1.2.1 From 9d10844277bc5f49724531b44b803ebfde3936ec Mon Sep 17 00:00:00 2001 From: Matti Picus Date: Tue, 26 Oct 2021 19:57:19 +0300 Subject: DOC: add release note and move NEP 49 to Final (#20194) * DOC: add release note and move NEP 49 to Final * fix link * fix release note --- doc/source/reference/c-api/data_memory.rst | 2 ++ 1 file changed, 2 insertions(+) (limited to 'doc/source/reference/c-api') diff --git a/doc/source/reference/c-api/data_memory.rst b/doc/source/reference/c-api/data_memory.rst index 8e2989403..c17f98a2c 100644 --- a/doc/source/reference/c-api/data_memory.rst +++ b/doc/source/reference/c-api/data_memory.rst @@ -1,3 +1,5 @@ +.. _data_memory: + Memory management in NumPy ========================== -- cgit v1.2.1 From 50fc62cc73f1551cea6f3a1e1dd73e03d12065b8 Mon Sep 17 00:00:00 2001 From: warren Date: Mon, 25 Oct 2021 21:07:05 -0400 Subject: DOC: Two small changes in array.rst: * Include the return values of `PyArray_RegisterCanCast` in the description. * Correct the documentation of `PyArray_Scalar`. --- doc/source/reference/c-api/array.rst | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) (limited to 'doc/source/reference/c-api') diff --git a/doc/source/reference/c-api/array.rst b/doc/source/reference/c-api/array.rst index 6a135fd71..2654b135d 100644 --- a/doc/source/reference/c-api/array.rst +++ b/doc/source/reference/c-api/array.rst @@ -1323,7 +1323,7 @@ User-defined data types data-type object, *descr*, of the given *scalar* kind. Use *scalar* = :c:data:`NPY_NOSCALAR` to register that an array of data-type *descr* can be cast safely to a data-type whose type_number is - *totype*. + *totype*. The return value is 0 on success or -1 on failure. .. c:function:: int PyArray_TypeNumFromName( \ char const *str) @@ -2778,13 +2778,19 @@ Array Scalars whenever 0-dimensional arrays could be returned to Python. .. c:function:: PyObject* PyArray_Scalar( \ - void* data, PyArray_Descr* dtype, PyObject* itemsize) - - Return an array scalar object of the given enumerated *typenum* - and *itemsize* by **copying** from memory pointed to by *data* - . If *swap* is nonzero then this function will byteswap the data - if appropriate to the data-type because array scalars are always - in correct machine-byte order. + void* data, PyArray_Descr* dtype, PyObject* base) + + Return an array scalar object of the given *dtype* by **copying** + from memory pointed to by *data*. *base* is expected to be the + array object that is the owner of the data. *base* is required + if `dtype` is a ``void`` scalar, or if the ``NPY_USE_GETITEM`` + flag is set and it is known that the ``getitem`` method uses + the ``arr`` argument without checking if it is ``NULL``. Otherwise + `base` may be ``NULL``. + + If the data is not in native byte order (as indicated by + ``dtype->byteorder``) then this function will byteswap the data, + because array scalars are always in correct machine-byte order. .. c:function:: PyObject* PyArray_ToScalar(void* data, PyArrayObject* arr) -- cgit v1.2.1 From aebf38662647b328e5ac10c52a24202b3a22cf66 Mon Sep 17 00:00:00 2001 From: Matti Picus Date: Wed, 27 Oct 2021 23:09:51 +0300 Subject: MAINT: Only warn for transferred ownership if env variable is set (#20200) Builds on gh-20194. Fixes breakage of SciPy in https://github.com/scipy/scipy/issues/14917 At some point we could flip the default to "warn" instead of "no warning" * make warning conditional on NUMPY_WARN_IF_NO_MEM_POLICY * add test, fix example code * fixes from review * typo --- doc/source/reference/c-api/array.rst | 7 +++--- doc/source/reference/c-api/data_memory.rst | 37 ++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 3 deletions(-) (limited to 'doc/source/reference/c-api') diff --git a/doc/source/reference/c-api/array.rst b/doc/source/reference/c-api/array.rst index 6a135fd71..232690486 100644 --- a/doc/source/reference/c-api/array.rst +++ b/doc/source/reference/c-api/array.rst @@ -325,8 +325,7 @@ From scratch should be increased after the pointer is passed in, and the base member of the returned ndarray should point to the Python object that owns the data. This will ensure that the provided memory is not - freed while the returned array is in existence. To free memory as soon - as the ndarray is deallocated, set the OWNDATA flag on the returned ndarray. + freed while the returned array is in existence. .. c:function:: PyObject* PyArray_SimpleNewFromDescr( \ int nd, npy_int const* dims, PyArray_Descr* descr) @@ -1463,7 +1462,9 @@ of the constant names is deprecated in 1.7. .. c:macro:: NPY_ARRAY_OWNDATA - The data area is owned by this array. + The data area is owned by this array. Should never be set manually, instead + create a ``PyObject`` wrapping the data and set the array's base to that + object. For an example, see the test in ``test_mem_policy``. .. c:macro:: NPY_ARRAY_ALIGNED diff --git a/doc/source/reference/c-api/data_memory.rst b/doc/source/reference/c-api/data_memory.rst index c17f98a2c..11a37adc4 100644 --- a/doc/source/reference/c-api/data_memory.rst +++ b/doc/source/reference/c-api/data_memory.rst @@ -119,3 +119,40 @@ For an example of setting up and using the PyDataMem_Handler, see the test in operations that might cause new allocation events (such as the creation/destruction numpy objects, or creating/destroying Python objects which might cause a gc) + +What happens when deallocating if there is no policy set +-------------------------------------------------------- + +A rare but useful technique is to allocate a buffer outside NumPy, use +:c:func:`PyArray_NewFromDescr` to wrap the buffer in a ``ndarray``, then switch +the ``OWNDATA`` flag to true. When the ``ndarray`` is released, the +appropriate function from the ``ndarray``'s ``PyDataMem_Handler`` should be +called to free the buffer. But the ``PyDataMem_Handler`` field was never set, +it will be ``NULL``. For backward compatibility, NumPy will call ``free()`` to +release the buffer. If ``NUMPY_WARN_IF_NO_MEM_POLICY`` is set to ``1``, a +warning will be emitted. The current default is not to emit a warning, this may +change in a future version of NumPy. + +A better technique would be to use a ``PyCapsule`` as a base object: + +.. code-block:: c + + /* define a PyCapsule_Destructor, using the correct deallocator for buff */ + void free_wrap(void *capsule){ + void * obj = PyCapsule_GetPointer(capsule, PyCapsule_GetName(capsule)); + free(obj); + }; + + /* then inside the function that creates arr from buff */ + ... + arr = PyArray_NewFromDescr(... buf, ...); + if (arr == NULL) { + return NULL; + } + capsule = PyCapsule_New(buf, "my_wrapped_buffer", + (PyCapsule_Destructor)&free_wrap); + if (PyArray_SetBaseObject(arr, capsule) == -1) { + Py_DECREF(arr); + return NULL; + } + ... -- cgit v1.2.1 From 6312c18e99dad89231b65693a05c92c1f06d3671 Mon Sep 17 00:00:00 2001 From: mattip Date: Wed, 10 Nov 2021 00:16:56 +0200 Subject: ENH: add a 'version' field to PyDataMem_Handler --- doc/source/reference/c-api/data_memory.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'doc/source/reference/c-api') diff --git a/doc/source/reference/c-api/data_memory.rst b/doc/source/reference/c-api/data_memory.rst index 11a37adc4..b779026b4 100644 --- a/doc/source/reference/c-api/data_memory.rst +++ b/doc/source/reference/c-api/data_memory.rst @@ -62,7 +62,8 @@ reallocate or free the data memory of the instance. .. code-block:: c typedef struct { - char name[128]; /* multiple of 64 to keep the struct aligned */ + char name[127]; /* multiple of 64 to keep the struct aligned */ + uint8_t version; /* currently 1 */ PyDataMemAllocator allocator; } PyDataMem_Handler; -- cgit v1.2.1 From ee6a23b1186343394d4e927b786c0c53e9446034 Mon Sep 17 00:00:00 2001 From: Matti Picus Date: Thu, 18 Nov 2021 20:25:46 +0200 Subject: DEP: remove allocation_tracking, deprecate PyDataMem_SetEventHook (#20394) * DEP: remove allocation_tracking * DOC: add release note * DEP: deprecate PyDataMem_SetEventHook * DOC: fix name of release note * fixes from review * DOC: document deprecation of PyDataMem_EventHookFunc --- doc/source/reference/c-api/data_memory.rst | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) (limited to 'doc/source/reference/c-api') diff --git a/doc/source/reference/c-api/data_memory.rst b/doc/source/reference/c-api/data_memory.rst index b779026b4..2084ab5d0 100644 --- a/doc/source/reference/c-api/data_memory.rst +++ b/doc/source/reference/c-api/data_memory.rst @@ -20,8 +20,8 @@ Historical overview Since version 1.7.0, NumPy has exposed a set of ``PyDataMem_*`` functions (:c:func:`PyDataMem_NEW`, :c:func:`PyDataMem_FREE`, :c:func:`PyDataMem_RENEW`) which are backed by `alloc`, `free`, `realloc` respectively. In that version -NumPy also exposed the `PyDataMem_EventHook` function described below, which -wrap the OS-level calls. +NumPy also exposed the `PyDataMem_EventHook` function (now deprecated) +described below, which wrap the OS-level calls. Since those early days, Python also improved its memory management capabilities, and began providing @@ -50,10 +50,10 @@ management routines can use :c:func:`PyDataMem_SetHandler`, which uses a :c:type:`PyDataMem_Handler` structure to hold pointers to functions used to manage the data memory. The calls are still wrapped by internal routines to call :c:func:`PyTraceMalloc_Track`, :c:func:`PyTraceMalloc_Untrack`, and will -use the :c:func:`PyDataMem_EventHookFunc` mechanism. Since the functions may -change during the lifetime of the process, each ``ndarray`` carries with it the -functions used at the time of its instantiation, and these will be used to -reallocate or free the data memory of the instance. +use the deprecated :c:func:`PyDataMem_EventHookFunc` mechanism. Since the +functions may change during the lifetime of the process, each ``ndarray`` +carries with it the functions used at the time of its instantiation, and these +will be used to reallocate or free the data memory of the instance. .. c:type:: PyDataMem_Handler @@ -119,7 +119,9 @@ For an example of setting up and using the PyDataMem_Handler, see the test in thread. The hook should be written to be reentrant, if it performs operations that might cause new allocation events (such as the creation/destruction numpy objects, or creating/destroying Python - objects which might cause a gc) + objects which might cause a gc). + + Deprecated in v1.23 What happens when deallocating if there is no policy set -------------------------------------------------------- -- cgit v1.2.1