MAINT: Only warn for transferred ownership if env variable is set (#20200)

Builds on gh-20194. Fixes breakage of SciPy in https://github.com/scipy/scipy/issues/14917

At some point we could flip the default to "warn" instead of "no warning"

* make warning conditional on NUMPY_WARN_IF_NO_MEM_POLICY

* add test, fix example code

* fixes from review

* typo

3 files changed, 51 insertions, 3 deletions

diff --git a/doc/source/reference/c-api/array.rst b/doc/source/reference/c-api/array.rst
index 6a135fd71..232690486 100644
--- a/doc/source/reference/c-api/array.rst
+++ b/doc/source/reference/c-api/array.rst
@@ -325,8 +325,7 @@ From scratch
     should be increased after the pointer is passed in, and the base member
     of the returned ndarray should point to the Python object that owns
     the data. This will ensure that the provided memory is not
-    freed while the returned array is in existence. To free memory as soon
-    as the ndarray is deallocated, set the OWNDATA flag on the returned ndarray.
+    freed while the returned array is in existence.
 
 .. c:function:: PyObject* PyArray_SimpleNewFromDescr( \
         int nd, npy_int const* dims, PyArray_Descr* descr)
@@ -1463,7 +1462,9 @@ of the constant names is deprecated in 1.7.
 
 .. c:macro:: NPY_ARRAY_OWNDATA
 
-    The data area is owned by this array.
+    The data area is owned by this array. Should never be set manually, instead
+    create a ``PyObject`` wrapping the data and set the array's base to that
+    object. For an example, see the test in ``test_mem_policy``.
 
 .. c:macro:: NPY_ARRAY_ALIGNED
 
diff --git a/doc/source/reference/c-api/data_memory.rst b/doc/source/reference/c-api/data_memory.rst
index c17f98a2c..11a37adc4 100644
--- a/doc/source/reference/c-api/data_memory.rst
+++ b/doc/source/reference/c-api/data_memory.rst
@@ -119,3 +119,40 @@ For an example of setting up and using the PyDataMem_Handler, see the test in
     operations that might cause new allocation events (such as the
     creation/destruction numpy objects, or creating/destroying Python
     objects which might cause a gc)
+
+What happens when deallocating if there is no policy set
+--------------------------------------------------------
+
+A rare but useful technique is to allocate a buffer outside NumPy, use
+:c:func:`PyArray_NewFromDescr` to wrap the buffer in a ``ndarray``, then switch
+the ``OWNDATA`` flag to true. When the ``ndarray`` is released, the
+appropriate function from the ``ndarray``'s ``PyDataMem_Handler`` should be
+called to free the buffer. But the ``PyDataMem_Handler`` field was never set,
+it will be ``NULL``. For backward compatibility, NumPy will call ``free()`` to
+release the buffer. If ``NUMPY_WARN_IF_NO_MEM_POLICY`` is set to ``1``, a
+warning will be emitted. The current default is not to emit a warning, this may
+change in a future version of NumPy.
+
+A better technique would be to use a ``PyCapsule`` as a base object:
+
+.. code-block:: c
+
+    /* define a PyCapsule_Destructor, using the correct deallocator for buff */
+    void free_wrap(void *capsule){
+        void * obj = PyCapsule_GetPointer(capsule, PyCapsule_GetName(capsule));
+        free(obj); 
+    };
+
+    /* then inside the function that creates arr from buff */
+    ...
+    arr = PyArray_NewFromDescr(... buf, ...);
+    if (arr == NULL) {
+        return NULL;
+    }
+    capsule = PyCapsule_New(buf, "my_wrapped_buffer",
+                            (PyCapsule_Destructor)&free_wrap);
+    if (PyArray_SetBaseObject(arr, capsule) == -1) {
+        Py_DECREF(arr);
+        return NULL;
+    }
+    ...
diff --git a/doc/source/reference/global_state.rst b/doc/source/reference/global_state.rst
index f18481235..20874ceaa 100644
--- a/doc/source/reference/global_state.rst
+++ b/doc/source/reference/global_state.rst
@@ -84,3 +84,13 @@ contiguous in memory.
 Most users will have no reason to change these; for details
 see the :ref:`memory layout <memory-layout>` documentation.
 
+
+Warn if no memory allocation policy when deallocating data
+----------------------------------------------------------
+
+Some users might pass ownership of the data pointer to the ``ndarray`` by
+setting the ``OWNDATA`` flag. If they do this without setting (manually) a
+memory allocation policy, the default will be to call ``free``. If
+``NUMPY_WARN_IF_NO_MEM_POLICY`` is set to ``"1"``, a ``RuntimeWarning`` will
+be emitted. A better alternative is to use a ``PyCapsule`` with a deallocator
+and set the ``ndarray.base``.