Merge branch 'master' into fix-issue-10244

1 files changed, 61 insertions, 52 deletions

diff --git a/doc/source/reference/arrays.interface.rst b/doc/source/reference/arrays.interface.rst
index 4e95535c0..6a8c5f9c4 100644
--- a/doc/source/reference/arrays.interface.rst
+++ b/doc/source/reference/arrays.interface.rst
@@ -49,9 +49,9 @@ Python side
 ===========
 
 This approach to the interface consists of the object having an
-:data:`__array_interface__` attribute.
+:data:`~object.__array_interface__` attribute.
 
-.. data:: __array_interface__
+.. data:: object.__array_interface__
 
    A dictionary of items (3 required and 5 optional).  The optional
    keys in the dictionary have implied defaults if they are not
@@ -60,17 +60,15 @@ This approach to the interface consists of the object having an
    The keys are:
 
    **shape** (required)
-
        Tuple whose elements are the array size in each dimension. Each
-       entry is an integer (a Python int or long).  Note that these
-       integers could be larger than the platform "int" or "long"
-       could hold (a Python int is a C long). It is up to the code
+       entry is an integer (a Python :py:class:`int`).  Note that these
+       integers could be larger than the platform ``int`` or ``long``
+       could hold (a Python :py:class:`int` is a C ``long``). It is up to the code
        using this attribute to handle this appropriately; either by
        raising an error when overflow is possible, or by using
-       :c:data:`Py_LONG_LONG` as the C type for the shapes.
+       ``long long`` as the C type for the shapes.
 
    **typestr** (required)
-
        A string providing the basic type of the homogeneous array The
        basic string format consists of 3 parts: a character describing
        the byteorder of the data (``<``: little-endian, ``>``:
@@ -97,7 +95,6 @@ This approach to the interface consists of the object having an
        =====  ================================================================
 
    **descr** (optional)
-
        A list of tuples providing a more detailed description of the
        memory layout for each item in the homogeneous array.  Each
        tuple in the list has two or three elements.  Normally, this
@@ -127,7 +124,6 @@ This approach to the interface consists of the object having an
        **Default**: ``[('', typestr)]``
 
    **data** (optional)
-
        A 2-tuple whose first argument is an integer (a long integer
        if necessary) that points to the data-area storing the array
        contents.  This pointer must point to the first element of
@@ -136,7 +132,7 @@ This approach to the interface consists of the object having an
        means the data area is read-only).
 
        This attribute can also be an object exposing the
-       :c:func:`buffer interface <PyObject_AsCharBuffer>` which
+       :ref:`buffer interface <bufferobjects>` which
        will be used to share the data. If this key is not present (or
        returns None), then memory sharing will be done
        through the buffer interface of the object itself.  In this
@@ -148,25 +144,23 @@ This approach to the interface consists of the object having an
        **Default**: None
 
    **strides** (optional)
-
-       Either None to indicate a C-style contiguous array or
+       Either ``None`` to indicate a C-style contiguous array or
        a Tuple of strides which provides the number of bytes needed
        to jump to the next array element in the corresponding
        dimension. Each entry must be an integer (a Python
-       :const:`int` or :const:`long`). As with shape, the values may
-       be larger than can be represented by a C "int" or "long"; the
+       :py:class:`int`). As with shape, the values may
+       be larger than can be represented by a C ``int`` or ``long``; the
        calling code should handle this appropriately, either by
-       raising an error, or by using :c:type:`Py_LONG_LONG` in C. The
-       default is None which implies a C-style contiguous
-       memory buffer.  In this model, the last dimension of the array
+       raising an error, or by using ``long long`` in C. The
+       default is ``None`` which implies a C-style contiguous
+       memory buffer. In this model, the last dimension of the array
        varies the fastest.  For example, the default strides tuple
        for an object whose array entries are 8 bytes long and whose
-       shape is (10,20,30) would be (4800, 240, 8)
+       shape is ``(10, 20, 30)`` would be ``(4800, 240, 8)``
 
-       **Default**: None (C-style contiguous)
+       **Default**: ``None`` (C-style contiguous)
 
    **mask** (optional)
-
        None or an object exposing the array interface.  All
        elements of the mask array should be interpreted only as true
        or not true indicating which elements of this array are valid.
@@ -177,15 +171,13 @@ This approach to the interface consists of the object having an
        **Default**: None (All array values are valid)
 
    **offset** (optional)
-
        An integer offset into the array data region. This can only be
-       used when data is None or returns a :class:`buffer`
+       used when data is ``None`` or returns a :class:`buffer`
        object.
 
        **Default**: 0.
 
    **version** (required)
-
        An integer showing the version of the interface (i.e. 3 for
        this version).  Be careful not to use this to invalidate
        objects exposing future versions of the interface.
@@ -197,11 +189,11 @@ C-struct access
 This approach to the array interface allows for faster access to an
 array using only one attribute lookup and a well-defined C-structure.
 
-.. c:var:: __array_struct__
+.. data:: object.__array_struct__
 
-   A :c:type: `PyCObject` whose :c:data:`voidptr` member contains a
+   A :c:type:`PyCapsule` whose ``pointer`` member contains a
    pointer to a filled :c:type:`PyArrayInterface` structure.  Memory
-   for the structure is dynamically created and the :c:type:`PyCObject`
+   for the structure is dynamically created and the :c:type:`PyCapsule`
    is also created with an appropriate destructor so the retriever of
    this attribute simply has to apply :c:func:`Py_DECREF()` to the
    object returned by this attribute when it is finished.  Also,
@@ -211,7 +203,7 @@ array using only one attribute lookup and a well-defined C-structure.
    must also not reallocate their memory if other objects are
    referencing them.
 
-The PyArrayInterface structure is defined in ``numpy/ndarrayobject.h``
+The :c:type:`PyArrayInterface` structure is defined in ``numpy/ndarrayobject.h``
 as::
 
   typedef struct {
@@ -231,29 +223,32 @@ as::
 
 The flags member may consist of 5 bits showing how the data should be
 interpreted and one bit showing how the Interface should be
-interpreted.  The data-bits are :const:`CONTIGUOUS` (0x1),
-:const:`FORTRAN` (0x2), :const:`ALIGNED` (0x100), :const:`NOTSWAPPED`
-(0x200), and :const:`WRITEABLE` (0x400).  A final flag
-:const:`ARR_HAS_DESCR` (0x800) indicates whether or not this structure
+interpreted.  The data-bits are :c:macro:`NPY_ARRAY_C_CONTIGUOUS` (0x1),
+:c:macro:`NPY_ARRAY_F_CONTIGUOUS` (0x2), :c:macro:`NPY_ARRAY_ALIGNED` (0x100),
+:c:macro:`NPY_ARRAY_NOTSWAPPED` (0x200), and :c:macro:`NPY_ARRAY_WRITEABLE` (0x400).  A final flag
+:c:macro:`NPY_ARR_HAS_DESCR` (0x800) indicates whether or not this structure
 has the arrdescr field.  The field should not be accessed unless this
 flag is present.
 
+   .. c:macro:: NPY_ARR_HAS_DESCR
+
 .. admonition:: New since June 16, 2006:
 
-   In the past most implementations used the "desc" member of the
-   :c:type:`PyCObject` itself (do not confuse this with the "descr" member of
+   In the past most implementations used the ``desc`` member of the ``PyCObject``
+   (now :c:type:`PyCapsule`) itself (do not confuse this with the "descr" member of
    the :c:type:`PyArrayInterface` structure above --- they are two separate
    things) to hold the pointer to the object exposing the interface.
-   This is now an explicit part of the interface.  Be sure to own a
-   reference to the object when the :c:type:`PyCObject` is created using
-   :c:type:`PyCObject_FromVoidPtrAndDesc`.
+   This is now an explicit part of the interface.  Be sure to take a
+   reference to the object and call :c:func:`PyCapsule_SetContext` before
+   returning the :c:type:`PyCapsule`, and configure a destructor to decref this
+   reference.
 
 
 Type description examples
 =========================
 
 For clarity it is useful to provide some examples of the type
-description and corresponding :data:`__array_interface__` 'descr'
+description and corresponding :data:`~object.__array_interface__` 'descr'
 entries.  Thanks to Scott Gilbert for these examples:
 
 In every case, the 'descr' key is optional, but of course provides
@@ -315,25 +310,39 @@ largely aesthetic.  In particular:
 1. The PyArrayInterface structure had no descr member at the end
    (and therefore no flag ARR_HAS_DESCR)
 
-2. The desc member of the PyCObject returned from __array_struct__ was
+2. The ``context`` member of the :c:type:`PyCapsule` (formally the ``desc``
+   member of the ``PyCObject``) returned from ``__array_struct__`` was
    not specified.  Usually, it was the object exposing the array (so
    that a reference to it could be kept and destroyed when the
-   C-object was destroyed).  Now it must be a tuple whose first
-   element is a string with "PyArrayInterface Version #" and whose
-   second element is the object exposing the array.
+   C-object was destroyed). It is now an explicit requirement that this field
+   be used in some way to hold a reference to the owning object.
+
+   .. note::
+
+       Until August 2020, this said:
+
+           Now it must be a tuple whose first element is a string with
+           "PyArrayInterface Version #" and whose second element is the object
+           exposing the array.
+
+       This design was retracted almost immediately after it was proposed, in
+       <https://mail.python.org/pipermail/numpy-discussion/2006-June/020995.html>.
+       Despite 14 years of documentation to the contrary, at no point was it
+       valid to assume that ``__array_interface__`` capsules held this tuple
+       content.
 
-3. The tuple returned from __array_interface__['data'] used to be a
+3. The tuple returned from ``__array_interface__['data']`` used to be a
    hex-string (now it is an integer or a long integer).
 
-4. There was no __array_interface__ attribute instead all of the keys
-   (except for version) in the __array_interface__ dictionary were
+4. There was no ``__array_interface__`` attribute instead all of the keys
+   (except for version) in the ``__array_interface__`` dictionary were
    their own attribute: Thus to obtain the Python-side information you
    had to access separately the attributes:
 
-   * __array_data__
-   * __array_shape__
-   * __array_strides__
-   * __array_typestr__
-   * __array_descr__
-   * __array_offset__
-   * __array_mask__
+   * ``__array_data__``
+   * ``__array_shape__``
+   * ``__array_strides__``
+   * ``__array_typestr__``
+   * ``__array_descr__``
+   * ``__array_offset__``
+   * ``__array_mask__``