diff options
| -rw-r--r-- | doc/CAPI.rst.txt | 320 | ||||
| -rw-r--r-- | doc/source/dev/conduct/report_handling_manual.rst | 2 | ||||
| -rw-r--r-- | doc/source/dev/index.rst | 1 | ||||
| -rw-r--r-- | doc/source/dev/style_guide.rst | 8 | ||||
| -rw-r--r-- | doc/source/reference/alignment.rst (renamed from doc/source/dev/alignment.rst) | 8 | ||||
| -rw-r--r-- | doc/source/reference/c-api.array.rst | 26 | ||||
| -rw-r--r-- | doc/source/reference/c-api.dtype.rst | 2 | ||||
| -rw-r--r-- | doc/source/reference/distutils.rst | 3 | ||||
| -rw-r--r-- | doc/source/reference/distutils_guide.rst | 7 | ||||
| -rw-r--r-- | doc/source/reference/index.rst | 3 | ||||
| -rw-r--r-- | doc/source/reference/internals.rst | 1 |
11 files changed, 46 insertions, 335 deletions
diff --git a/doc/CAPI.rst.txt b/doc/CAPI.rst.txt deleted file mode 100644 index ccee0fdb6..000000000 --- a/doc/CAPI.rst.txt +++ /dev/null @@ -1,320 +0,0 @@ -=============== -C-API for NumPy -=============== - -:Author: Travis Oliphant -:Discussions to: `numpy-discussion@python.org`__ -:Created: October 2005 - -__ https://scipy.org/scipylib/mailing-lists.html - -The C API of NumPy is (mostly) backward compatible with Numeric. - -There are a few non-standard Numeric usages (that were not really part -of the API) that will need to be changed: - -* If you used any of the function pointers in the ``PyArray_Descr`` - structure you will have to modify your usage of those. First, - the pointers are all under the member named ``f``. So ``descr->cast`` - is now ``descr->f->cast``. In addition, the - casting functions have eliminated the strides argument (use - ``PyArray_CastTo`` if you need strided casting). All functions have - one or two ``PyArrayObject *`` arguments at the end. This allows the - flexible arrays and mis-behaved arrays to be handled. - -* The ``descr->zero`` and ``descr->one`` constants have been replaced with - function calls, ``PyArray_Zero``, and ``PyArray_One`` (be sure to read the - code and free the resulting memory if you use these calls). - -* If you passed ``array->dimensions`` and ``array->strides`` around - to functions, you will need to fix some code. These are now - ``npy_intp*`` pointers. On 32-bit systems there won't be a problem. - However, on 64-bit systems, you will need to make changes to avoid - errors and segfaults. - - -The header files ``arrayobject.h`` and ``ufuncobject.h`` contain many defines -that you may find useful. The files ``__ufunc_api.h`` and -``__multiarray_api.h`` contain the available C-API function calls with -their function signatures. - -All of these headers are installed to -``<YOUR_PYTHON_LOCATION>/site-packages/numpy/core/include`` - - -Getting arrays in C-code -========================= - -All new arrays can be created using ``PyArray_NewFromDescr``. A simple interface -equivalent to ``PyArray_FromDims`` is ``PyArray_SimpleNew(nd, dims, typenum)`` -and to ``PyArray_FromDimsAndData`` is -``PyArray_SimpleNewFromData(nd, dims, typenum, data)``. - -This is a very flexible function. - -:: - - PyObject * PyArray_NewFromDescr(PyTypeObject *subtype, PyArray_Descr *descr, - int nd, npy_intp *dims, - npy_intp *strides, char *data, - int flags, PyObject *obj); - -``subtype`` : ``PyTypeObject *`` - The subtype that should be created (either pass in - ``&PyArray_Type``, or ``obj->ob_type``, - where ``obj`` is an instance of a subtype (or subclass) of - ``PyArray_Type``). - -``descr`` : ``PyArray_Descr *`` - The type descriptor for the array. This is a Python object (this - function steals a reference to it). The easiest way to get one is - using ``PyArray_DescrFromType(<typenum>)``. If you want to use a - flexible size array, then you need to use - ``PyArray_DescrNewFromType(<flexible typenum>)`` and set its ``elsize`` - parameter to the desired size. The typenum in both of these cases - is one of the ``PyArray_XXXX`` enumerated types. - -``nd`` : ``int`` - The number of dimensions (<``MAX_DIMS``) - -``*dims`` : ``npy_intp *`` - A pointer to the size in each dimension. Information will be - copied from here. - -``*strides`` : ``npy_intp *`` - The strides this array should have. For new arrays created by this - routine, this should be ``NULL``. If you pass in memory for this array - to use, then you can pass in the strides information as well - (otherwise it will be created for you and default to C-contiguous - or Fortran contiguous). Any strides will be copied into the array - structure. Do not pass in bad strides information!!!! - - ``PyArray_CheckStrides(...)`` can help but you must call it if you are - unsure. You cannot pass in strides information when data is ``NULL`` - and this routine is creating its own memory. - -``*data`` : ``char *`` - ``NULL`` for creating brand-new memory. If you want this array to wrap - another memory area, then pass the pointer here. You are - responsible for deleting the memory in that case, but do not do so - until the new array object has been deleted. The best way to - handle that is to get the memory from another Python object, - ``INCREF`` that Python object after passing it's data pointer to this - routine, and set the ``->base`` member of the returned array to the - Python object. *You are responsible for* setting ``PyArray_BASE(ret)`` - to the base object. Failure to do so will create a memory leak. - - If you pass in a data buffer, the ``flags`` argument will be the flags - of the new array. If you create a new array, a non-zero flags - argument indicates that you want the array to be in Fortran order. - -``flags`` : ``int`` - Either the flags showing how to interpret the data buffer passed - in, or if a new array is created, nonzero to indicate a Fortran - order array. See below for an explanation of the flags. - -``obj`` : ``PyObject *`` - If subtypes is ``&PyArray_Type``, this argument is - ignored. Otherwise, the ``__array_finalize__`` method of the subtype - is called (if present) and passed this object. This is usually an - array of the type to be created (so the ``__array_finalize__`` method - must handle an array argument. But, it can be anything...) - -Note: The returned array object will be uninitialized unless the type is -``PyArray_OBJECT`` in which case the memory will be set to ``NULL``. - -``PyArray_SimpleNew(nd, dims, typenum)`` is a drop-in replacement for -``PyArray_FromDims`` (except it takes ``npy_intp*`` dims instead of ``int*`` dims -which matters on 64-bit systems) and it does not initialize the memory -to zero. - -``PyArray_SimpleNew`` is just a macro for ``PyArray_New`` with default arguments. -Use ``PyArray_FILLWBYTE(arr, 0)`` to fill with zeros. - -The ``PyArray_FromDims`` and family of functions are still available and -are loose wrappers around this function. These functions still take -``int *`` arguments. This should be fine on 32-bit systems, but on 64-bit -systems you may run into trouble if you frequently passed -``PyArray_FromDims`` the dimensions member of the old ``PyArrayObject`` structure -because ``sizeof(npy_intp) != sizeof(int)``. - - -Getting an arrayobject from an arbitrary Python object -====================================================== - -``PyArray_FromAny(...)`` - -This function replaces ``PyArray_ContiguousFromObject`` and friends (those -function calls still remain but they are loose wrappers around the -``PyArray_FromAny`` call). - -:: - - static PyObject * - PyArray_FromAny(PyObject *op, PyArray_Descr *dtype, int min_depth, - int max_depth, int requires, PyObject *context) - - -``op`` : ``PyObject *`` - The Python object to "convert" to an array object - -``dtype`` : ``PyArray_Descr *`` - The desired data-type descriptor. This can be ``NULL``, if the - descriptor should be determined by the object. Unless ``FORCECAST`` is - present in ``flags``, this call will generate an error if the data - type cannot be safely obtained from the object. - -``min_depth`` : ``int`` - The minimum depth of array needed or 0 if doesn't matter - -``max_depth`` : ``int`` - The maximum depth of array allowed or 0 if doesn't matter - -``requires`` : ``int`` - A flag indicating the "requirements" of the returned array. These - are the usual ndarray flags (see `NDArray flags`_ below). In - addition, there are three flags used only for the ``FromAny`` - family of functions: - - - ``ENSURECOPY``: always copy the array. Returned arrays always - have ``CONTIGUOUS``, ``ALIGNED``, and ``WRITEABLE`` set. - - ``ENSUREARRAY``: ensure the returned array is an ndarray. - - ``FORCECAST``: cause a cast to occur regardless of whether or - not it is safe. - -``context`` : ``PyObject *`` - If the Python object ``op`` is not a numpy array, but has an - ``__array__`` method, context is passed as the second argument to - that method (the first is the typecode). Almost always this - parameter is ``NULL``. - - -``PyArray_ContiguousFromAny(op, typenum, min_depth, max_depth)`` is -equivalent to ``PyArray_ContiguousFromObject(...)`` (which is still -available), except it will return the subclass if op is already a -subclass of the ndarray. The ``ContiguousFromObject`` version will -always return an ndarray. - -Passing Data Type information to C-code -======================================= - -All datatypes are handled using the ``PyArray_Descr *`` structure. -This structure can be obtained from a Python object using -``PyArray_DescrConverter`` and ``PyArray_DescrConverter2``. The former -returns the default ``PyArray_LONG`` descriptor when the input object -is None, while the latter returns ``NULL`` when the input object is ``None``. - -See the ``arraymethods.c`` and ``multiarraymodule.c`` files for many -examples of usage. - -Getting at the structure of the array. --------------------------------------- - -You should use the ``#defines`` provided to access array structure portions: - -- ``PyArray_DATA(obj)`` : returns a ``void *`` to the array data -- ``PyArray_BYTES(obj)`` : return a ``char *`` to the array data -- ``PyArray_ITEMSIZE(obj)`` -- ``PyArray_NDIM(obj)`` -- ``PyArray_DIMS(obj)`` -- ``PyArray_DIM(obj, n)`` -- ``PyArray_STRIDES(obj)`` -- ``PyArray_STRIDE(obj,n)`` -- ``PyArray_DESCR(obj)`` -- ``PyArray_BASE(obj)`` - -see more in ``arrayobject.h`` - - -NDArray Flags -============= - -The ``flags`` attribute of the ``PyArrayObject`` structure contains important -information about the memory used by the array (pointed to by the data member) -This flags information must be kept accurate or strange results and even -segfaults may result. - -There are 6 (binary) flags that describe the memory area used by the -data buffer. These constants are defined in ``arrayobject.h`` and -determine the bit-position of the flag. Python exposes a nice attribute- -based interface as well as a dictionary-like interface for getting -(and, if appropriate, setting) these flags. - -Memory areas of all kinds can be pointed to by an ndarray, necessitating -these flags. If you get an arbitrary ``PyArrayObject`` in C-code, -you need to be aware of the flags that are set. -If you need to guarantee a certain kind of array -(like ``NPY_CONTIGUOUS`` and ``NPY_BEHAVED``), then pass these requirements into the -PyArray_FromAny function. - - -``NPY_CONTIGUOUS`` - True if the array is (C-style) contiguous in memory. -``NPY_FORTRAN`` - True if the array is (Fortran-style) contiguous in memory. - -Notice that contiguous 1-d arrays are always both ``NPY_FORTRAN`` contiguous -and C contiguous. Both of these flags can be checked and are convenience -flags only as whether or not an array is ``NPY_CONTIGUOUS`` or ``NPY_FORTRAN`` -can be determined by the ``strides``, ``dimensions``, and ``itemsize`` -attributes. - -``NPY_OWNDATA`` - True if the array owns the memory (it will try and free it using - ``PyDataMem_FREE()`` on deallocation --- so it better really own it). - -These three flags facilitate using a data pointer that is a memory-mapped -array, or part of some larger record array. But, they may have other uses... - -``NPY_ALIGNED`` - True if the data buffer is aligned for the type and the strides - are multiples of the alignment factor as well. This can be - checked. - -``NPY_WRITEABLE`` - True only if the data buffer can be "written" to. - -``NPY_WRITEBACKIFCOPY`` - This is a special flag that is set if this array represents a copy - made because a user required certain flags in ``PyArray_FromAny`` and - a copy had to be made of some other array (and the user asked for - this flag to be set in such a situation). The base attribute then - points to the "misbehaved" array (which is set read_only). If you use - this flag, you are must call ``PyArray_ResolveWritebackIfCopy`` before - deallocating this array (i.e. before calling ``Py_DECREF`` the last time) - which will write the data contents back to the "misbehaved" array (casting - if necessary) and will reset the "misbehaved" array to ``WRITEABLE``. If - the "misbehaved" array was not ``WRITEABLE`` to begin with then - ``PyArray_FromAny`` would have returned an error because ``WRITEBACKIFCOPY`` - would not have been possible. In error conditions, call - ``PyArray_DiscardWritebackIfCopy`` to throw away the scratch buffer, then - ``Py_DECREF`` or ``Py_XDECREF``. - -``NPY_UPDATEIFCOPY`` - Similar to ``NPY_WRITEBACKIFCOPY``, but deprecated since it copied the - contents back when the array is deallocated, which is not explicit and - relies on refcount semantics. Refcount semantics are unreliable on - alternative implementations of python such as PyPy. - -``PyArray_UpdateFlags(obj, flags)`` will update the ``obj->flags`` for -``flags`` which can be any of ``NPY_CONTIGUOUS``, ``NPY_FORTRAN``, ``NPY_ALIGNED``, or -``NPY_WRITEABLE``. - -Some useful combinations of these flags: - -- ``NPY_BEHAVED = NPY_ALIGNED | NPY_WRITEABLE`` -- ``NPY_CARRAY = NPY_DEFAULT = NPY_CONTIGUOUS | NPY_BEHAVED`` -- ``NPY_CARRAY_RO = NPY_CONTIGUOUS | NPY_ALIGNED`` -- ``NPY_FARRAY = NPY_FORTRAN | NPY_BEHAVED`` -- ``NPY_FARRAY_RO = NPY_FORTRAN | NPY_ALIGNED`` - -The macro ``PyArray_CHECKFLAGS(obj, flags)`` can test any combination of flags. -There are several default combinations defined as macros already -(see ``arrayobject.h``) - -In particular, there are ``ISBEHAVED``, ``ISBEHAVED_RO``, ``ISCARRAY`` -and ``ISFARRAY`` macros that also check to make sure the array is in -native byte order (as determined) by the data-type descriptor. - -There are more C-API enhancements which you can discover in the code, -or buy the book (http://www.trelgol.com) diff --git a/doc/source/dev/conduct/report_handling_manual.rst b/doc/source/dev/conduct/report_handling_manual.rst index 5f5e32f13..d39b615bb 100644 --- a/doc/source/dev/conduct/report_handling_manual.rst +++ b/doc/source/dev/conduct/report_handling_manual.rst @@ -1,3 +1,5 @@ +:orphan: + .. _CoC_reporting_manual: NumPy Code of Conduct - How to follow up on a report diff --git a/doc/source/dev/index.rst b/doc/source/dev/index.rst index 9ce04cc1b..825b93b53 100644 --- a/doc/source/dev/index.rst +++ b/doc/source/dev/index.rst @@ -8,6 +8,7 @@ Contributing to NumPy conduct/code_of_conduct gitwash/index development_environment + style_guide releasing governance/index diff --git a/doc/source/dev/style_guide.rst b/doc/source/dev/style_guide.rst new file mode 100644 index 000000000..bede3424d --- /dev/null +++ b/doc/source/dev/style_guide.rst @@ -0,0 +1,8 @@ +.. _style_guide: + +=================== +NumPy C Style Guide +=================== + +.. include:: ../../C_STYLE_GUIDE.rst.txt + :start-line: 4 diff --git a/doc/source/dev/alignment.rst b/doc/source/reference/alignment.rst index f067f0d03..c749972b4 100644 --- a/doc/source/dev/alignment.rst +++ b/doc/source/reference/alignment.rst @@ -1,8 +1,10 @@ .. _alignment: +Memory Alignment +================ Numpy Alignment Goals -===================== +--------------------- There are three use-cases related to memory alignment in numpy (as of 1.14): @@ -33,7 +35,7 @@ alignment of 4 and "uint" alignment of 8 (equal to the true alignment of ``uint64``). Variables in Numpy which control and describe alignment -======================================================= +------------------------------------------------------- There are 4 relevant uses of the word ``align`` used in numpy: @@ -60,7 +62,7 @@ There are 4 relevant uses of the word ``align`` used in numpy: an analagous way to how ``IsAligned`` checks for true-alignment. Consequences of alignment -========================= +------------------------- Here is how the variables above are used: diff --git a/doc/source/reference/c-api.array.rst b/doc/source/reference/c-api.array.rst index 9265b1a97..85814f5a9 100644 --- a/doc/source/reference/c-api.array.rst +++ b/doc/source/reference/c-api.array.rst @@ -202,7 +202,8 @@ From scratch PyTypeObject* subtype, PyArray_Descr* descr, int nd, npy_intp* dims, \ npy_intp* strides, void* data, int flags, PyObject* obj) - This function steals a reference to *descr*. + This function steals a reference to *descr*. The easiest way to get one + is using :c:func:`PyArray_DescrFromType`. This is the main array creation function. Most new arrays are created with this flexible function. @@ -216,9 +217,11 @@ From scratch :c:data:`&PyArray_Type<PyArray_Type>`, then *obj* is the object to pass to the :obj:`~numpy.class.__array_finalize__` method of the subclass. - If *data* is ``NULL``, then new memory will be allocated and *flags* - can be non-zero to indicate a Fortran-style contiguous array. If - *data* is not ``NULL``, then it is assumed to point to the memory + If *data* is ``NULL``, then new unitinialized memory will be allocated and + *flags* can be non-zero to indicate a Fortran-style contiguous array. Use + :c:ref:`PyArray_FILLWBYTE` to initialze the memory. + + If *data* is not ``NULL``, then it is assumed to point to the memory to be used for the array and the *flags* argument is used as the new flags for the array (except the state of :c:data:`NPY_OWNDATA`, :c:data:`NPY_ARRAY_WRITEBACKIFCOPY` and :c:data:`NPY_ARRAY_UPDATEIFCOPY` @@ -232,6 +235,12 @@ From scratch provided *dims* and *strides* are copied into newly allocated dimension and strides arrays for the new array object. + :c:func:`PyArray_CheckStrides` can help verify non- ``NULL`` stride + information. + + If ``data`` is provided, it must stay alive for the life of the array. One + way to manage this is through :c:func:`PyArray_SetBaseObject` + .. c:function:: PyObject* PyArray_NewLikeArray( \ PyArrayObject* prototype, NPY_ORDER order, PyArray_Descr* descr, \ int subok) @@ -407,10 +416,6 @@ From other objects the array is constructed that way. Almost always this parameter is ``NULL``. - In versions 1.6 and earlier of NumPy, the following flags - did not have the ``_ARRAY_`` macro namespace in them. That form - of the constant names is deprecated in 1.7. - .. c:var:: NPY_ARRAY_C_CONTIGUOUS Make sure the returned array is C-style contiguous @@ -2849,7 +2854,10 @@ Data-type descriptors Returns a data-type object corresponding to *typenum*. The *typenum* can be one of the enumerated types, a character code for - one of the enumerated types, or a user-defined type. + one of the enumerated types, or a user-defined type. If you want to use a + flexible size array, then you need to ``flexible typenum`` and set the + results ``elsize`` parameter to the desired size. The typenum is one of the + :c:data:`NPY_TYPES`. .. c:function:: int PyArray_DescrConverter(PyObject* obj, PyArray_Descr** dtype) diff --git a/doc/source/reference/c-api.dtype.rst b/doc/source/reference/c-api.dtype.rst index 8af3a9080..9ac46b284 100644 --- a/doc/source/reference/c-api.dtype.rst +++ b/doc/source/reference/c-api.dtype.rst @@ -25,6 +25,8 @@ select the precision desired. Enumerated Types ---------------- +.. c:var:: NPY_TYPES + There is a list of enumerated types defined providing the basic 24 data types plus some useful generic names. Whenever the code requires a type number, one of these enumerated types is requested. The types diff --git a/doc/source/reference/distutils.rst b/doc/source/reference/distutils.rst index 289822909..88e533832 100644 --- a/doc/source/reference/distutils.rst +++ b/doc/source/reference/distutils.rst @@ -13,8 +13,7 @@ distutils, use the :func:`setup <core.setup>` command from :mod:`numpy.distutils.misc_util` that can make it easier to construct keyword arguments to pass to the setup function (by passing the dictionary obtained from the todict() method of the class). More -information is available in the NumPy Distutils Users Guide in -``<site-packages>/numpy/doc/DISTUTILS.txt``. +information is available in the :ref:`distutils-user-guide`. .. index:: diff --git a/doc/source/reference/distutils_guide.rst b/doc/source/reference/distutils_guide.rst new file mode 100644 index 000000000..081719d16 --- /dev/null +++ b/doc/source/reference/distutils_guide.rst @@ -0,0 +1,7 @@ +.. _distutils-user-guide: + +NumPy Distutils - Users Guide +============================= + +.. include:: ../../DISTUTILS.rst.txt + :start-line: 6 diff --git a/doc/source/reference/index.rst b/doc/source/reference/index.rst index 2140c57f7..6accb8535 100644 --- a/doc/source/reference/index.rst +++ b/doc/source/reference/index.rst @@ -23,6 +23,7 @@ For learning how to use NumPy, see also :ref:`user`. ufuncs routines distutils + distutils_guide c-api internals swig @@ -35,4 +36,4 @@ Large parts of this manual originate from Travis E. Oliphant's book `Guide to NumPy <https://archive.org/details/NumPyBook>`__ (which generously entered Public Domain in August 2008). The reference documentation for many of the functions are written by numerous contributors and developers of -NumPy.
\ No newline at end of file +NumPy. diff --git a/doc/source/reference/internals.rst b/doc/source/reference/internals.rst index e1d6644a6..03d081bf9 100644 --- a/doc/source/reference/internals.rst +++ b/doc/source/reference/internals.rst @@ -5,5 +5,6 @@ NumPy internals .. toctree:: internals.code-explanations + alignment .. automodule:: numpy.doc.internals |