diff options
author | Marten van Kerkwijk <mhvk@astro.utoronto.ca> | 2017-04-02 14:00:34 -0400 |
---|---|---|
committer | Charles Harris <charlesr.harris@gmail.com> | 2017-04-27 13:37:50 -0600 |
commit | 6b41d110b092ae32252253b4d0a54d40b5628b93 (patch) | |
tree | 488b1a396bba1e32e7ef41abc0193e068552e1eb | |
parent | 39c2273e03070e8682f91852667286fe6f7d436b (diff) | |
download | numpy-6b41d110b092ae32252253b4d0a54d40b5628b93.tar.gz |
DOC: clarify use of super and getattr
-rw-r--r-- | doc/source/reference/arrays.classes.rst | 84 | ||||
-rw-r--r-- | numpy/doc/subclassing.py | 47 |
2 files changed, 98 insertions, 33 deletions
diff --git a/doc/source/reference/arrays.classes.rst b/doc/source/reference/arrays.classes.rst index 387ac2de1..1ece99af6 100644 --- a/doc/source/reference/arrays.classes.rst +++ b/doc/source/reference/arrays.classes.rst @@ -64,14 +64,14 @@ NumPy provides several hooks that classes can customize: The method should return either the result of the operation, or :obj:`NotImplemented` if the operation requested is not implemented. - If one of the arguments has a :func:`__array_ufunc__` method, it is - executed *instead* of the ufunc. If more than one of the input + If one of the input or output arguments has a :func:`__array_ufunc__` + method, it is executed *instead* of the ufunc. If more than one of the arguments implements :func:`__array_ufunc__`, they are tried in the - order: subclasses before superclasses, otherwise left to right. The - first routine returning something other than :obj:`NotImplemented` - determines the result. If all of the :func:`__array_ufunc__` - operations return :obj:`NotImplemented`, a :exc:`TypeError` is - raised. + order: subclasses before superclasses, inputs before outputs, otherwise + left to right. The first routine returning something other than + :obj:`NotImplemented` determines the result. If all of the + :func:`__array_ufunc__` operations return :obj:`NotImplemented`, a + :exc:`TypeError` is raised. .. note:: In addition to ufuncs, :func:`__array_ufunc__` also overrides the behavior of :func:`numpy.dot` and :func:`numpy.matmul`. @@ -80,7 +80,7 @@ NumPy provides several hooks that classes can customize: (which are overridden). We intend to extend this behaviour to other relevant functions. - Like with other methods in python, such as ``__hash__`` and + Like with some other special methods in python, such as ``__hash__`` and ``__iter__``, it is possible to indicate that your class does *not* support ufuncs by setting ``__array_ufunc__ = None``. With this, inside ufuncs, your class will be treated as if it returned @@ -99,20 +99,60 @@ NumPy provides several hooks that classes can customize: :class:`ndarray` will unconditionally return :obj:`NotImplemented`, so that your reverse methods will get called. - .. note:: If you subclass :class:`ndarray`: - - - We strongly recommend that you avoid confusion by neither setting - :func:`__array_ufunc__` to :obj:`None`, which makes no sense for - an array subclass, nor by defining it and also defining reverse - methods, which methods will be called by ``CPython`` in - preference over the :class:`ndarray` forward methods. - - :class:`ndarray` defines its own :func:`__array_ufunc__`, which - corresponds to ``getattr(ufunc, method)(*inputs, **kwargs)``. Hence, - a typical override of :func:`__array_ufunc__` would convert any - instances of one's own class, pass these on to its superclass using - ``super().__array_ufunc__(*inputs, **kwargs)``, and finally return - the results after possible back-conversion. This practice ensures - that it is possible to have a hierarchy of subclasses. See + The presence of :func:`__array_ufunc__` also influences how + :class:`ndarray` handles binary operations like ``arr + obj`` and ``arr + < obj`` when ``arr`` is an :class:`ndarray` and ``obj`` is an instance + of a custom class. There are two possibilities. If + ``obj.__array_ufunc__`` is present and not :obj:`None`, then + ``ndarray.__add__`` and friends will delegate to the ufunc machinery, + meaning that ``arr + obj`` becomes ``np.add(arr, obj)``, and then + :func:`~numpy.add` invokes ``obj.__array_ufunc__``. This is useful if you + want to define an object that acts like an array. + + Alternatively, if ``obj.__array_ufunc__`` is set to :obj:`None`, then as a + special case, special methods like ``ndarray.__add__`` will notice this + and *unconditionally* return :obj:`NotImplemented`, so that Python will + dispatch to ``obj.__radd__`` instead. This is useful if you want to define + a special object that interacts with arrays via binary operations, but + is not itself an array. For example, a units handling system might have + an object ``m`` representing the "meters" unit, and want to support the + syntax ``arr * m`` to represent that the array has units of "meters", but + not want to otherwise interact with arrays via ufuncs or otherwise. This + can be done by setting ``__array_ufunc__ = None`` and defining ``__mul__`` + and ``__rmul__`` methods. (Note that this means that writing an + ``__array_ufunc__`` that always returns :obj:`NotImplemented` is not + quite the same as setting ``__array_ufunc__ = None``: in the former + case, ``arr + obj`` will raise :exc:`TypeError`, while in the latter + case it is possible to define a ``__radd__`` method to prevent this.) + + The above does not hold for in-place operators, for which :class:`ndarray` + never returns :obj:`NotImplemented`. Hence, ``arr += obj`` would always + lead to a :exc:`TypeError`. This is because for arrays in-place operations + cannot generically be replaced by a simple reverse operation. (For + instance, by default, ``arr[:] += obj`` would be translated to ``arr[:] = + arr[:] + obj``, which would likely be wrong.) + + .. note:: If you define ``__array_ufunc__``: + + - If you are not a subclass of :class:`ndarray`, we recommend your + class define special methods like ``__add__`` and ``__lt__`` that + delegate to ufuncs just like ndarray does. We hope to provide a + helper mixin class for this. + - If you subclass :class:`ndarray`, we strongly recommend that you + avoid confusion by neither setting :func:`__array_ufunc__` to + :obj:`None`, which makes no sense for an array subclass, nor by + defining it and also defining reverse methods, which methods will + be called by ``CPython`` in preference over the :class:`ndarray` + forward methods. + - :class:`ndarray` defines its own :func:`__array_ufunc__`, which, + evaluates the ufunc if no arguments have overrides, and returns + :obj:`NotImplemented` otherwise. This may be useful for subclasses + for which :func:`__array_ufunc__` converts any instances of its own + class to :class:`ndarray`: it can then pass these on to its + superclass using ``super().__array_ufunc__(*inputs, **kwargs)``, + and finally return the results after possible back-conversion. The + advantage of this practice is that it ensures that it is possible + to have a hierarchy of subclasses that extend the behaviour. See :ref:`Subclassing ndarray <basics.subclassing>` for details. .. note:: If a class defines the :func:`__array_ufunc__` method, diff --git a/numpy/doc/subclassing.py b/numpy/doc/subclassing.py index 3e16ae870..c42d5e330 100644 --- a/numpy/doc/subclassing.py +++ b/numpy/doc/subclassing.py @@ -480,6 +480,9 @@ following. results = super(A, self).__array_ufunc__(ufunc, method, *args, **kwargs) + if results is NotImplemented: + return NotImplemented + if not isinstance(results, tuple): if not isinstance(results, np.ndarray): return results @@ -508,27 +511,49 @@ which inputs and outputs it converted. Hence, e.g., {'outputs': [0]} >>> a = np.arange(5.).view(A) >>> b = np.ones(1).view(A) +>>> c = a + b +>>> c.info +{'inputs': [0, 1]} >>> a += b >>> a.info {'inputs': [0, 1], 'outputs': [0]} -Note that one might also consider just doing ``getattr(ufunc, -methods)(*inputs, **kwargs)`` instead of the ``super`` call. This would -work (indeed, ``ndarray.__array_ufunc__`` effectively does just that), but -by using ``super`` one can more easily have a class hierarchy. E.g., -suppose we had another class ``B`` that defined ``__array_ufunc__`` and -then made a subclass ``C`` depending on both, i.e., ``class C(A, B)`` -without yet another ``__array_ufunc__`` override. Then any ufunc on an -instance of ``C`` would pass on to ``A.__array_ufunc__``, the ``super`` -call in ``A`` would go to ``B.__array_ufunc__``, and the ``super`` call in -``B`` would go to ``ndarray.__array_ufunc__``. +Note that another approach would be to to use ``getattr(ufunc, +methods)(*inputs, **kwargs)`` instead of the ``super`` call. For this example, +the result would be identical, but there is a difference if another operand +also defines ``__array_ufunc__``. E.g., lets assume that we evalulate +``np.add(a, b)``, where ``b`` is an instance of another class ``B`` that has +an override. If you use ``super`` as in the example, +``ndarray.__array_ufunc__`` will notice that ``b`` has an override, which +means it cannot evaluate the result itself. Thus, it will return +`NotImplemented` and so will our class ``A``. Then, control will be passed +over to ``b``, which either knows how to deal with us and produces a result, +or does not and returns `NotImplemented`, raising a ``TypeError``. + +If instead, we replace our ``super`` call with ``getattr(ufunc, method)``, we +effectively do ``np.add(a.view(np.ndarray), b)``. Again, ``B.__array_ufunc__`` +will be called, but now it sees an ``ndarray`` as the other argument. Likely, +it will know how to handle this, and return a new instance of the ``B`` class +to us. Our example class is not set up to handle this, but it might well be +the best approach if, e.g., one were to re-implement ``MaskedArray`` using + ``__array_ufunc__``. + +As a final note: if the ``super`` route is suited to a given class, an +advantage of using it is that it helps in constructing class hierarchies. +E.g., suppose that our other class ``B`` also used the ``super`` in its +``__array_ufunc__`` implementation, and we created a class ``C`` that depended +on both, i.e., ``class C(A, B)`` (with, for simplicity, not another +``__array_ufunc__`` override). Then any ufunc on an instance of ``C`` would +pass on to ``A.__array_ufunc__``, the ``super`` call in ``A`` would go to +``B.__array_ufunc__``, and the ``super`` call in ``B`` would go to +``ndarray.__array_ufunc__``, thus allowing ``A`` and ``B`` to collaborate. .. _array-wrap: ``__array_wrap__`` for ufuncs and other functions ------------------------------------------------- -Prior to numpy 1.13, the behaviour of ufuncs could be tuned using +Prior to numpy 1.13, the behaviour of ufuncs could only be tuned using ``__array_wrap__`` and ``__array_prepare__``. These two allowed one to change the output type of a ufunc, but, in constrast to ``__array_ufunc__``, did not allow one to make any changes to the inputs. |