diff options
author | Pauli Virtanen <pav@iki.fi> | 2013-10-01 22:30:09 +0300 |
---|---|---|
committer | Pauli Virtanen <pav@iki.fi> | 2013-10-19 23:23:49 +0300 |
commit | fd5d3088ed71bbc2fe5a774178be5e0ba04e4cd1 (patch) | |
tree | 9783211ff93f6ec17a30382a8aae3818ec059eb5 /doc | |
parent | 18acfa462a63bcdaf86360f0c94bc9347ecafad5 (diff) | |
download | numpy-fd5d3088ed71bbc2fe5a774178be5e0ba04e4cd1.tar.gz |
BUG: core: ensure __r*__ has precedence over __numpy_ufunc__
Add a special case to the implementation of ndarray.__mul__ et al. that
refuses to work on other objects that are not ndarray subclasses and
implement both __numpy_ufunc__ and __r*__.
This way, execution passes first to the custom __r*__ method, which
makes it possible to have e.g. __mul__ and np.multiply do different
things.
Additionally, disable one __array_priority__ special case handling when
also __numpy_ufunc__ is defined.
Diffstat (limited to 'doc')
-rw-r--r-- | doc/neps/ufunc-overrides.rst | 82 | ||||
-rw-r--r-- | doc/source/reference/arrays.classes.rst | 29 |
2 files changed, 111 insertions, 0 deletions
diff --git a/doc/neps/ufunc-overrides.rst b/doc/neps/ufunc-overrides.rst index 1c0ab1c78..f57e77054 100644 --- a/doc/neps/ufunc-overrides.rst +++ b/doc/neps/ufunc-overrides.rst @@ -158,6 +158,88 @@ If none of the input arguments has a ``__numpy_ufunc__`` method, the execution falls back on the default ufunc behaviour. +In combination with Python's binary operations +---------------------------------------------- + +The ``__numpy_ufunc__`` mechanism is fully independent of Python's +standard operator override mechanism, and the two do not interact +directly. + +They however have indirect interactions, because Numpy's ``ndarray`` +type implements its binary operations via Ufuncs. Effectively, we have:: + + class ndarray(object): + ... + def __mul__(self, other): + return np.multiply(self, other) + +Suppose now we have a second class:: + + class MyObject(object): + def __numpy_ufunc__(self, *a, **kw): + return "ufunc" + def __mul__(self, other): + return 1234 + def __rmul__(self, other): + return 4321 + +In this case, standard Python override rules combined with the above +discussion imply:: + + a = MyObject() + b = np.array([0]) + + a * b # == 1234 OK + b * a # == "ufunc" surprising + +This is not what would be naively expected, and is therefore somewhat +undesirable behavior. + +The reason why this occurs is: because ``MyObject`` is not an ndarray +subclass, Python resolves the expression ``b * a`` by calling first +``b.__mul__``. Since Numpy implements this via an Ufunc, the call is +forwarded to ``__numpy_ufunc__`` and not to ``__rmul__``. Note that if +``MyObject`` is a subclass of ``ndarray``, Python calls ``a.__rmul__`` +first. The issue is therefore that ``__numpy_ufunc__`` implements +"virtual subclassing" of ndarray behavior, without actual subclassing. + +This issue can be resolved by a modification of the binary operation +methods in Numpy:: + + class ndarray(object): + ... + def __mul__(self, other): + if (not isinstance(other, self.__class__) + and hasattr(other, '__numpy_ufunc__') + and hasattr(other, '__rmul__')): + return NotImplemented + return np.multiply(self, other) + + def __imul__(self, other): + if (other.__class__ is not self.__class__ + and hasattr(other, '__numpy_ufunc__') + and hasattr(other, '__rmul__')): + return NotImplemented + return np.multiply(self, other, out=self) + + b * a # == 4321 OK + +The rationale here is the following: since the user class explicitly +defines both ``__numpy_ufunc__`` and ``__rmul__``, the implementor has +very likely made sure that the ``__rmul__`` method can process ndarrays. +If not, the special case is simple to deal with (just call +``np.multiply``). + +The exclusion of subclasses of self can be made because Python itself +calls the right-hand method first in this case. Moreover, it is +desirable that ndarray subclasses are able to inherit the right-hand +binary operation methods from ndarray. + +The same priority shuffling needs to be done also for the in-place +operations, so that ``MyObject.__rmul__`` is prioritized over +``ndarray.__imul__``. + + Demo ==== diff --git a/doc/source/reference/arrays.classes.rst b/doc/source/reference/arrays.classes.rst index 48c0e1759..036185782 100644 --- a/doc/source/reference/arrays.classes.rst +++ b/doc/source/reference/arrays.classes.rst @@ -80,6 +80,35 @@ Numpy provides several hooks that classes can customize: overrides the behavior of :func:`numpy.dot` even though it is not an Ufunc. + .. note:: If you also define right-hand binary operator override + methods (such as ``__rmul__``) or comparison operations (such as + ``__gt__``) in your class, they take precedence over the + :func:`__numpy_ufunc__` mechanism when resolving results of + binary operations (such as ``ndarray_obj * your_obj``). + + The technical special case is: ``ndarray.__mul__`` returns + ``NotImplemented`` if the other object is *not* a subclass of + :class:`ndarray`, and defines both ``__numpy_ufunc__`` and + ``__rmul__``. Similar exception applies for the other operations + than multiplication. + + In such a case, when computing a binary operation such as + ``ndarray_obj * your_obj``, your ``__numpy_ufunc__`` method + *will not* be called. Instead, the execution passes on to your + right-hand ``__rmul__`` operation, as per standard Python + operator override rules. + + Similar special case applies to *in-place operations*: If you + define ``__rmul__``, then ``ndarray_obj *= your_obj`` *will not* + call your ``__numpy_ufunc__`` implementation. Instead, the + default Python behavior ``ndarray_obj = ndarray_obj * your_obj`` + occurs. + + Note that the above discussion applies only to Python's builtin + binary operation mechanism. ``np.multiply(ndarray_obj, + your_obj)`` always calls only your ``__numpy_ufunc__``, as + expected. + .. function:: class.__array_finalize__(self) This method is called whenever the system internally allocates a |