summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorPauli Virtanen <pav@iki.fi>2013-10-01 22:30:09 +0300
committerPauli Virtanen <pav@iki.fi>2013-10-19 23:23:49 +0300
commitfd5d3088ed71bbc2fe5a774178be5e0ba04e4cd1 (patch)
tree9783211ff93f6ec17a30382a8aae3818ec059eb5 /doc
parent18acfa462a63bcdaf86360f0c94bc9347ecafad5 (diff)
downloadnumpy-fd5d3088ed71bbc2fe5a774178be5e0ba04e4cd1.tar.gz
BUG: core: ensure __r*__ has precedence over __numpy_ufunc__
Add a special case to the implementation of ndarray.__mul__ et al. that refuses to work on other objects that are not ndarray subclasses and implement both __numpy_ufunc__ and __r*__. This way, execution passes first to the custom __r*__ method, which makes it possible to have e.g. __mul__ and np.multiply do different things. Additionally, disable one __array_priority__ special case handling when also __numpy_ufunc__ is defined.
Diffstat (limited to 'doc')
-rw-r--r--doc/neps/ufunc-overrides.rst82
-rw-r--r--doc/source/reference/arrays.classes.rst29
2 files changed, 111 insertions, 0 deletions
diff --git a/doc/neps/ufunc-overrides.rst b/doc/neps/ufunc-overrides.rst
index 1c0ab1c78..f57e77054 100644
--- a/doc/neps/ufunc-overrides.rst
+++ b/doc/neps/ufunc-overrides.rst
@@ -158,6 +158,88 @@ If none of the input arguments has a ``__numpy_ufunc__`` method, the
execution falls back on the default ufunc behaviour.
+In combination with Python's binary operations
+----------------------------------------------
+
+The ``__numpy_ufunc__`` mechanism is fully independent of Python's
+standard operator override mechanism, and the two do not interact
+directly.
+
+They however have indirect interactions, because Numpy's ``ndarray``
+type implements its binary operations via Ufuncs. Effectively, we have::
+
+ class ndarray(object):
+ ...
+ def __mul__(self, other):
+ return np.multiply(self, other)
+
+Suppose now we have a second class::
+
+ class MyObject(object):
+ def __numpy_ufunc__(self, *a, **kw):
+ return "ufunc"
+ def __mul__(self, other):
+ return 1234
+ def __rmul__(self, other):
+ return 4321
+
+In this case, standard Python override rules combined with the above
+discussion imply::
+
+ a = MyObject()
+ b = np.array([0])
+
+ a * b # == 1234 OK
+ b * a # == "ufunc" surprising
+
+This is not what would be naively expected, and is therefore somewhat
+undesirable behavior.
+
+The reason why this occurs is: because ``MyObject`` is not an ndarray
+subclass, Python resolves the expression ``b * a`` by calling first
+``b.__mul__``. Since Numpy implements this via an Ufunc, the call is
+forwarded to ``__numpy_ufunc__`` and not to ``__rmul__``. Note that if
+``MyObject`` is a subclass of ``ndarray``, Python calls ``a.__rmul__``
+first. The issue is therefore that ``__numpy_ufunc__`` implements
+"virtual subclassing" of ndarray behavior, without actual subclassing.
+
+This issue can be resolved by a modification of the binary operation
+methods in Numpy::
+
+ class ndarray(object):
+ ...
+ def __mul__(self, other):
+ if (not isinstance(other, self.__class__)
+ and hasattr(other, '__numpy_ufunc__')
+ and hasattr(other, '__rmul__')):
+ return NotImplemented
+ return np.multiply(self, other)
+
+ def __imul__(self, other):
+ if (other.__class__ is not self.__class__
+ and hasattr(other, '__numpy_ufunc__')
+ and hasattr(other, '__rmul__')):
+ return NotImplemented
+ return np.multiply(self, other, out=self)
+
+ b * a # == 4321 OK
+
+The rationale here is the following: since the user class explicitly
+defines both ``__numpy_ufunc__`` and ``__rmul__``, the implementor has
+very likely made sure that the ``__rmul__`` method can process ndarrays.
+If not, the special case is simple to deal with (just call
+``np.multiply``).
+
+The exclusion of subclasses of self can be made because Python itself
+calls the right-hand method first in this case. Moreover, it is
+desirable that ndarray subclasses are able to inherit the right-hand
+binary operation methods from ndarray.
+
+The same priority shuffling needs to be done also for the in-place
+operations, so that ``MyObject.__rmul__`` is prioritized over
+``ndarray.__imul__``.
+
+
Demo
====
diff --git a/doc/source/reference/arrays.classes.rst b/doc/source/reference/arrays.classes.rst
index 48c0e1759..036185782 100644
--- a/doc/source/reference/arrays.classes.rst
+++ b/doc/source/reference/arrays.classes.rst
@@ -80,6 +80,35 @@ Numpy provides several hooks that classes can customize:
overrides the behavior of :func:`numpy.dot` even though it is
not an Ufunc.
+ .. note:: If you also define right-hand binary operator override
+ methods (such as ``__rmul__``) or comparison operations (such as
+ ``__gt__``) in your class, they take precedence over the
+ :func:`__numpy_ufunc__` mechanism when resolving results of
+ binary operations (such as ``ndarray_obj * your_obj``).
+
+ The technical special case is: ``ndarray.__mul__`` returns
+ ``NotImplemented`` if the other object is *not* a subclass of
+ :class:`ndarray`, and defines both ``__numpy_ufunc__`` and
+ ``__rmul__``. Similar exception applies for the other operations
+ than multiplication.
+
+ In such a case, when computing a binary operation such as
+ ``ndarray_obj * your_obj``, your ``__numpy_ufunc__`` method
+ *will not* be called. Instead, the execution passes on to your
+ right-hand ``__rmul__`` operation, as per standard Python
+ operator override rules.
+
+ Similar special case applies to *in-place operations*: If you
+ define ``__rmul__``, then ``ndarray_obj *= your_obj`` *will not*
+ call your ``__numpy_ufunc__`` implementation. Instead, the
+ default Python behavior ``ndarray_obj = ndarray_obj * your_obj``
+ occurs.
+
+ Note that the above discussion applies only to Python's builtin
+ binary operation mechanism. ``np.multiply(ndarray_obj,
+ your_obj)`` always calls only your ``__numpy_ufunc__``, as
+ expected.
+
.. function:: class.__array_finalize__(self)
This method is called whenever the system internally allocates a