NEP: rewrite the backwards compatibility NEP contents

Also add examples of how deprecations and future warnings should
be done, taken from the current code base.

1 files changed, 180 insertions, 70 deletions

diff --git a/doc/neps/nep-0023-backwards-compatibility.rst b/doc/neps/nep-0023-backwards-compatibility.rst
index 0b58a5f4b..acfb379b7 100644
--- a/doc/neps/nep-0023-backwards-compatibility.rst
+++ b/doc/neps/nep-0023-backwards-compatibility.rst
@@ -19,44 +19,195 @@ processes for individual cases where breaking backwards compatibility
 is considered.
 
 
-Detailed description
+Motivation and Scope
 --------------------
 
 NumPy has a very large user base.  Those users rely on NumPy being stable
 and the code they write that uses NumPy functionality to keep working.
 NumPy is also actively maintained and improved -- and sometimes improvements
-require, or are made much easier, by breaking backwards compatibility.
+require, or are made much easier by, breaking backwards compatibility.
 Finally, there are trade-offs in stability for existing users vs. avoiding
 errors or having a better user experience for new users.  These competing
-needs often give rise to heated debates and delays in accepting or rejecting
+needs often give rise to long debates and to delays in accepting or rejecting
 contributions.  This NEP tries to address that by providing a policy as well
 as examples and rationales for when it is or isn't a good idea to break
 backwards compatibility.
 
-General principles:
+In scope for this NEP are:
 
-- Aim not to break users' code unnecessarily.
-- Aim never to change code in ways that can result in users silently getting
-  incorrect results from their previously working code.
-- Backwards incompatible changes can be made, provided the benefits outweigh
-  the costs.
-- When assessing the costs, keep in mind that most users do not read the mailing
-  list, do not look at deprecation warnings, and sometimes wait more than one or
-  two years before upgrading from their old version.  And that NumPy has
-  millions of users, so "no one will do or use this" is very likely incorrect.
-- Benefits include improved functionality, usability and performance,
-  as well as lower maintenance cost and improved future extensibility.
-- Bug fixes are exempt from the backwards compatibility policy.  However in case
-  of serious impact on users (e.g. a downstream library doesn't build anymore),
-  even bug fixes may have to be delayed for one or more releases.
-- The Python API and the C API will be treated in the same way.
+- Principles of NumPy's approach to backwards compatibility.
+- How to deprecate functionality, and when to remove already deprecated
+  functionality.
+- Decision making process for deprecations and removals.
 
+Out of scope are:
 
-Examples
-^^^^^^^^
+- Making concrete decisions about deprecations of particular functionality.
+- NumPy's versioning scheme.
 
-We now discuss a number of concrete examples to illustrate typical issues
-and trade-offs.
+
+General principles
+------------------
+
+When considering proposed changes that are backwards incompatible, the
+main principles the NumPy developers use when making a decision are:
+
+1. Changes need to benefit users more than they harm them.
+2. NumPy is widely used so breaking changes should by default be assumed to be
+   fairly harmful.
+3. Decisions should be based on data and actual effects on users and downstream
+   packages rather than, e.g., appealing to the docs or for stylistic reasons.
+4. Silently getting a wrong answer is much worse than getting a loud error.
+
+When assessing the costs of proposed changes, keep in mind that most users do
+not read the mailing list, do not look at deprecation warnings, and sometimes
+wait more than one or two years before upgrading from their old version. And
+that NumPy has millions of users, so "no one will do or use this" is very
+likely incorrect.
+
+Benefits include improved functionality, usability and performance, as well as
+lower maintenance cost and improved future extensibility.
+
+Fixes for clear bugs are exempt from this backwards compatibility policy.
+However in case of serious impact on users (e.g. a downstream library doesn't
+build anymore or would start giving incorrect results), even bug fixes may have
+to be delayed for one or more releases.
+
+
+Strategies related to deprecations
+----------------------------------
+
+Getting hard data on the impact of a deprecation of often difficult. Strategies
+that can be used to assess such impact include:
+
+- Use a code search engine ([1]_) or static ([2]_) or dynamic ([3]_) code
+  analysis tools to determine where and how the functionality is used.
+- Testing prominent downstream libraries against a development build of NumPy
+  containing the proposed change to get real-world data on its impact.
+- Making a change in master and reverting it, if needed, before a release. We
+  do encourage other packages to test against NumPy's master branch, so this
+  often turns up issues quickly.
+
+If the impact is unclear or significant, it is often good to consider
+alternatives to deprecations. For example discouraging use in documentation
+only, or moving the documentation for the functionality to a less prominent
+place or even removing it completely. Commenting on open issues related to it
+that they are low-prio or labeling them as "wontfix" will also be a signal to
+users, and reduce the maintenance effort needing to be spent.
+
+
+Implementing deprecations and removals
+--------------------------------------
+
+Deprecation warnings are necessary in all cases where functionality
+will eventually be removed.  If there is no intent to remove functionality,
+then it should not be deprecated either. A "please don't use this" in the
+documentation or other type of warning should be used instead.
+
+Deprecations:
+
+- shall include the version number of the release in which the functionality
+  was deprecated.
+- shall include information on alternatives to the deprecated functionality, or a
+  reason for the deprecation if no clear alternative is available.
+- shall use ``VisibleDeprecationWarning`` rather than ``DeprecationWarning``
+  for cases of relevance to end users. For cases only relevant to
+  downstream libraries, a regular ``DeprecationWarning`` is fine.
+  *Rationale: regular deprecation warnings are invisible by default; library
+  authors should be aware how deprecations work and test for them, but we can't
+  expect this from all users.*
+- shall be listed in the release notes of the release where the deprecation is
+  first present.
+- shall set a ``stacklevel``, so the warning appears to come from the correct
+  place.
+- shall be mentioned in the documentation for the functionality. A
+  ``.. deprecated::`` directive can be used for this.
+
+Examples of good deprecation warnings:
+
+.. code-block:: python
+
+    warnings.warn('np.asscalar(a) is deprecated since NumPy 1.16.0, use '
+                  'a.item() instead', DeprecationWarning, stacklevel=3)
+
+    warnings.warn("Importing from numpy.testing.utils is deprecated "
+                  "since 1.15.0, import from numpy.testing instead.",
+                  DeprecationWarning, stacklevel=2)
+
+    # A change in NumPy 1.14.0 for Python 3 loadtxt/genfromtext, slightly
+    # tweaked in this NEP (original didn't have version number).
+    warnings.warn(
+        "Reading unicode strings without specifying the encoding "
+        "argument is deprecated since NumPy 1.14.0. Set the encoding, "
+        "use None for the system default.",
+        np.VisibleDeprecationWarning, stacklevel=2)
+
+Removal of deprecated functionality:
+
+- shall be done after at least 2 releases (assuming the current 6-monthly
+  release cycle; if that changes, there shall be at least 1 year between
+  deprecation and removal).
+- shall be listed in the release notes of the release where the removal happened.
+- can be done in any minor (but not bugfix) release.
+
+For backwards incompatible changes that aren't "deprecate and remove" but for
+which code will start behaving differently, a ``FutureWarning`` should be
+used. Release notes, mentioning version number and using ``stacklevel`` should
+be done in the same way as for deprecation warnings. A ``.. versionchanged::``
+directive can be used in the documentation to indicate when the behavior
+changed:
+
+.. code-block:: python
+
+    def argsort(self, axis=np._NoValue, ...):
+        """
+        Parameters
+        ----------
+        axis : int, optional
+            Axis along which to sort. If None, the default, the flattened array
+            is used.
+
+            ..  versionchanged:: 1.13.0
+                Previously, the default was documented to be -1, but that was
+                in error. At some future date, the default will change to -1, as
+                originally intended.
+                Until then, the axis should be given explicitly when
+                ``arr.ndim > 1``, to avoid a FutureWarning.
+        """
+        ...
+        warnings.warn(
+            "In the future the default for argsort will be axis=-1, not the "
+            "current None, to match its documentation and np.argsort. "
+            "Explicitly pass -1 or None to silence this warning.",
+            MaskedArrayFutureWarning, stacklevel=3)
+
+
+Decision making
+~~~~~~~~~~~~~~~
+
+In concrete cases where this policy needs to be applied, decisions are made according
+to the `NumPy governance model
+<https://docs.scipy.org/doc/numpy/dev/governance/index.html>`_.
+
+All deprecations must be proposed on the mailing list, in order to give everyone
+with an interest in NumPy development to be able to comment. Removal of
+deprecated functionality does not need discussion on the mailing list.
+
+
+Functionality with more strict deprecation policies
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- ``numpy.random`` has its own backwards compatibility policy,
+  see `NEP 19 <http://www.numpy.org/neps/nep-0019-rng-policy.html>`_.
+- The file format for ``.npy`` and ``.npz`` files must not be changed in a backwards
+  incompatible way.
+
+
+Example cases
+-------------
+
+We now discuss a few concrete examples from NumPy's history to illustrate
+typical issues and trade-offs.
 
 **Changing the behavior of a function**
 
@@ -128,53 +279,6 @@ separate ``numpy-financial`` package and removing them from NumPy after a
 deprecation period made sense.
 
 
-Policy
-------
-
-1. Code changes that have the potential to silently change the results of a users'
-   code must never be made (except in the case of clear bugs).
-2. Code changes that break users' code (i.e. the user will see a clear exception)
-   can be made, *provided the benefit is worth the cost* and suitable deprecation
-   warnings have been raised first.
-3. Deprecation warnings are in all cases warnings that functionality will be removed.
-   If there is no intent to remove functionality, then deprecation in documentation
-   only or other types of warnings shall be used.
-4. Deprecations for stylistic reasons (e.g. consistency between functions) are
-   strongly discouraged.
-
-Deprecations:
-
-- shall include the version numbers of both when the functionality was deprecated
-  and when it will be removed (either two releases after the warning is
-  introduced, or in the next major version).
-- shall include information on alternatives to the deprecated functionality, or a
-  reason for the deprecation if no clear alternative is available.
-- shall use ``VisibleDeprecationWarning`` rather than ``DeprecationWarning``
-  for cases of relevance to end users (as opposed to cases only relevant to
-  libraries building on top of NumPy).
-- shall be listed in the release notes of the release where the deprecation happened.
-
-Removal of deprecated functionality:
-
-- shall be done after 2 releases (assuming a 6-monthly release cycle; if that changes,
-  there shall be at least 1 year between deprecation and removal), unless the
-  impact of the removal is such that a major version number increase is
-  warranted.
-- shall be listed in the release notes of the release where the removal happened.
-- can be done in any minor (but not bugfix) release.
-
-In concrete cases where this policy needs to be applied, decisions are made according
-to the `NumPy governance model
-<https://docs.scipy.org/doc/numpy/dev/governance/index.html>`_.
-
-Functionality with more strict policies:
-
-- ``numpy.random`` has its own backwards compatibility policy,
-  see `NEP 19 <http://www.numpy.org/neps/nep-0019-rng-policy.html>`_.
-- The file format for ``.npy`` and ``.npz`` files must not be changed in a backwards
-  incompatible way.
-
-
 Alternatives
 ------------
 
@@ -200,6 +304,12 @@ References and Footnotes
 
 - `Issue requesting semantic versioning <https://github.com/numpy/numpy/issues/10156>`__
 
+.. [1] https://searchcode.com/
+
+.. [2] https://github.com/Quansight-Labs/python-api-inspect
+
+.. [3] https://github.com/data-apis/python-record-api
+
 
 Copyright
 ---------