diff options
-rw-r--r-- | doc/neps/nep-0023-backwards-compatibility.rst | 250 |
1 files changed, 180 insertions, 70 deletions
diff --git a/doc/neps/nep-0023-backwards-compatibility.rst b/doc/neps/nep-0023-backwards-compatibility.rst index 0b58a5f4b..acfb379b7 100644 --- a/doc/neps/nep-0023-backwards-compatibility.rst +++ b/doc/neps/nep-0023-backwards-compatibility.rst @@ -19,44 +19,195 @@ processes for individual cases where breaking backwards compatibility is considered. -Detailed description +Motivation and Scope -------------------- NumPy has a very large user base. Those users rely on NumPy being stable and the code they write that uses NumPy functionality to keep working. NumPy is also actively maintained and improved -- and sometimes improvements -require, or are made much easier, by breaking backwards compatibility. +require, or are made much easier by, breaking backwards compatibility. Finally, there are trade-offs in stability for existing users vs. avoiding errors or having a better user experience for new users. These competing -needs often give rise to heated debates and delays in accepting or rejecting +needs often give rise to long debates and to delays in accepting or rejecting contributions. This NEP tries to address that by providing a policy as well as examples and rationales for when it is or isn't a good idea to break backwards compatibility. -General principles: +In scope for this NEP are: -- Aim not to break users' code unnecessarily. -- Aim never to change code in ways that can result in users silently getting - incorrect results from their previously working code. -- Backwards incompatible changes can be made, provided the benefits outweigh - the costs. -- When assessing the costs, keep in mind that most users do not read the mailing - list, do not look at deprecation warnings, and sometimes wait more than one or - two years before upgrading from their old version. And that NumPy has - millions of users, so "no one will do or use this" is very likely incorrect. -- Benefits include improved functionality, usability and performance, - as well as lower maintenance cost and improved future extensibility. -- Bug fixes are exempt from the backwards compatibility policy. However in case - of serious impact on users (e.g. a downstream library doesn't build anymore), - even bug fixes may have to be delayed for one or more releases. -- The Python API and the C API will be treated in the same way. +- Principles of NumPy's approach to backwards compatibility. +- How to deprecate functionality, and when to remove already deprecated + functionality. +- Decision making process for deprecations and removals. +Out of scope are: -Examples -^^^^^^^^ +- Making concrete decisions about deprecations of particular functionality. +- NumPy's versioning scheme. -We now discuss a number of concrete examples to illustrate typical issues -and trade-offs. + +General principles +------------------ + +When considering proposed changes that are backwards incompatible, the +main principles the NumPy developers use when making a decision are: + +1. Changes need to benefit users more than they harm them. +2. NumPy is widely used so breaking changes should by default be assumed to be + fairly harmful. +3. Decisions should be based on data and actual effects on users and downstream + packages rather than, e.g., appealing to the docs or for stylistic reasons. +4. Silently getting a wrong answer is much worse than getting a loud error. + +When assessing the costs of proposed changes, keep in mind that most users do +not read the mailing list, do not look at deprecation warnings, and sometimes +wait more than one or two years before upgrading from their old version. And +that NumPy has millions of users, so "no one will do or use this" is very +likely incorrect. + +Benefits include improved functionality, usability and performance, as well as +lower maintenance cost and improved future extensibility. + +Fixes for clear bugs are exempt from this backwards compatibility policy. +However in case of serious impact on users (e.g. a downstream library doesn't +build anymore or would start giving incorrect results), even bug fixes may have +to be delayed for one or more releases. + + +Strategies related to deprecations +---------------------------------- + +Getting hard data on the impact of a deprecation of often difficult. Strategies +that can be used to assess such impact include: + +- Use a code search engine ([1]_) or static ([2]_) or dynamic ([3]_) code + analysis tools to determine where and how the functionality is used. +- Testing prominent downstream libraries against a development build of NumPy + containing the proposed change to get real-world data on its impact. +- Making a change in master and reverting it, if needed, before a release. We + do encourage other packages to test against NumPy's master branch, so this + often turns up issues quickly. + +If the impact is unclear or significant, it is often good to consider +alternatives to deprecations. For example discouraging use in documentation +only, or moving the documentation for the functionality to a less prominent +place or even removing it completely. Commenting on open issues related to it +that they are low-prio or labeling them as "wontfix" will also be a signal to +users, and reduce the maintenance effort needing to be spent. + + +Implementing deprecations and removals +-------------------------------------- + +Deprecation warnings are necessary in all cases where functionality +will eventually be removed. If there is no intent to remove functionality, +then it should not be deprecated either. A "please don't use this" in the +documentation or other type of warning should be used instead. + +Deprecations: + +- shall include the version number of the release in which the functionality + was deprecated. +- shall include information on alternatives to the deprecated functionality, or a + reason for the deprecation if no clear alternative is available. +- shall use ``VisibleDeprecationWarning`` rather than ``DeprecationWarning`` + for cases of relevance to end users. For cases only relevant to + downstream libraries, a regular ``DeprecationWarning`` is fine. + *Rationale: regular deprecation warnings are invisible by default; library + authors should be aware how deprecations work and test for them, but we can't + expect this from all users.* +- shall be listed in the release notes of the release where the deprecation is + first present. +- shall set a ``stacklevel``, so the warning appears to come from the correct + place. +- shall be mentioned in the documentation for the functionality. A + ``.. deprecated::`` directive can be used for this. + +Examples of good deprecation warnings: + +.. code-block:: python + + warnings.warn('np.asscalar(a) is deprecated since NumPy 1.16.0, use ' + 'a.item() instead', DeprecationWarning, stacklevel=3) + + warnings.warn("Importing from numpy.testing.utils is deprecated " + "since 1.15.0, import from numpy.testing instead.", + DeprecationWarning, stacklevel=2) + + # A change in NumPy 1.14.0 for Python 3 loadtxt/genfromtext, slightly + # tweaked in this NEP (original didn't have version number). + warnings.warn( + "Reading unicode strings without specifying the encoding " + "argument is deprecated since NumPy 1.14.0. Set the encoding, " + "use None for the system default.", + np.VisibleDeprecationWarning, stacklevel=2) + +Removal of deprecated functionality: + +- shall be done after at least 2 releases (assuming the current 6-monthly + release cycle; if that changes, there shall be at least 1 year between + deprecation and removal). +- shall be listed in the release notes of the release where the removal happened. +- can be done in any minor (but not bugfix) release. + +For backwards incompatible changes that aren't "deprecate and remove" but for +which code will start behaving differently, a ``FutureWarning`` should be +used. Release notes, mentioning version number and using ``stacklevel`` should +be done in the same way as for deprecation warnings. A ``.. versionchanged::`` +directive can be used in the documentation to indicate when the behavior +changed: + +.. code-block:: python + + def argsort(self, axis=np._NoValue, ...): + """ + Parameters + ---------- + axis : int, optional + Axis along which to sort. If None, the default, the flattened array + is used. + + .. versionchanged:: 1.13.0 + Previously, the default was documented to be -1, but that was + in error. At some future date, the default will change to -1, as + originally intended. + Until then, the axis should be given explicitly when + ``arr.ndim > 1``, to avoid a FutureWarning. + """ + ... + warnings.warn( + "In the future the default for argsort will be axis=-1, not the " + "current None, to match its documentation and np.argsort. " + "Explicitly pass -1 or None to silence this warning.", + MaskedArrayFutureWarning, stacklevel=3) + + +Decision making +~~~~~~~~~~~~~~~ + +In concrete cases where this policy needs to be applied, decisions are made according +to the `NumPy governance model +<https://docs.scipy.org/doc/numpy/dev/governance/index.html>`_. + +All deprecations must be proposed on the mailing list, in order to give everyone +with an interest in NumPy development to be able to comment. Removal of +deprecated functionality does not need discussion on the mailing list. + + +Functionality with more strict deprecation policies +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- ``numpy.random`` has its own backwards compatibility policy, + see `NEP 19 <http://www.numpy.org/neps/nep-0019-rng-policy.html>`_. +- The file format for ``.npy`` and ``.npz`` files must not be changed in a backwards + incompatible way. + + +Example cases +------------- + +We now discuss a few concrete examples from NumPy's history to illustrate +typical issues and trade-offs. **Changing the behavior of a function** @@ -128,53 +279,6 @@ separate ``numpy-financial`` package and removing them from NumPy after a deprecation period made sense. -Policy ------- - -1. Code changes that have the potential to silently change the results of a users' - code must never be made (except in the case of clear bugs). -2. Code changes that break users' code (i.e. the user will see a clear exception) - can be made, *provided the benefit is worth the cost* and suitable deprecation - warnings have been raised first. -3. Deprecation warnings are in all cases warnings that functionality will be removed. - If there is no intent to remove functionality, then deprecation in documentation - only or other types of warnings shall be used. -4. Deprecations for stylistic reasons (e.g. consistency between functions) are - strongly discouraged. - -Deprecations: - -- shall include the version numbers of both when the functionality was deprecated - and when it will be removed (either two releases after the warning is - introduced, or in the next major version). -- shall include information on alternatives to the deprecated functionality, or a - reason for the deprecation if no clear alternative is available. -- shall use ``VisibleDeprecationWarning`` rather than ``DeprecationWarning`` - for cases of relevance to end users (as opposed to cases only relevant to - libraries building on top of NumPy). -- shall be listed in the release notes of the release where the deprecation happened. - -Removal of deprecated functionality: - -- shall be done after 2 releases (assuming a 6-monthly release cycle; if that changes, - there shall be at least 1 year between deprecation and removal), unless the - impact of the removal is such that a major version number increase is - warranted. -- shall be listed in the release notes of the release where the removal happened. -- can be done in any minor (but not bugfix) release. - -In concrete cases where this policy needs to be applied, decisions are made according -to the `NumPy governance model -<https://docs.scipy.org/doc/numpy/dev/governance/index.html>`_. - -Functionality with more strict policies: - -- ``numpy.random`` has its own backwards compatibility policy, - see `NEP 19 <http://www.numpy.org/neps/nep-0019-rng-policy.html>`_. -- The file format for ``.npy`` and ``.npz`` files must not be changed in a backwards - incompatible way. - - Alternatives ------------ @@ -200,6 +304,12 @@ References and Footnotes - `Issue requesting semantic versioning <https://github.com/numpy/numpy/issues/10156>`__ +.. [1] https://searchcode.com/ + +.. [2] https://github.com/Quansight-Labs/python-api-inspect + +.. [3] https://github.com/data-apis/python-record-api + Copyright --------- |