DOC: reorganize developer docs, use scikit-image as a base for change

8 files changed, 347 insertions, 58 deletions

diff --git a/doc/source/dev/gitwash/development_workflow.rst b/doc/source/dev/development_workflow.rst
index 9561e25f7..200d95b92 100644
--- a/doc/source/dev/gitwash/development_workflow.rst
+++ b/doc/source/dev/development_workflow.rst
@@ -507,4 +507,4 @@ them to ``upstream`` as follows:
     want.
 
 
-.. include:: git_links.inc
+.. include:: gitwash/git_links.inc
diff --git a/doc/source/dev/gitwash/following_latest.rst b/doc/source/dev/gitwash/following_latest.rst
index ad497bf9a..0e98b4ec4 100644
--- a/doc/source/dev/gitwash/following_latest.rst
+++ b/doc/source/dev/gitwash/following_latest.rst
@@ -1,9 +1,5 @@
 .. _following-latest:
 
-=============================
- Following the latest source
-=============================
-
 These are the instructions if you just want to follow the latest
 *NumPy* source, but you don't need to do any development for now.
 If you do want to contribute a patch (excellent!) or do more extensive
diff --git a/doc/source/dev/gitwash/git_development.rst b/doc/source/dev/gitwash/git_development.rst
deleted file mode 100644
index 5d7d47f89..000000000
--- a/doc/source/dev/gitwash/git_development.rst
+++ /dev/null
@@ -1,14 +0,0 @@
-.. _git-development:
-
-=====================
- Git for development
-=====================
-
-Contents:
-
-.. toctree::
-   :maxdepth: 2
-
-   development_setup
-   configure_git
-   dot2_dot3
diff --git a/doc/source/dev/gitwash/git_intro.rst b/doc/source/dev/gitwash/git_intro.rst
index 3ce322f8f..47994f69c 100644
--- a/doc/source/dev/gitwash/git_intro.rst
+++ b/doc/source/dev/gitwash/git_intro.rst
@@ -1,42 +1,15 @@
-============
-Introduction
-============
-
-These pages describe a git_ and github_ workflow for the NumPy_
-project.
-
-There are several different workflows here, for different ways of
-working with *NumPy*.
-
-This is not a comprehensive git_ reference, it's just a workflow for our
-own project.  It's tailored to the github_ hosting service. You may well
-find better or quicker ways of getting stuff done with git_, but these
-should get you started.
-
-For general resources for learning git_ see :ref:`git-resources`.
-
-.. _install-git:
-
 Install git
 ===========
 
-Overview
---------
+Developing with git can be done entirely without github. Git is a distributed
+version control system. In order to use git on your machine you must install
+it.
 
 ================ =============
 Debian / Ubuntu  ``sudo apt-get install git-core``
 Fedora           ``sudo yum install git-core``
-Windows          Download and install msysGit_
+Windows          Download and install gitforwindows_
 OS X             Use the git-osx-installer_
 ================ =============
 
-In detail
----------
-
-See the git_ page for the most recent information.
-
-Have a look at the github_ install help pages available from `github help`_
-
-There are good instructions here: http://book.git-scm.com/2_installing_git.html
-
 .. include:: git_links.inc
diff --git a/doc/source/dev/gitwash/git_links.inc b/doc/source/dev/gitwash/git_links.inc
index cebbb3a67..0e2fd94b8 100644
--- a/doc/source/dev/gitwash/git_links.inc
+++ b/doc/source/dev/gitwash/git_links.inc
@@ -10,9 +10,9 @@
 
 .. git stuff
 .. _git: https://git-scm.com/
-.. _github: https://github.com
+.. _github: https://github.com/numpy/numpy
 .. _github help: https://help.github.com
-.. _msysgit: https://code.google.com/p/msysgit/downloads/list
+.. _gitforwindows: https://gitforwindows.org/
 .. _git-osx-installer: https://code.google.com/p/git-osx-installer/downloads/list
 .. _subversion: http://subversion.tigris.org/
 .. _git cheat sheet: http://cheat.errtheblog.com/s/git
diff --git a/doc/source/dev/gitwash/index.rst b/doc/source/dev/gitwash/index.rst
index b867bbd97..afbb5e019 100644
--- a/doc/source/dev/gitwash/index.rst
+++ b/doc/source/dev/gitwash/index.rst
@@ -1,7 +1,22 @@
 .. _using-git:
+.. _git-development:
+
+=====================
+ Git for development
+=====================
+
+These pages describe a general git_ and github_ workflow.
+
+This is not a comprehensive git_ reference. It's tailored to the github_
+hosting service. You may well find better or quicker ways of getting stuff done
+with git_, but these should get you started.
+
+For general resources for learning git_ see :ref:`git-resources`.
+
+Have a look at the github_ install help pages available from `github help`_
+
+.. _install-git:
 
-Working with *NumPy* source code
-================================
 
 Contents:
 
@@ -10,6 +25,9 @@ Contents:
 
    git_intro
    following_latest
-   git_development
-   development_workflow
+   development_setup
+   configure_git
+   dot2_dot3
    git_resources
+
+.. include:: git_links.inc
diff --git a/doc/source/dev/index.rst b/doc/source/dev/index.rst
index 43aff1931..8fcdaa592 100644
--- a/doc/source/dev/index.rst
+++ b/doc/source/dev/index.rst
@@ -2,15 +2,331 @@
 Contributing to NumPy
 #####################
 
+Development process
+===================
+
+Here's the long and short of it:
+
+1. If you are a first-time contributor:
+
+   * Go to `https://github.com/numpy/numpy
+     <https://github.com/numpy/numpy>`_ and click the
+     "fork" button to create your own copy of the project.
+
+   * Clone the project to your local computer::
+
+      git clone https://github.com/your-username/numpy.git
+
+   * Change the directory::
+
+      cd numpy
+
+   * Add the upstream repository::
+
+      git remote add upstream https://github.com/numpy/numpy.git
+
+   * Now, `git remote -v` will show two remote repositories named:
+
+     - ``upstream``, which refers to the ``numpy`` repository
+     - ``origin``, which refers to your personal fork
+
+2. Develop your contribution:
+
+   * Pull the latest changes from upstream::
+
+      git checkout master
+      git pull upstream master
+
+   * Create a branch for the feature you want to work on. Since the
+     branch name will appear in the merge message, use a sensible name
+     such as 'transform-speedups'::
+
+      git checkout -b transform-speedups
+
+   * Commit locally as you progress (``git add`` and ``git commit``)
+     Use a `properly formatted <writing-the-commit-message>` commit message,
+     write tests that fail before your change and pass afterward, run all the
+     `tests locally <development-environment>` after running ``git clean -xfd``
+     to be sure you have not broken anything, and be sure to document any
+     changed behavior in `numpydoc-appropriate docstrings <howto-document>`.
+
+3. To submit your contribution:
+
+   * Push your changes back to your fork on GitHub::
+
+      git push origin transform-speedups
+
+   * Enter your GitHub username and password (repeat contributors or advanced
+     users can remove this step by connecting to GitHub with `SSH <set-up-and-
+     configure-a-github-account>`.
+
+   * Go to GitHub. The new branch will show up with a green Pull Request
+     button. Make sure the title and message are clear, concise, and self-
+     explanatory. Then click the button to submit it.
+
+   * If your commit introduces a new feature or changes functionality, post on
+     the `mailing list`_ to explain your changes. For bug fixes, documentation
+     updates, etc., this is generally not necessary, though if you do not get
+     any reaction, do feel free to ask for review.
+
+4. Review process:
+
+   * Reviewers (the other developers and interested community members) will
+     write inline and/or general comments on your Pull Request (PR) to help
+     you improve its implementation, documentation and style.  Every single
+     developer working on the project has their code reviewed, and we've come
+     to see it as friendly conversation from which we all learn and the
+     overall code quality benefits.  Therefore, please don't let the review
+     discourage you from contributing: its only aim is to improve the quality
+     of project, not to criticize (we are, after all, very grateful for the
+     time you're donating!).
+
+   * To update your PR, make your changes on your local repository, commit,
+     **run tests, and only if they succeed** push to your fork. As soon as
+     those changes are pushed up (to the same branch as before) the PR will
+     update automatically. If you have no idea how to fix the test failures,
+     you may push your changes anyway and ask for help in a PR comment.
+
+   * Various continuous integration (CI) services are triggered after each PR
+     update to build the code, run unit tests, measure code coverage and check
+     coding style of your branch. The CI tests must pass before your PR can be
+     merged. If CI fails, you can find out why by clicking on the "failed"
+     icon (red cross) and inspecting the build and test log. To avoid overuse
+     and waste of this resource, `test your work <recommended-development-
+     setup>` locally before committing.
+
+   * A code-change PR must be **approved** by at least two core team members
+     before merging. Approval means the core team member has carefully reviewed
+     the changes, and the PR is ready for merging.
+
+5. Document changes
+
+   Beyond changes to a functions docstring and possible description in the
+   general documentation, if your change introduces any user-facing
+   modifications, update the current release notice under
+   ``doc/release/X.XX-notes.rst``
+
+   If your change introduces a deprecation, make sure to follow the
+   `deprecation policy <http://www.numpy.org/neps/nep-0023-backwards-
+   compatibility.html>`_.
+
+6. Cross referencing issues
+
+   If the PR relates to any issues, you can add the text ``xref #xxxx`` where
+   ``xxxx`` is the number of the issue to github comments. Likewise, if the PR
+   solves an issue, replace the ``xref`` with ``closes``, ``fixes`` or any of
+   the other flavors `github accepts <https://help.github.com/en/articles/
+   closing-issues-using-keywords>`_.
+
+   In the source code, be sure to preface any issue or PR reference with
+   ``gh-xxxx`` since the code may move to a different host.
+
+For a more detailed discussion, read on and follow the links at the bottom of
+this page.
+
+Divergence between ``upstream/master`` and your feature branch
+--------------------------------------------------------------
+
+If GitHub indicates that the branch of your Pull Request can no longer
+be merged automatically, you have to incorporate changes that have been made
+since you started into your branch. Our recommended way to do this is to
+`rebase on master <rebasing-on-master>`.
+
+Guidelines
+----------
+
+* All code should have tests (see `test coverage`_ below for more details).
+* All code should be `documented <docstring-standard>`.
+* No changes are ever committed without review and approval by a core
+  team member.  Ask on the `mailing list`_ if you get no response to your pull
+  request.
+
+Stylistic Guidelines
+--------------------
+
+* Set up your editor to follow `PEP08 <https://www.python.org/dev/peps/
+  pep-0008/>`_ (remove trailing white space, no tabs, etc.).  Check code with
+  pyflakes / flake8.
+
+* Use numpy data types instead of strings (``np.uint8`` instead of
+  ``"uint8"``).
+
+* Use the following import conventions::
+
+   import numpy as np
+
+   cimport numpy as cnp  # in Cython code
+
+* Use ``np_intp`` as data type for all indexing, shape and size variables
+  in C/C++ and Cython code.
+
+* For C code, see the `numpy-c-style-guide`
+
+
+Test coverage
+-------------
+
+Pull requests (PRs) that modify code should either have new tests, or modify existing
+tests to fail before the PR and pass afterwards. You should `run the tests
+<development-environment>` before pushing a PR.
+
+Tests for a module should ideally cover all code in that module,
+i.e., statement coverage should be at 100%.
+
+To measure the test coverage, install
+`pytest-cov <https://pytest-cov.readthedocs.io/en/latest/>`__
+and then run::
+
+  $ python runtests.py --coverage
+
+This will create a report in `build/coverage`, which can be viewed with
+
+```
+firefox build/coverage/index.html
+```
+
+Building docs
+-------------
+
+To build docs, run ``make`` from the ``doc`` directory. ``make help`` lists
+all targets. For example, to build the HTML documentation, you can run:
+
+.. code:: sh
+
+    make html
+
+Then, all the HTML files will be generated in ``doc/build/html/``.
+To rebuild a full clean documentation, run:
+
+.. code:: sh
+
+    make dist
+
+Requirements
+~~~~~~~~~~~~
+
+`Sphinx <http://www.sphinx-doc.org/en/stable/>`__ and LaTeX are needed to build
+the documentation.
+
+**Sphinx:**
+
+Sphinx and other python packages needed to build the documentation
+Python dependencies can be installed using:
+
+.. code:: sh
+
+    pip install -r sphinx matplotlib scipy
+
+**LaTeX Ubuntu:**
+
+.. code:: sh
+
+    sudo apt-get install -qq texlive texlive-latex-extra dvipng
+
+**LaTeX Mac:**
+
+Install the full `MacTex <https://www.tug.org/mactex/>`__ installation or
+install the smaller
+`BasicTex <https://www.tug.org/mactex/morepackages.html>`__ and add *ucs*
+and *dvipng* packages:
+
+.. code:: sh
+
+    sudo tlmgr install ucs dvipng
+
+Fixing Warnings
+~~~~~~~~~~~~~~~
+
+-  "citation not found: R###" There is probably an underscore after a
+   reference in the first line of a docstring (e.g. [1]\_). Use this
+   method to find the source file: $ cd doc/build; grep -rin R####
+
+-  "Duplicate citation R###, other instance in..."" There is probably a
+   [2] without a [1] in one of the docstrings
+
+-  Make sure to use pre-sphinxification paths to images (not the
+   \_images directory)
+
+Deprecation cycle
+-----------------
+
+If the behavior of the library has to be changed, a deprecation cycle must be
+followed to warn users.
+
+- a deprecation cycle is *not* necessary when:
+
+    * adding a new function, or
+    * adding a new keyword argument to the *end* of a function signature, or
+    * fixing what was buggy behaviour
+
+- a deprecation cycle is necessary for *any breaking API change*, meaning a
+    change where the function, invoked with the same arguments, would return a
+    different result after the change. This includes:
+
+    * changing the order of arguments or keyword arguments, or
+    * adding arguments or keyword arguments to a function, or
+    * changing a function's name or submodule, or
+    * changing the default value of a function's arguments.
+
+Usually, our policy is to put in place a deprecation cycle over two releases.
+
+For the sake of illustration, we consider the modification of a default value in
+a function signature. In version N (therefore, next release will be N+1), we
+have
+
+.. code-block:: python
+
+    def a_function(array, rescale=True):
+        out = do_something(array, rescale=rescale)
+        return out
+
+that has to be changed to
+
+.. code-block:: python
+
+    def a_function(array, rescale=None):
+        if rescale is None:
+            # 2019-02-24
+            warn('The default value of rescale will change to `False`
+                 in version N+3', DeprecationWarning, stacklevel=2)
+            rescale = True
+        out = do_something(image, rescale=rescale)
+        return out
+
+and in version N+3
+
+.. code-block:: python
+
+    def a_function(array, rescale=False):
+        out = do_something(array, rescale=rescale)
+        return out
+
+Here is the process for a 2-release deprecation cycle:
+
+- In the signature, set default to `None`, and modify the docstring to specify
+  that it's `True`.
+- In the function, _if_ rescale is set to `None`, set to `True` and warn that the
+  default will change to `False` in version N+3.
+- In ``doc/release/X.XX-notes.rst``, under deprecations, add "In
+  `a_function`, the `rescale` argument will default to `False` in N+3."
+
+Note that the 2-release deprecation cycle is not a strict rule and in some
+cases, the developers can agree on a different procedure upon justification
+(like when we can't detect the change, or it involves moving or deleting an
+entire function for example).
+
 .. toctree::
    :maxdepth: 2
 
    conduct/code_of_conduct
-   gitwash/index
+   Git Basics <gitwash/index>
    development_environment
+   development_workflow
    ../benchmarking
    style_guide
    releasing
    governance/index
 
-For core developers: see :ref:`development-workflow`.
+NumPy-specific workflow is in `development-workflow`.
+
+.. _`mailing list`: https://mail.python.org/mailman/listinfo/numpy-devel
diff --git a/doc/source/dev/gitwash/pull_button.png b/doc/source/dev/pull_button.png
index e5031681b..e5031681b 100644
--- a/doc/source/dev/gitwash/pull_button.png
+++ b/doc/source/dev/pull_button.png