Merge branch 'master' of https://github.com/numpy/numpy into patch-2

9 files changed, 350 insertions, 21 deletions

diff --git a/doc/TESTS.rst.txt b/doc/TESTS.rst.txt
index 9bde1849f..92f236f5a 100644
--- a/doc/TESTS.rst.txt
+++ b/doc/TESTS.rst.txt
@@ -107,6 +107,11 @@ whether a certain assumption is valid. If the assertion fails, the test fails.
 Note that the Python builtin ``assert`` should not be used, because it is
 stripped during compilation with ``-O``.
 
+Note that ``test_`` functions or methods should not have a docstring, because
+that makes it hard to identify the test from the output of running the test
+suite with ``verbose=2`` (or similar verbosity setting).  Use plain comments
+(``#``) if necessary.
+
 Sometimes it is convenient to run ``test_yyy.py`` by itself, so we add
 
 ::
@@ -174,7 +179,7 @@ nose decorators::
 
   @nose.with_setup(setup_func, teardown_func)
   def test_with_extras():
-      """This test uses the setup/teardown functions."""
+      # This test uses the setup/teardown functions.
       global helpful_variable
       print "  In test_with_extras"
       print "  Helpful is %s" % helpful_variable
@@ -198,6 +203,15 @@ Note that 'check_even' is not itself a test (no 'test' in the name),
 but 'test_evens' is a generator that returns a series of tests, using
 'check_even', across a range of inputs.
 
+A problem with generator tests can be that if a test is failing, it's
+hard to see for which parameters.  To avoid this problem, ensure that:
+
+  - No computation related to the features tested is done in the 
+    ``test_*`` generator function, but delegated to a corresponding
+    ``check_*`` function (can be inside the generator, to share namespace).
+  - The generators are used *solely* for loops over parameters.
+  - Those parameters are *not* arrays.
+
 .. warning::
 
    Parametric tests cannot be implemented on classes derived from
diff --git a/doc/release/1.7.0-notes.rst b/doc/release/1.7.0-notes.rst
new file mode 100644
index 000000000..e8b1de72d
--- /dev/null
+++ b/doc/release/1.7.0-notes.rst
@@ -0,0 +1,279 @@
+=========================
+NumPy 1.7.0 Release Notes
+=========================
+
+This release includes several new features as well as numerous bug fixes and
+refactorings. It supports Python 2.4 - 2.7 and 3.1 - 3.2.
+
+Highlights
+==========
+
+* ``where=`` parameter to ufuncs (allows the use of boolean arrays to choose
+  where a computation should be done)
+* ``vectorize`` improvements (added 'excluded' and 'cache' keyword, general
+  cleanup and bug fixes)
+* ``numpy.random.choice`` (random sample generating function)
+
+
+Compatibility notes
+===================
+
+In a future version of numpy, the functions np.diag, np.diagonal, and
+the diagonal method of ndarrays will return a view onto the original
+array, instead of producing a copy as they do now. This makes a
+difference if you write to the array returned by any of these
+functions. To facilitate this transition, numpy 1.7 produces a
+FutureWarning if it detects that you may be attempting to write to
+such an array. See the documentation for np.diagonal for details.
+
+The default casting rule for UFunc out= parameters has been changed from
+'unsafe' to 'same_kind'.  Most usages which violate the 'same_kind'
+rule are likely bugs, so this change may expose previously undetected
+errors in projects that depend on NumPy.
+
+Full-array boolean indexing has been optimized to use a different,
+optimized code path.   This code path should produce the same results,
+but any feedback about changes to your code would be appreciated. 
+
+Attempting to write to a read-only array (one with
+``arr.flags.writeable`` set to ``False``) used to raise either a
+RuntimeError, ValueError, or TypeError inconsistently, depending on
+which code path was taken. It now consistently raises a ValueError.
+
+The <ufunc>.reduce functions evaluate some reductions in a different
+order than in previous versions of NumPy, generally providing higher
+performance. Because of the nature of floating-point arithmetic, this
+may subtly change some results, just as linking NumPy to a different
+BLAS implementations such as MKL can.
+
+If upgrading from 1.5, then generally in 1.6 and 1.7 there have been
+substantial code added and some code paths altered, particularly in
+the areas of type resolution and buffered iteration over universal
+functions.   This might have an impact on your code particularly if
+you relied on accidental behavior in the past. 
+
+New features
+============
+
+Mask-based NA missing values
+----------------------------
+
+Preliminary support for NA missing values similar to those in R has
+been implemented.  This was done by adding optional NA masks to the core
+array object.
+
+.. note:: The NA API is *experimental*, and may undergo changes in future
+   versions of NumPy.  The current implementation based on masks will likely be
+   supplemented by a second one based on bit-patterns, and it is possible that
+   a difference will be made between missing and ignored data.
+
+While a significant amount of the NumPy functionality has been extended to
+support NA masks, not everything is yet supported. Here is an (incomplete)
+list of things that do and do not work with NA values:
+
+What works with NA:
+    * Basic indexing and slicing, as well as full boolean mask indexing.
+    * All element-wise ufuncs.
+    * All UFunc.reduce methods, with a new skipna parameter.
+    * np.all and np.any satisfy the rules NA | True == True and
+      NA & False == False
+    * The nditer object.
+    * Array methods:
+       + ndarray.clip, ndarray.min, ndarray.max, ndarray.sum, ndarray.prod,
+         ndarray.conjugate, ndarray.diagonal, ndarray.flatten
+       + numpy.concatenate, numpy.column_stack, numpy.hstack,
+         numpy.vstack, numpy.dstack, numpy.squeeze, numpy.mean, numpy.std,
+         numpy.var
+
+What doesn't work with NA:
+    * Fancy indexing, such as with lists and partial boolean masks.
+    * ndarray.flat and any other methods that use the old iterator
+      mechanism instead of the newer nditer.
+    * Struct dtypes, which will have corresponding struct masks with
+      one mask value per primitive field of the struct dtype.
+    * UFunc.accumulate, UFunc.reduceat.
+    * Ufunc calls with both NA masks and a where= mask at the same time.
+    * File I/O has not been extended to support NA-masks.
+    * np.logical_and and np.logical_or don't satisfy the
+      rules NA | True == True and NA & False == False yet.
+    * Array methods:
+       + ndarray.argmax, ndarray.argmin,
+       + numpy.repeat, numpy.delete (relies on fancy indexing),
+         numpy.append, numpy.insert (relies on fancy indexing),
+         numpy.where,
+
+Differences with R:
+    * R's parameter rm.na=T is spelled skipna=True in NumPy.
+    * np.isna(nan) is False, but R's is.na(nan) is TRUE. This is because
+      NumPy's NA is treated independently of the underlying data type.
+    * Boolean indexing, where the result is compressed to just
+      the elements with true in the mask, raises if the boolean mask
+      has an NA value in it. This is because that value could be either
+      True or False, meaning the count of the output array is actually
+      NA. R treats this case in a manner inconsistent with the NA model,
+      returning NA values in the spots where the boolean index has NA.
+      This may have a practical advantage in spite of violating the
+      NA theoretical model, so NumPy could adopt the behavior if necessary
+
+For more information, see `doc/neps/missing-data.rst <https://github.com/numpy/numpy/blob/maintenance/1.7.x/doc/neps/missing-data.rst>`_.
+
+Reduction UFuncs Generalize axis= Parameter
+-------------------------------------------
+
+Any ufunc.reduce function call, as well as other reductions like
+sum, prod, any, all, max and min support the ability to choose
+a subset of the axes to reduce over. Previously, one could say
+axis=None to mean all the axes or axis=# to pick a single axis.
+Now, one can also say axis=(#,#) to pick a list of axes for reduction.
+
+Reduction UFuncs New keepdims= Parameter
+----------------------------------------
+
+There is a new keepdims= parameter, which if set to True, doesn't
+throw away the reduction axes but instead sets them to have size one.
+When this option is set, the reduction result will broadcast correctly
+to the original operand which was reduced.
+
+Datetime support
+----------------
+
+.. note:: The datetime API is *experimental* in 1.7.0, and may undergo changes
+   in future versions of NumPy.
+
+There have been a lot of fixes and enhancements to datetime64 compared
+to NumPy 1.6:
+
+* the parser is quite strict about only accepting ISO 8601 dates, with a few
+  convenience extensions
+* converts between units correctly
+* datetime arithmetic works correctly
+* business day functionality (allows the datetime to be used in contexts where
+  only certain days of the week are valid)
+
+The notes in `doc/source/reference/arrays.datetime.rst <https://github.com/numpy/numpy/blob/maintenance/1.7.x/doc/source/reference/arrays.datetime.rst>`_
+(also available in the online docs at `arrays.datetime.html
+<http://docs.scipy.org/doc/numpy/reference/arrays.datetime.html>`_) should be
+consulted for more details.
+
+Custom formatter for printing arrays
+------------------------------------
+
+See the new ``formatter`` parameter of the ``numpy.set_printoptions``
+function.
+
+New function numpy.random.choice
+---------------------------------
+
+A generic sampling function has been added which will generate samples from
+a given array-like. The samples can be with or without replacement, and
+with uniform or given non-uniform probabilities.
+
+New function isclose
+--------------------
+
+Returns a boolean array where two arrays are element-wise equal within a
+tolerance. Both relative and absolute tolerance can be specified. The
+function is NA aware.
+
+Preliminary multi-dimensional support in the polynomial package
+---------------------------------------------------------------
+
+Axis keywords have been added to the integration and differentiation
+functions and a tensor keyword was added to the evaluation functions.
+These additions allow multi-dimensional coefficient arrays to be used in
+those functions. New functions for evaluating 2-D and 3-D coefficient
+arrays on grids or sets of points were added together with 2-D and 3-D
+pseudo-Vandermonde matrices that can be used for fitting.
+
+Support for mask-based NA values in the polynomial package fits
+---------------------------------------------------------------
+
+The fitting functions recognize and remove masked data from the fit.
+
+Ability to pad rank-n arrays
+----------------------------
+
+A pad module containing functions for padding n-dimensional arrays has
+been added. The various private padding functions are exposed as options to
+a public 'pad' function.  Example::
+
+    pad(a, 5, mode='mean')
+
+Current modes are ``constant``, ``edge``, ``linear_ramp``, ``maximum``,
+``mean``, ``median``, ``minimum``, ``reflect``, ``symmetric``, ``wrap``, and
+``<function>``.
+
+
+New argument to searchsorted
+----------------------------
+
+The function searchsorted now accepts a 'sorter' argument that is a
+permuation array that sorts the array to search.
+
+C API
+-----
+
+New function ``PyArray_RequireWriteable`` provides a consistent
+interface for checking array writeability -- any C code which works
+with arrays whose WRITEABLE flag is not known to be True a priori,
+should make sure to call this function before writing.
+
+NumPy C Style Guide added (``doc/C_STYLE_GUIDE.rst.txt``).
+
+Changes
+=======
+
+General
+-------
+
+The function np.concatenate tries to match the layout of its input
+arrays. Previously, the layout did not follow any particular reason,
+and depended in an undesirable way on the particular axis chosen for
+concatenation. A bug was also fixed which silently allowed out of bounds
+axis arguments.
+
+The ufuncs logical_or, logical_and, and logical_not now follow Python's
+behavior with object arrays, instead of trying to call methods on the
+objects. For example the expression (3 and 'test') produces the string
+'test', and now np.logical_and(np.array(3, 'O'), np.array('test', 'O'))
+produces 'test' as well.
+
+C-API
+-----
+
+The following macros now require trailing semicolons::
+
+    NPY_BEGIN_THREADS_DEF
+    NPY_BEGIN_THREADS
+    NPY_ALLOW_C_API
+    NPY_ALLOW_C_API_DEF
+    NPY_DISABLE_C_API
+
+
+Deprecations
+============
+
+General
+-------
+
+Specifying a custom string formatter with a `_format` array attribute is
+deprecated. The new `formatter` keyword in ``numpy.set_printoptions`` or
+``numpy.array2string`` can be used instead.
+
+The deprecated imports in the polynomial package have been removed.
+
+C-API
+-----
+
+Direct access to the fields of PyArrayObject* has been deprecated. Direct
+access has been recommended against for many releases. Expect similar
+deprecations for PyArray_Descr* and other core objects in the future as
+preparation for NumPy 2.0.
+
+The macros in old_defines.h are deprecated and will be removed in the next
+minor release (>= 1.8). The sed script tools/replace_old_macros.sed can
+be used to replace these macros with the newer versions.
+
+You can test your code against the deprecated C API by #defining
+NPY_NO_DEPRECATED_API to the target version number, for example
+NPY_1_7_API_VERSION, before including any NumPy headers.
diff --git a/doc/source/dev/gitwash/development_setup.rst b/doc/source/dev/gitwash/development_setup.rst
index 489a2e75d..0cff3c214 100644
--- a/doc/source/dev/gitwash/development_setup.rst
+++ b/doc/source/dev/gitwash/development_setup.rst
@@ -38,8 +38,8 @@ Create your own forked copy of NumPy_
 
    .. image:: forking_button.png
 
-   Now, after a short pause and some 'Hardcore forking action', you
-   should find yourself at the home page for your own forked copy of NumPy_.
+   After a short pause, you should find yourself at the home page for
+   your own forked copy of NumPy_.
 
 .. include:: git_links.inc
 
@@ -49,7 +49,7 @@ Create your own forked copy of NumPy_
 Set up your fork
 ################
 
-First you follow the instructions for :ref:`forking`. 
+First you follow the instructions for :ref:`forking`.
 
 Overview
 ========
@@ -110,4 +110,3 @@ Just for your own satisfaction, show yourself that you now have a new
    origin	git@github.com:your-user-name/numpy.git (push)
 
 .. include:: git_links.inc
-
diff --git a/doc/source/reference/c-api.array.rst b/doc/source/reference/c-api.array.rst
index 8736cbc3f..bf8077a69 100644
--- a/doc/source/reference/c-api.array.rst
+++ b/doc/source/reference/c-api.array.rst
@@ -289,6 +289,7 @@ From scratch
 
     Fill the array pointed to by *obj* ---which must be a (subclass
     of) bigndarray---with the contents of *val* (evaluated as a byte).
+    This macro calls memset, so obj must be contiguous.
 
 .. cfunction:: PyObject* PyArray_Zeros(int nd, npy_intp* dims, PyArray_Descr* dtype, int fortran)
 
diff --git a/doc/source/reference/index.rst b/doc/source/reference/index.rst
index 2e881542e..128339e16 100644
--- a/doc/source/reference/index.rst
+++ b/doc/source/reference/index.rst
@@ -7,6 +7,7 @@ NumPy Reference
 :Release: |version|
 :Date: |today|
 
+
 .. module:: numpy
 
 This reference manual details functions, modules, and objects
@@ -20,7 +21,7 @@ For learning how to use NumPy, see also :ref:`user`.
    arrays
    ufuncs
    routines
-   ctypes
+   ctypeslib
    distutils
    c-api
    internals
diff --git a/doc/source/reference/routines.matlib.rst b/doc/source/reference/routines.matlib.rst
index 7f8a9eabb..a35eaec78 100644
--- a/doc/source/reference/routines.matlib.rst
+++ b/doc/source/reference/routines.matlib.rst
@@ -7,5 +7,30 @@ This module contains all functions in the :mod:`numpy` namespace, with
 the following replacement functions that return :class:`matrices
 <matrix>` instead of :class:`ndarrays <ndarray>`.
 
-.. automodule:: numpy.matlib
+.. currentmodule:: numpy
+
+Functions that are also in the numpy namespace and return matrices
+
+.. autosummary::
+
+   mat
+   matrix
+   asmatrix
+   bmat
+
+
+Replacement functions in `matlib`
+
+.. currentmodule:: numpy.matlib
+
+.. autosummary::
+   :toctree: generated/
 
+   empty
+   zeros
+   ones
+   eye
+   identity
+   repmat
+   rand
+   randn
diff --git a/doc/source/reference/routines.polynomials.rst b/doc/source/reference/routines.polynomials.rst
index 3e9b2603f..e85d0549b 100644
--- a/doc/source/reference/routines.polynomials.rst
+++ b/doc/source/reference/routines.polynomials.rst
@@ -1,12 +1,22 @@
 Polynomials
 ***********
 
-The polynomial package is newer and more complete than poly1d and the
-convenience classes are better behaved in the numpy environment. When
-backwards compatibility is not an issue it should be the package of choice.
-Note that the  various routines in the polynomial package all deal with
-series whose coefficients go from degree zero upward, which is the reverse
-of the poly1d convention. The easy way to remember this is that indexes
+Polynomials in NumPy can be *created*, *manipulated*, and even *fitted* using
+the :doc:`routines.polynomials.classes`
+of the `numpy.polynomial` package, introduced in NumPy 1.4.
+
+Prior to NumPy 1.4, `numpy.poly1d` was the class of choice and it is still
+available in order to maintain backward compatibility.
+However, the newer Polynomial package is more complete than `numpy.poly1d`
+and its convenience classes are better behaved in the numpy environment.
+Therefore Polynomial is recommended for new coding.
+
+Transition notice
+-----------------
+The  various routines in the Polynomial package all deal with
+series whose coefficients go from degree zero upward,
+which is the *reverse order* of the Poly1d convention.
+The easy way to remember this is that indexes
 correspond to degree, i.e., coef[i] is the coefficient of the term of
 degree i.
 
@@ -14,10 +24,9 @@ degree i.
 .. toctree::
    :maxdepth: 2
 
-   routines.polynomials.poly1d
-
+   routines.polynomials.package
 
 .. toctree::
    :maxdepth: 2
 
-   routines.polynomials.package
+   routines.polynomials.poly1d
diff --git a/doc/source/reference/routines.random.rst b/doc/source/reference/routines.random.rst
index 84765c035..c8b097d7d 100644
--- a/doc/source/reference/routines.random.rst
+++ b/doc/source/reference/routines.random.rst
@@ -37,7 +37,7 @@ Distributions
    beta
    binomial
    chisquare
-   mtrand.dirichlet
+   dirichlet
    exponential
    f
    gamma
@@ -75,7 +75,7 @@ Random generator
 .. autosummary::
    :toctree: generated/
 
-   mtrand.RandomState
+   RandomState
    seed
    get_state
    set_state
diff --git a/doc/source/reference/routines.statistics.rst b/doc/source/reference/routines.statistics.rst
index b41b62839..c420705af 100644
--- a/doc/source/reference/routines.statistics.rst
+++ b/doc/source/reference/routines.statistics.rst
@@ -4,17 +4,18 @@ Statistics
 .. currentmodule:: numpy
 
 
-Extremal values
----------------
+Order statistics
+----------------
 
 .. autosummary::
    :toctree: generated/
 
    amin
    amax
-   nanmax
    nanmin
+   nanmax
    ptp
+   percentile
 
 Averages and variances
 ----------------------