diff options
Diffstat (limited to 'doc/source/reference')
11 files changed, 247 insertions, 109 deletions
diff --git a/doc/source/reference/random/bit_generators/index.rst b/doc/source/reference/random/bit_generators/index.rst index 94d3d8a3c..315657172 100644 --- a/doc/source/reference/random/bit_generators/index.rst +++ b/doc/source/reference/random/bit_generators/index.rst @@ -19,7 +19,7 @@ The included BitGenerators are: and can be advanced by an arbitrary amount. See the documentation for :meth:`~.PCG64.advance`. PCG-64 has a period of :math:`2^{128}`. See the `PCG author's page`_ for more details about this class of PRNG. -* MT19937 - The standard Python BitGenerator. Adds a `~mt19937.MT19937.jumped` +* MT19937 - The standard Python BitGenerator. Adds a `MT19937.jumped` function that returns a new generator with state as-if :math:`2^{128}` draws have been made. * Philox - A counter-based generator capable of being advanced an diff --git a/doc/source/reference/random/c-api.rst b/doc/source/reference/random/c-api.rst new file mode 100644 index 000000000..3c901f3b4 --- /dev/null +++ b/doc/source/reference/random/c-api.rst @@ -0,0 +1,177 @@ +Cython API for random +--------------------- + +.. currentmodule:: numpy.random + +Typed versions of many of the `Generator` and `BitGenerator` can be accessed +directly from Cython: the complete list is given below. + +The ``_bit_generator`` module is usable via:: + + cimport numpy.random._bit_generator + +It provides function pointers for quickly accessing the next bytes in the +`BitGenerator`:: + + struct bitgen: + void *state + uint64_t (*next_uint64)(void *st) nogil + uint32_t (*next_uint32)(void *st) nogil + double (*next_double)(void *st) nogil + uint64_t (*next_raw)(void *st) nogil + + ctypedef bitgen bitgen_t + +See `extending` for examples of using these functions. + +The ``_generator`` module is usable via:: + + cimport numpy.random._generator + +It provides low-level functions for various distributions. All the functions require a ``bitgen_t`` BitGenerator structure. The functions are named with the followig cconventions: + +- "standard" refers to the reference values for any parameters. For instance + "standard_uniform" means a uniform distribution on the interval ``0.0`` to + ``1.0`` + +- "fill" functions will fill the provided ``out`` with ``cnt`` values. + +- The functions without "standard" in their name require additional parameters + to describe the distributions. + +.. c:function:: double random_standard_uniform(bitgen_t *bitgen_state) + +.. c:function:: void random_standard_uniform_fill(bitgen_t* bitgen_state, np.npy_intp cnt, double *out) + +.. c:function:: double random_standard_exponential(bitgen_t *bitgen_state) + +.. c:function:: void random_standard_exponential_fill(bitgen_t *bitgen_state, np.npy_intp cnt, double *out) + +.. c:function:: double random_standard_exponential_zig(bitgen_t *bitgen_state) + +.. c:function:: void random_standard_exponential_zig_fill(bitgen_t *bitgen_state, np.npy_intp cnt, double *out) + +.. c:function:: double random_standard_normal(bitgen_t* bitgen_state) + +.. c:function:: void random_standard_normal_fill(bitgen_t *bitgen_state, np.npy_intp count, double *out) + +.. c:function:: void random_standard_normal_fill_f(bitgen_t *bitgen_state, np.npy_intp count, float *out) + +.. c:function:: double random_standard_gamma(bitgen_t *bitgen_state, double shape) + +.. c:function:: float random_standard_uniform_f(bitgen_t *bitgen_state) + +.. c:function:: void random_standard_uniform_fill_f(bitgen_t* bitgen_state, np.npy_intp cnt, float *out) + +.. c:function:: float random_standard_exponential_f(bitgen_t *bitgen_state) + +.. c:function:: float random_standard_exponential_zig_f(bitgen_t *bitgen_state) + +.. c:function:: void random_standard_exponential_fill_f(bitgen_t *bitgen_state, np.npy_intp cnt, float *out) + +.. c:function:: void random_standard_exponential_zig_fill_f(bitgen_t *bitgen_state, np.npy_intp cnt, float *out) + +.. c:function:: float random_standard_normal_f(bitgen_t* bitgen_state) + +.. c:function:: float random_standard_gamma_f(bitgen_t *bitgen_state, float shape) + +.. c:function:: double random_normal(bitgen_t *bitgen_state, double loc, double scale) + +.. c:function:: double random_gamma(bitgen_t *bitgen_state, double shape, double scale) + +.. c:function:: float random_gamma_f(bitgen_t *bitgen_state, float shape, float scale) + +.. c:function:: double random_exponential(bitgen_t *bitgen_state, double scale) + +.. c:function:: double random_uniform(bitgen_t *bitgen_state, double lower, double range) +.. c:function:: double random_beta(bitgen_t *bitgen_state, double a, double b) + +.. c:function:: double random_chisquare(bitgen_t *bitgen_state, double df) + +.. c:function:: double random_f(bitgen_t *bitgen_state, double dfnum, double dfden) + +.. c:function:: double random_standard_cauchy(bitgen_t *bitgen_state) + +.. c:function:: double random_pareto(bitgen_t *bitgen_state, double a) + +.. c:function:: double random_weibull(bitgen_t *bitgen_state, double a) + +.. c:function:: double random_power(bitgen_t *bitgen_state, double a) + +.. c:function:: double random_laplace(bitgen_t *bitgen_state, double loc, double scale) + +.. c:function:: double random_gumbel(bitgen_t *bitgen_state, double loc, double scale) + +.. c:function:: double random_logistic(bitgen_t *bitgen_state, double loc, double scale) + +.. c:function:: double random_lognormal(bitgen_t *bitgen_state, double mean, double sigma) + +.. c:function:: double random_rayleigh(bitgen_t *bitgen_state, double mode) + +.. c:function:: double random_standard_t(bitgen_t *bitgen_state, double df) + +.. c:function:: double random_noncentral_chisquare(bitgen_t *bitgen_state, double df, + double nonc) +.. c:function:: double random_noncentral_f(bitgen_t *bitgen_state, double dfnum, + double dfden, double nonc) +.. c:function:: double random_wald(bitgen_t *bitgen_state, double mean, double scale) + +.. c:function:: double random_vonmises(bitgen_t *bitgen_state, double mu, double kappa) + +.. c:function:: double random_triangular(bitgen_t *bitgen_state, double left, double mode, + double right) + +.. c:function:: int64_t random_poisson(bitgen_t *bitgen_state, double lam) + +.. c:function:: int64_t random_negative_binomial(bitgen_t *bitgen_state, double n, double p) + +.. c:function:: int64_t random_binomial(bitgen_t *bitgen_state, double p, int64_t n, binomial_t *binomial) + +.. c:function:: int64_t random_logseries(bitgen_t *bitgen_state, double p) + +.. c:function:: int64_t random_geometric_search(bitgen_t *bitgen_state, double p) + +.. c:function:: int64_t random_geometric_inversion(bitgen_t *bitgen_state, double p) + +.. c:function:: int64_t random_geometric(bitgen_t *bitgen_state, double p) + +.. c:function:: int64_t random_zipf(bitgen_t *bitgen_state, double a) + +.. c:function:: int64_t random_hypergeometric(bitgen_t *bitgen_state, int64_t good, int64_t bad, + int64_t sample) + +.. c:function:: uint64_t random_interval(bitgen_t *bitgen_state, uint64_t max) + +.. c:function:: void random_multinomial(bitgen_t *bitgen_state, int64_t n, int64_t *mnix, + double *pix, np.npy_intp d, binomial_t *binomial) + +.. c:function:: int random_mvhg_count(bitgen_t *bitgen_state, + int64_t total, + size_t num_colors, int64_t *colors, + int64_t nsample, + size_t num_variates, int64_t *variates) + +.. c:function:: void random_mvhg_marginals(bitgen_t *bitgen_state, + int64_t total, + size_t num_colors, int64_t *colors, + int64_t nsample, + size_t num_variates, int64_t *variates) + +Generate a single integer + +.. c:function:: int64_t random_positive_int64(bitgen_t *bitgen_state) + +.. c:function:: int32_t random_positive_int32(bitgen_t *bitgen_state) + +.. c:function:: int64_t random_positive_int(bitgen_t *bitgen_state) + +.. c:function:: uint64_t random_uint(bitgen_t *bitgen_state) + + +Generate random uint64 numbers in closed interval [off, off + rng]. + +.. c:function:: uint64_t random_bounded_uint64(bitgen_t *bitgen_state, + uint64_t off, uint64_t rng, + uint64_t mask, bint use_masked) + + diff --git a/doc/source/reference/random/examples/cython/extending.pyx b/doc/source/reference/random/examples/cython/extending.pyx new file mode 100644 index 000000000..0cfbc146f --- /dev/null +++ b/doc/source/reference/random/examples/cython/extending.pyx @@ -0,0 +1,4 @@ +extending.pyx +------------- + +.. include:: ../../../../../../numpy/random/examples/extending.pyx diff --git a/doc/source/reference/random/examples/cython/extending.pyx.rst b/doc/source/reference/random/examples/cython/extending.pyx.rst new file mode 100644 index 000000000..bc31488d7 --- /dev/null +++ b/doc/source/reference/random/examples/cython/extending.pyx.rst @@ -0,0 +1,5 @@ +extending.pyx +------------- + +.. literalinclude:: ../../../../../../numpy/random/examples/cython/extending.pyx + :language: cython diff --git a/doc/source/reference/random/examples/cython/extending_distributions.pyx.rst b/doc/source/reference/random/examples/cython/extending_distributions.pyx.rst new file mode 100644 index 000000000..a1bb01f45 --- /dev/null +++ b/doc/source/reference/random/examples/cython/extending_distributions.pyx.rst @@ -0,0 +1,5 @@ +extending_distributions.pyx +--------------------------- + +.. literalinclude:: ../../../../../../numpy/random/examples/cython/extending_distributions.pyx + :language: cython diff --git a/doc/source/reference/random/examples/cython/index.rst b/doc/source/reference/random/examples/cython/index.rst new file mode 100644 index 000000000..9c5da9559 --- /dev/null +++ b/doc/source/reference/random/examples/cython/index.rst @@ -0,0 +1,9 @@ + +Extending `numpy.random` via Cython +----------------------------------- + + +.. toctree:: + setup.py.rst + extending.pyx + extending_distributions.pyx diff --git a/doc/source/reference/random/examples/cython/setup.py.rst b/doc/source/reference/random/examples/cython/setup.py.rst new file mode 100644 index 000000000..381b45fd4 --- /dev/null +++ b/doc/source/reference/random/examples/cython/setup.py.rst @@ -0,0 +1,5 @@ +setup.py +-------- + +.. literalinclude:: ../../../../../../numpy/random/examples/cython/setup.py + :language: python diff --git a/doc/source/reference/random/examples/numba.rst b/doc/source/reference/random/examples/numba.rst new file mode 100644 index 000000000..a780afde7 --- /dev/null +++ b/doc/source/reference/random/examples/numba.rst @@ -0,0 +1,5 @@ +Extending via Numba +------------------- + +.. literalinclude:: ../../../../../numpy/random/examples/numba/extending.py + :language: python diff --git a/doc/source/reference/random/examples/numba_cffi.rst b/doc/source/reference/random/examples/numba_cffi.rst new file mode 100644 index 000000000..ad4767a7a --- /dev/null +++ b/doc/source/reference/random/examples/numba_cffi.rst @@ -0,0 +1,5 @@ +Extending via Numba and CFFI +---------------------------- + +.. literalinclude:: ../../../../../numpy/random/examples/numba/extending_distributions.py + :language: python diff --git a/doc/source/reference/random/extending.rst b/doc/source/reference/random/extending.rst index 22f9cb7e4..c63fb1a1b 100644 --- a/doc/source/reference/random/extending.rst +++ b/doc/source/reference/random/extending.rst @@ -12,128 +12,42 @@ Numba Numba can be used with either CTypes or CFFI. The current iteration of the BitGenerators all export a small set of functions through both interfaces. -This example shows how numba can be used to produce Box-Muller normals using +This example shows how numba can be used to produce gaussian samples using a pure Python implementation which is then compiled. The random numbers are provided by ``ctypes.next_double``. -.. code-block:: python - - from numpy.random import PCG64 - import numpy as np - import numba as nb - - x = PCG64() - f = x.ctypes.next_double - s = x.ctypes.state - state_addr = x.ctypes.state_address - - def normals(n, state): - out = np.empty(n) - for i in range((n+1)//2): - x1 = 2.0*f(state) - 1.0 - x2 = 2.0*f(state) - 1.0 - r2 = x1*x1 + x2*x2 - while r2 >= 1.0 or r2 == 0.0: - x1 = 2.0*f(state) - 1.0 - x2 = 2.0*f(state) - 1.0 - r2 = x1*x1 + x2*x2 - g = np.sqrt(-2.0*np.log(r2)/r2) - out[2*i] = g*x1 - if 2*i+1 < n: - out[2*i+1] = g*x2 - return out - - # Compile using Numba - print(normals(10, s).var()) - # Warm up - normalsj = nb.jit(normals, nopython=True) - # Must use state address not state with numba - normalsj(1, state_addr) - %timeit normalsj(1000000, state_addr) - print('1,000,000 Box-Muller (numba/PCG64) randoms') - %timeit np.random.standard_normal(1000000) - print('1,000,000 Box-Muller (NumPy) randoms') - +.. literalinclude:: ../../../../numpy/random/examples/numba/extending.py + :language: python + :end-before: example 2 Both CTypes and CFFI allow the more complicated distributions to be used -directly in Numba after compiling the file distributions.c into a DLL or so. -An example showing the use of a more complicated distribution is in the -examples folder. +directly in Numba after compiling the file distributions.c into a ``DLL`` or +``so``. An example showing the use of a more complicated distribution is in +the `examples` section below. -.. _randomgen_cython: +.. _random_cython: Cython ====== Cython can be used to unpack the ``PyCapsule`` provided by a BitGenerator. -This example uses `~pcg64.PCG64` and -``random_gauss_zig``, the Ziggurat-based generator for normals, to fill an -array. The usual caveats for writing high-performance code using Cython -- -removing bounds checks and wrap around, providing array alignment information --- still apply. - -.. code-block:: cython - - import numpy as np - cimport numpy as np - cimport cython - from cpython.pycapsule cimport PyCapsule_IsValid, PyCapsule_GetPointer - from numpy.random.common cimport * - from numpy.random.distributions cimport random_gauss_zig - from numpy.random import PCG64 - - - @cython.boundscheck(False) - @cython.wraparound(False) - def normals_zig(Py_ssize_t n): - cdef Py_ssize_t i - cdef bitgen_t *rng - cdef const char *capsule_name = "BitGenerator" - cdef double[::1] random_values - - x = PCG64() - capsule = x.capsule - if not PyCapsule_IsValid(capsule, capsule_name): - raise ValueError("Invalid pointer to anon_func_state") - rng = <bitgen_t *> PyCapsule_GetPointer(capsule, capsule_name) - random_values = np.empty(n) - # Best practice is to release GIL and acquire the lock - with x.lock, nogil: - for i in range(n): - random_values[i] = random_gauss_zig(rng) - randoms = np.asarray(random_values) - return randoms +This example uses `PCG64` and the example from above. The usual caveats +for writing high-performance code using Cython -- removing bounds checks and +wrap around, providing array alignment information -- still apply. + +.. literalinclude:: ../../../../numpy/random/examples/cython/extending_distributions.pyx + :language: cython + :end-before: example 2 The BitGenerator can also be directly accessed using the members of the basic RNG structure. -.. code-block:: cython - - @cython.boundscheck(False) - @cython.wraparound(False) - def uniforms(Py_ssize_t n): - cdef Py_ssize_t i - cdef bitgen_t *rng - cdef const char *capsule_name = "BitGenerator" - cdef double[::1] random_values - - x = PCG64() - capsule = x.capsule - # Optional check that the capsule if from a BitGenerator - if not PyCapsule_IsValid(capsule, capsule_name): - raise ValueError("Invalid pointer to anon_func_state") - # Cast the pointer - rng = <bitgen_t *> PyCapsule_GetPointer(capsule, capsule_name) - random_values = np.empty(n) - with x.lock, nogil: - for i in range(n): - # Call the function - random_values[i] = rng.next_double(rng.state) - randoms = np.asarray(random_values) - return randoms +.. literalinclude:: ../../../../numpy/random/examples/cython/extending_distributions.pyx + :language: cython + :start-after: example 2 These functions along with a minimal setup file are included in the -examples folder. +`examples` folder, ``numpy.random.examples``. New Basic RNGs ============== @@ -163,3 +77,11 @@ the next 64-bit unsigned integer function if not needed. Functions inside .. code-block:: c bitgen_state->next_uint64(bitgen_state->state) + +Examples +======== + +.. toctree:: + Numba <examples/numba> + CFFI + Numba <examples/numba_cffi> + Cython <examples/cython/index> diff --git a/doc/source/reference/random/index.rst b/doc/source/reference/random/index.rst index 15c161244..d28646df9 100644 --- a/doc/source/reference/random/index.rst +++ b/doc/source/reference/random/index.rst @@ -32,7 +32,7 @@ instance's methods are imported into the numpy.random namespace, see Quick Start ----------- -By default, `~Generator` uses bits provided by `~pcg64.PCG64` which +By default, `~Generator` uses bits provided by `PCG64` which has better statistical properties than the legacy mt19937 random number generator in `~.RandomState`. @@ -155,7 +155,7 @@ What's New or Different (`~.PCG64.ctypes`) and CFFI (`~.PCG64.cffi`). This allows the bit generators to be used in numba. * The bit generators can be used in downstream projects via - :ref:`Cython <randomgen_cython>`. + :ref:`Cython <random_cython>`. * `~.Generator.integers` is now the canonical way to generate integer random numbers from a discrete uniform distribution. The ``rand`` and ``randn`` methods are only available through the legacy `~.RandomState`. @@ -199,7 +199,8 @@ Features Multithreaded Generation <multithreading> new-or-different Comparing Performance <performance> - extending + c-api + Examples of using Numba, Cython, CFFI <extending> Original Source ~~~~~~~~~~~~~~~ |