summaryrefslogtreecommitdiff
path: root/doc/source/reference/random
diff options
context:
space:
mode:
authorKevin Sheppard <kevin.k.sheppard@gmail.com>2019-05-24 10:41:58 +0100
committermattip <matti.picus@gmail.com>2019-05-27 22:58:35 +0300
commit58c0e72854c3f79d3d165d74f2dc721815a38b57 (patch)
tree3a8fff1f2c1a073248c81e53ba8d3a0367c00907 /doc/source/reference/random
parent3db5a7736cf26db59817eb8939b042ae18c482fa (diff)
downloadnumpy-58c0e72854c3f79d3d165d74f2dc721815a38b57.tar.gz
Revert "MAINT: Implement API changes for randomgen-derived code"
This reverts commit 17e0070df93f4262908f884dca4b08cb7d0bba7f.
Diffstat (limited to 'doc/source/reference/random')
-rw-r--r--doc/source/reference/random/bit_generators/index.rst8
-rw-r--r--doc/source/reference/random/bit_generators/pcg32.rst34
-rw-r--r--doc/source/reference/random/bit_generators/pcg64.rst34
-rw-r--r--doc/source/reference/random/index.rst1
-rw-r--r--doc/source/reference/random/legacy.rst4
-rw-r--r--doc/source/reference/random/multithreading.rst18
-rw-r--r--doc/source/reference/random/parallel.rst29
-rw-r--r--doc/source/reference/random/performance.py38
-rw-r--r--doc/source/reference/random/performance.rst68
9 files changed, 158 insertions, 76 deletions
diff --git a/doc/source/reference/random/bit_generators/index.rst b/doc/source/reference/random/bit_generators/index.rst
index 5fcb6ce37..3a9294bfb 100644
--- a/doc/source/reference/random/bit_generators/index.rst
+++ b/doc/source/reference/random/bit_generators/index.rst
@@ -5,11 +5,11 @@ Bit Generators
.. currentmodule:: numpy.random
-The random values produced by :class:`~Generator`
+The random values produced by :class:`~Generator`
orignate in a BitGenerator. The BitGenerators do not directly provide
random numbers and only contains methods used for seeding, getting or
-setting the state, jumping or advancing the state, and for accessing
-low-level wrappers for consumption by code that can efficiently
+setting the state, jumping or advancing the state, and for accessing
+low-level wrappers for consumption by code that can efficiently
access the functions provided, e.g., `numba <https://numba.pydata.org>`_.
Stable RNGs
@@ -20,6 +20,8 @@ Stable RNGs
DSFMT <dsfmt>
MT19937 <mt19937>
+ PCG32 <pcg32>
+ PCG64 <pcg64>
Philox <philox>
ThreeFry <threefry>
Xoshiro256** <xoshiro256>
diff --git a/doc/source/reference/random/bit_generators/pcg32.rst b/doc/source/reference/random/bit_generators/pcg32.rst
new file mode 100644
index 000000000..faaccaf9b
--- /dev/null
+++ b/doc/source/reference/random/bit_generators/pcg32.rst
@@ -0,0 +1,34 @@
+Parallel Congruent Generator (32-bit, PCG32)
+--------------------------------------------
+
+.. module:: numpy.random.pcg32
+
+.. currentmodule:: numpy.random.pcg32
+
+.. autoclass:: PCG32
+ :exclude-members:
+
+Seeding and State
+=================
+
+.. autosummary::
+ :toctree: generated/
+
+ ~PCG32.seed
+ ~PCG32.state
+
+Parallel generation
+===================
+.. autosummary::
+ :toctree: generated/
+
+ ~PCG32.advance
+ ~PCG32.jumped
+
+Extending
+=========
+.. autosummary::
+ :toctree: generated/
+
+ ~PCG32.cffi
+ ~PCG32.ctypes
diff --git a/doc/source/reference/random/bit_generators/pcg64.rst b/doc/source/reference/random/bit_generators/pcg64.rst
new file mode 100644
index 000000000..fa719cea4
--- /dev/null
+++ b/doc/source/reference/random/bit_generators/pcg64.rst
@@ -0,0 +1,34 @@
+Parallel Congruent Generator (64-bit, PCG64)
+--------------------------------------------
+
+.. module:: numpy.random.pcg64
+
+.. currentmodule:: numpy.random.pcg64
+
+.. autoclass:: PCG64
+ :exclude-members:
+
+Seeding and State
+=================
+
+.. autosummary::
+ :toctree: generated/
+
+ ~PCG64.seed
+ ~PCG64.state
+
+Parallel generation
+===================
+.. autosummary::
+ :toctree: generated/
+
+ ~PCG64.advance
+ ~PCG64.jumped
+
+Extending
+=========
+.. autosummary::
+ :toctree: generated/
+
+ ~PCG64.cffi
+ ~PCG64.ctypes
diff --git a/doc/source/reference/random/index.rst b/doc/source/reference/random/index.rst
index 45b5ed2c0..0b8145735 100644
--- a/doc/source/reference/random/index.rst
+++ b/doc/source/reference/random/index.rst
@@ -168,6 +168,7 @@ The included BitGenerators are:
`Random123`_ page for more details about this class of bit generators.
.. _`dSFMT authors' page`: http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/
+.. _`PCG author's page`: http://www.pcg-random.org/
.. _`xorshift, xoroshiro and xoshiro authors' page`: http://xoroshiro.di.unimi.it/
.. _`Random123`: https://www.deshawresearch.com/resources_random123.html
diff --git a/doc/source/reference/random/legacy.rst b/doc/source/reference/random/legacy.rst
index c5b92f1bb..d9391e9e2 100644
--- a/doc/source/reference/random/legacy.rst
+++ b/doc/source/reference/random/legacy.rst
@@ -19,8 +19,8 @@ are produced in pairs. It is important to use
.. warning::
:class:`~randomgen.legacy.LegacyGenerator` only contains functions
- that have changed. Since it does not contain other functions, it
- is not direclty possible to replace :class:`~numpy.random.RandomState`.
+ that have changed. Since it does not contain other functions, it
+ is not directly possible to replace :class:`~numpy.random.RandomState`.
In order to full replace :class:`~numpy.random.RandomState`, it is
necessary to use both :class:`~randomgen.legacy.LegacyGenerator`
and :class:`~randomgen.generator.RandomGenerator` both driven
diff --git a/doc/source/reference/random/multithreading.rst b/doc/source/reference/random/multithreading.rst
index 871425e6d..7ce90af99 100644
--- a/doc/source/reference/random/multithreading.rst
+++ b/doc/source/reference/random/multithreading.rst
@@ -21,14 +21,14 @@ seed will produce the same outputs.
import multiprocessing
import concurrent.futures
import numpy as np
-
+
class MultithreadedRNG(object):
def __init__(self, n, seed=None, threads=None):
rg = Xoshiro256(seed)
if threads is None:
threads = multiprocessing.cpu_count()
self.threads = threads
-
+
self._random_generators = [rg]
last_rg = rg
for _ in range(0, threads-1):
@@ -40,21 +40,21 @@ seed will produce the same outputs.
self.executor = concurrent.futures.ThreadPoolExecutor(threads)
self.values = np.empty(n)
self.step = np.ceil(n / threads).astype(np.int)
-
+
def fill(self):
def _fill(random_state, out, first, last):
random_state.standard_normal(out=out[first:last])
-
+
futures = {}
for i in range(self.threads):
- args = (_fill,
+ args = (_fill,
self._random_generators[i],
- self.values,
- i * self.step,
+ self.values,
+ i * self.step,
(i + 1) * self.step)
futures[self.executor.submit(*args)] = i
concurrent.futures.wait(futures)
-
+
def __del__(self):
self.executor.shutdown(False)
@@ -80,7 +80,7 @@ the time required to generate using a single thread.
In [4]: print(mrng.threads)
...: %timeit mrng.fill()
-
+
4
32.8 ms ± 2.71 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
diff --git a/doc/source/reference/random/parallel.rst b/doc/source/reference/random/parallel.rst
index 40f0bce63..ffbaea62b 100644
--- a/doc/source/reference/random/parallel.rst
+++ b/doc/source/reference/random/parallel.rst
@@ -12,20 +12,20 @@ or distributed).
Independent Streams
-------------------
-:class:`~threefry.ThreeFry` and :class:`~philox.Philox` support independent
-streams. This example shows how many streams can be created by passing in
-different index values in the second input while using the same seed in the
-first.
+:class:`~pcg64.PCG64`, :class:`~threefry.ThreeFry`
+and :class:`~philox.Philox` support independent streams. This
+example shows how many streams can be created by passing in different index
+values in the second input while using the same seed in the first.
.. code-block:: python
from numpy.random.entropy import random_entropy
- from numpy.random import ThreeFry
+ from numpy.random import PCG64
entropy = random_entropy(4)
# 128-bit number as a seed
seed = sum([int(entropy[i]) * 2 ** (32 * i) for i in range(4)])
- streams = [ThreeFry(seed, stream) for stream in range(10)]
+ streams = [PCG64(seed, stream) for stream in range(10)]
:class:`~philox.Philox` and :class:`~threefry.ThreeFry` are
@@ -68,6 +68,8 @@ are listed below.
+-----------------+-------------------------+-------------------------+-------------------------+
| MT19937 | :math:`2^{19937}` | :math:`2^{128}` | 32 |
+-----------------+-------------------------+-------------------------+-------------------------+
+| PCG64 | :math:`2^{128}` | :math:`2^{64}` | 64 |
++-----------------+-------------------------+-------------------------+-------------------------+
| Philox | :math:`2^{256}` | :math:`2^{128}` | 64 |
+-----------------+-------------------------+-------------------------+-------------------------+
| ThreeFry | :math:`2^{256}` | :math:`2^{128}` | 64 |
@@ -96,8 +98,9 @@ overlap.
Advance
*******
``advance`` can be used to jump the state an arbitrary number of steps, and so
-is a more general approach than ``jumped``. :class:`~threefry.ThreeFry` and
-:class:`~philox.Philox` support ``advance``, and since these also support
+is a more general approach than ``jumped``. :class:`~pcg64.PCG64`,
+:class:`~threefry.ThreeFry` and :class:`~philox.Philox`
+support ``advance``, and since these also support
independent streams, it is not usually necessary to use ``advance``.
Advancing a BitGenerator updates the underlying state as-if a given number of
@@ -116,21 +119,21 @@ This occurs for two reasons:
Advancing the BitGenerator state resets any pre-computed random numbers. This
is required to ensure exact reproducibility.
-This example uses ``advance`` to advance a :class:`~threefry.ThreeFry`
+This example uses ``advance`` to advance a :class:`~pcg64.PCG64`
generator 2 ** 127 steps to set a sequence of random number generators.
.. code-block:: python
- from numpy.random import ThreeFry
- bit_generator = ThreeFry()
- bit_generator_copy = ThreeFry()
+ from numpy.random import PCG64
+ bit_generator = PCG64()
+ bit_generator_copy = PCG64()
bit_generator_copy.state = bit_generator.state
advance = 2**127
bit_generators = [bit_generator]
for _ in range(9):
bit_generator_copy.advance(advance)
- bit_generator = ThreeFry()
+ bit_generator = PCG64()
bit_generator.state = bit_generator_copy.state
bit_generators.append(bit_generator)
diff --git a/doc/source/reference/random/performance.py b/doc/source/reference/random/performance.py
index bbf17b1d0..54165226e 100644
--- a/doc/source/reference/random/performance.py
+++ b/doc/source/reference/random/performance.py
@@ -1,23 +1,25 @@
from collections import OrderedDict
from timeit import repeat
-import numpy as np
import pandas as pd
-from numpy.random import MT19937, DSFMT, ThreeFry, Philox, Xoshiro256, \
- Xoshiro512
+import numpy as np
+from numpy.random import MT19937, DSFMT, ThreeFry, PCG64, Philox, \
+ Xoshiro256, Xoshiro512
-PRNGS = [DSFMT, MT19937, Philox, ThreeFry, Xoshiro256, Xoshiro512]
+PRNGS = [DSFMT, MT19937, PCG64, Philox, ThreeFry, Xoshiro256, Xoshiro512]
-funcs = {'32-bit Unsigned Ints': 'integers(0, 2**32,size=1000000, dtype="uint32")',
- '64-bit Unsigned Ints': 'integers(0, 2**64,size=1000000, dtype="uint64")',
- 'Uniforms': 'random(size=1000000)',
- 'Normals': 'standard_normal(size=1000000)',
- 'Exponentials': 'standard_exponential(size=1000000)',
- 'Gammas': 'standard_gamma(3.0,size=1000000)',
- 'Binomials': 'binomial(9, .1, size=1000000)',
- 'Laplaces': 'laplace(size=1000000)',
- 'Poissons': 'poisson(3.0, size=1000000)', }
+funcs = OrderedDict()
+integers = 'integers(0, 2**{bits},size=1000000, dtype="uint{bits}")'
+funcs['32-bit Unsigned Ints'] = integers.format(bits=32)
+funcs['64-bit Unsigned Ints'] = integers.format(bits=64)
+funcs['Uniforms'] = 'random(size=1000000)'
+funcs['Normals'] = 'standard_normal(size=1000000)'
+funcs['Exponentials'] = 'standard_exponential(size=1000000)'
+funcs['Gammas'] = 'standard_gamma(3.0,size=1000000)'
+funcs['Binomials'] = 'binomial(9, .1, size=1000000)'
+funcs['Laplaces'] = 'laplace(size=1000000)'
+funcs['Poissons'] = 'poisson(3.0, size=1000000)'
setup = """
from numpy.random import {prng}, Generator
@@ -40,7 +42,7 @@ for prng in PRNGS:
npfuncs = OrderedDict()
npfuncs.update(funcs)
npfuncs['32-bit Unsigned Ints'] = 'randint(2**32,dtype="uint32",size=1000000)'
-npfuncs['64-bit Unsigned Ints'] = 'tomaxint(size=1000000)'
+npfuncs['64-bit Unsigned Ints'] = 'randint(2**64,dtype="uint64",size=1000000)'
setup = """
from numpy.random import RandomState
rg = RandomState()
@@ -51,7 +53,7 @@ for key in npfuncs:
setup.format(prng=prng().__class__.__name__),
number=1, repeat=3)
col[key] = 1000 * min(t)
-table['NumPy'] = pd.Series(col)
+table['RandomState'] = pd.Series(col)
table = pd.DataFrame(table)
table = table.reindex(table.mean(1).sort_values().index)
@@ -59,10 +61,12 @@ order = np.log(table).mean().sort_values().index
table = table.T
table = table.reindex(order)
table = table.T
+table = table.reindex([k for k in funcs], axis=0)
print(table.to_csv(float_format='%0.1f'))
-rel = table.loc[:, ['NumPy']].values @ np.ones((1, table.shape[1])) / table
-rel.pop(rel.columns[0])
+rel = table.loc[:, ['RandomState']].values @ np.ones(
+ (1, table.shape[1])) / table
+rel.pop('RandomState')
rel = rel.T
rel['Overall'] = np.exp(np.log(rel).mean(1))
rel *= 100
diff --git a/doc/source/reference/random/performance.rst b/doc/source/reference/random/performance.rst
index 395744eb8..82efdab60 100644
--- a/doc/source/reference/random/performance.rst
+++ b/doc/source/reference/random/performance.rst
@@ -7,12 +7,13 @@ Performance
Recommendation
**************
-The recommended generator for single use is
-:class:`~.xoshiro256.Xoshiro256`. The recommended generator
-for use in large-scale parallel applications is
+The recommended generator for single use is :class:`~.xoshiro256.Xoshiro256`.
+The recommended generator for use in large-scale parallel applications is
:class:`~.xoshiro512.Xoshiro512` where the `jumped` method is used to advance
the state. For very large scale applications -- requiring 1,000+ independent
-streams -- :class:`~.philox.Philox` is the best choice.
+streams -- is the best choice. For very large scale applications -- requiring
+1,000+ independent streams, :class:`~pcg64.PCG64` or :class:`~.philox.Philox`
+are the best choices.
Timings
*******
@@ -26,24 +27,27 @@ Integer performance has a similar ordering although `dSFMT` is slower since
it generates 53-bit floating point values rather than integer values.
The pattern is similar for other, more complex generators. The normal
-performance of NumPy's MT19937 is much lower than the other since it
-uses the Box-Muller transformation rather than the Ziggurat generator. The
-performance gap for Exponentials is also large due to the cost of computing
-the log function to invert the CDF.
+performance of the legacy :class:`~mtrand.RandomState` generator is much
+lower than the other since it uses the Box-Muller transformation rather
+than the Ziggurat generator. The performance gap for Exponentials is also
+large due to the cost of computing the log function to invert the CDF.
+The column labeled MT19973 is used the same 32-bit generator as
+:class:`~mtrand.RandomState` but produces random values using
+:class:`~generator.Generator`.
.. csv-table::
- :header: ,Xoshiro256**,Xoshiro512**,DSFMT,MT19937,Philox,NumPy,ThreeFry
- :widths: 14,14,14,14,14,14,14,14
+ :header: ,Xoshiro256**,Xoshiro512**,DSFMT,PCG64,MT19937,Philox,RandomState,ThreeFry
+ :widths: 14,14,14,14,14,14,14,14,14
- 32-bit Unsigned Ints,2.6,2.9,3.4,3.2,5.0,3.3,7.6
- 64-bit Unsigned Ints,3.1,4.0,5.6,5.7,6.6,8.1,13.4
- Uniforms,3.7,4.2,3.2,7.4,9.1,8.9,13.5
- Exponentials,4.3,5.3,7.3,8.2,9.7,42.4,14.9
- Normals,8.2,8.9,11.7,13.4,15.0,37.3,18.6
- Binomials,20.0,20.8,19.8,26.4,28.2,26.6,31.0
- Gammas,26.4,28.7,30.4,37.1,38.8,62.9,49.0
- Laplaces,40.2,40.0,39.4,48.7,51.2,47.4,51.4
- Poissons,48.8,51.6,47.8,73.6,82.3,74.0,90.6
+ 32-bit Unsigned Ints,2.6,2.9,3.5,3.2,3.3,4.8,3.2,7.6
+ 64-bit Unsigned Ints,3.3,4.3,5.7,4.8,5.7,6.9,5.7,12.8
+ Uniforms,3.4,4.0,3.2,5.0,7.3,8.0,7.3,12.8
+ Normals,7.9,9.0,11.8,11.3,13.0,13.7,34.4,18.1
+ Exponentials,4.7,5.2,7.4,6.7,7.9,8.6,40.3,14.7
+ Gammas,29.1,27.5,28.5,30.6,34.2,35.1,58.1,47.6
+ Binomials,22.7,23.1,21.1,25.7,27.7,28.4,25.9,32.1
+ Laplaces,38.5,38.1,36.9,41.1,44.5,45.4,46.9,50.2
+ Poissons,46.9,50.9,46.4,58.1,68.4,70.2,86.0,88.2
The next table presents the performance in percentage relative to values
@@ -51,19 +55,19 @@ generated by the legagy generator, `RandomState(MT19937())`. The overall
performance was computed using a geometric mean.
.. csv-table::
- :header: ,Xoshiro256**,Xoshiro512**,DSFMT,MT19937,Philox,ThreeFry
- :widths: 14,14,14,14,14,14,14
-
- 32-bit Unsigned Ints,129,113,98,103,66,44
- 64-bit Unsigned Ints,258,202,145,142,122,61
- Uniforms,244,214,283,121,98,66
- Exponentials,981,796,580,518,436,285
- Normals,453,417,319,278,249,200
- Binomials,133,128,134,101,94,86
- Gammas,238,219,207,170,162,129
- Laplaces,118,118,120,97,93,92
- Poissons,152,144,155,101,90,82
- Overall,233,209,194,152,130,98
+ :header: ,Xoshiro256**,Xoshiro256**,DSFMT,PCG64,MT19937,Philox,ThreeFry
+ :widths: 14,14,14,14,14,14,14,14
+
+ 32-bit Unsigned Ints,124,113,93,100,99,67,43
+ 64-bit Unsigned Ints,174,133,100,118,100,83,44
+ Uniforms,212,181,229,147,100,91,57
+ Normals,438,382,291,304,264,252,190
+ Exponentials,851,770,547,601,512,467,275
+ Gammas,200,212,204,190,170,166,122
+ Binomials,114,112,123,101,93,91,81
+ Laplaces,122,123,127,114,105,103,93
+ Poissons,183,169,185,148,126,123,98
+ Overall,212,194,180,167,145,131,93
.. note::