| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, saving format version 1 would align to 16 bytes,
and saving version 2 would align improperly (bug #8085).
Alignment is now always at least 64 bytes in either version,
which supports memory mapping of the saved files on Linux,
where mmap() offset must be a multiple of the page size.
Why 64 bytes? Simply because we don't know of a case where
more is needed. AVX alignment is 32 bytes; AVX-512 is 64.
Fixes #8085, closes #8598.
|
| |
|
|
|
|
|
|
| |
Since we only need to support python 2, we can remove any case where we just
pass a single string literal and use the b prefix instead.
What we can't do is transform asbytes("tests %d" % num), because %-formatting
fails on bytes in python 3.x < 3.5.
|
| | |
|
| |
|
|
| |
with empty dtypes. See #6430
|
| |
|
|
|
| |
Prevents overflow errors for large arrays on systems
where the default int type is int32.
|
| | |
|
| | |
|
| |
|
|
| |
Previously the passed in version was ignored
|
| |
|
|
|
|
|
| |
Previously read_array_header_1_0 & read_array_header_2_0 were not
returning the documented results.
Closes #5602
|
| | |
|
| |
|
|
|
|
| |
Add pickle compatibility flags to numpy.save and numpy.load. Allow only
combinations that cannot corrupt binary data in Numpy arrays. Use the
same default values as Python pickle.
|
| |
|
|
| |
As discussed on gh-5413. Version 2.0 was added in gh-4765.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
This only happens when C long is smaller than npy_intp, in particular,
on 64 bit windows. The 'L' suffix is not needed as long as safe_eval
is used to recover the values, and the long intrinsic causes problems
for python3.
The python3 read side has been fixed, but we might as well fix the
problem at the root as well.
|
| |
|
|
|
|
|
|
|
|
| |
The Python2 generated file had long integer literals like '1L' that
broke in Python3. The fix here is to filter out the 'L' and let
safe_eval take care of the integer type in converting the string.
The fix here comes from Nathaniel Smith with a few added fixups.
Closes #5170.
|
| |
|
|
| |
The rules enforced are the same as those used for scipy.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
The new format only increases the header length field to 4 bytes.
allows storing structured arrays with a large number of named columns.
The dtype serialization for these can exceed the 2 byte header length
field required by the 1.0 format.
The generic functions automatically use the 2.0 format if the to be
stored data requires it. To avoid unintentional incompatibilies a
UserWarning is emitted when this happens.
If the format is not required the more compatible 1.0 format is used.
Closes gh-4690
|
| |
|
|
|
|
|
| |
tostring returns bytes which are not equal to string, so provide a
tobytes function alias.
tostring does not emit a deprecation warning yet so rdepends do not need
to check two names to support older versions of numpy without warnings.
|
| |
|
|
|
|
|
|
|
| |
This wrapper function is used everywhere in format.py now to ensure to
correctly the handle the case when fp.read returns fewer bytes than
requested.
Also added a test for the orignal bug, loading an array of size more
than 64K from a zip file.
|
| |
|
|
|
|
| |
Issue 4093: Improved the handling of the case when the amount of data
read by fp.read is smaller than dtype.itemsize. Also changed the test
code to make sure this case is tested.
|
| | |
|
| | |
|
| |
|
|
|
|
| |
In Python 2.6.x the number of bytes read from a zip file is 2^16, which
is less than the 2^18 requested by lib.format.py. This change handles
the case where fp.read() returns fewer than the requested number of bytes.
|
| | |
|
| |
|
|
| |
one read
|
| |
|
|
|
|
|
| |
Run the 2to3 ws_comma fixer on *.py files. Some lines are now too long
and will need to be broken at some point. OTOH, some lines were already
too long and need to be broken at some point. Now seems as good a time
as any to do this with open PRs at a minimum.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
Maximum data size limitations in the crc32 module cause errors when
reading more than 2 ** 32 bytes from gzip streams. Work around this
issue when reading large arrays from npz files by chunking reads to
256mb.
This appears to resolve bug #2922.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The idioms fixer makes the following replacements.
1) int <- bool
2) comparison or identity of types <- isinstance
3) a.sort() <- sorted(a)
There were two problems that needed to be dealt with after the
application of the fixer. First, the replacement of comparison or
identity of types by isinstance was not always correct. The isinstance
function returns true for subtypes whereas many of the places where the
fixer made a substitution needed to check for exact type equality.
Second, the sorted function was applied to arrays, but because it treats
them as iterators and constructs a sorted list from the result, that is
the wrong thing to do.
Closes #3062.
|
| |
|
|
|
|
|
|
|
|
|
| |
The basestring class is not defined in Python 3 and the fixer replaces
it with str. In order to have a common code base we define basestring in
numpy/compat/py3k.py to be str when the Python version is >= 3,
otherwise basestring and import it where needed. That works for most
cases, but there are a few files where the version dependent define
needs to be in the file.
Closes #3042.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The numliterals fixer replaces the old style octal number like '01' by
'0o1' removes the 'L' suffix.
Octal values were previously mistakenly specified in some dates, those
uses have been corrected by removing the leading zeros.
Simply Removing the 'L' suffix should not be a problem, but in some
testing code it looks neccesary, so in those places the Python long
constructor is used instead.
The 'long' type is no longer defined in Python 3. Because we need to
have it defined for Python 2 it is added to numpy/compat/np3k.py where
it is defined as 'int' for Python 3 and 'long' for Python 2. The `long`
fixer then needs to be skipped so that it doesn't undo the good work.
Closes #3074, #3067.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In Python3 `dict.items()`, `dict.keys()`, and `dict.values()` are
iterators. This causes problems when a list is needed so the 2to3 fixer
explicitly constructs a list when is finds on of those functions.
However, that is usually not necessary, so a lot of the work here has
been cleaning up those places where the fix is not needed. The big
exception to that is the `numpy/f2py/crackfortran.py` file. The code
there makes extensive use of loops that modify the contents of the
dictionary being looped through, which raises an error. That together
with the obscurity of the code in that file made it safest to let the
`dict` fixer do its worst.
Closes #3050.
|
| |
|
|
|
|
|
| |
Add `print_function` to all `from __future__ import ...` statements
and use the python3 print function syntax everywhere.
Closes #3078.
|
| |\
| |
| | |
2to3: Apply `imports` fixer.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The `imports` fixer deals with the standard packages that have been
renamed, removed, or methods that have moved.
cPickle -- removed, use pickle
commands -- removed, getoutput, getstatusoutput moved to subprocess
urlparse -- removed, urlparse moved to urllib.parse
cStringIO -- removed, use StringIO or io.StringIO
copy_reg -- renamed copyreg
_winreg -- renamed winreg
ConfigParser -- renamed configparser
__builtin__ -- renamed builtins
In the case of `cPickle`, it is imported as `pickle` when python < 3 and
performance may be a consideration, but otherwise plain old `pickle` is
used.
Dealing with `StringIO` is a bit tricky. There is an `io.StringIO`
function in the `io` module, available since Python 2.6, but it expects
unicode whereas `StringIO.StringIO` expects ascii. The Python 3
equivalent is then `io.BytesIO`. What I have done here is used BytesIO
for anything that is emulating a file for testing purposes. That is more
explicit than using a redefined StringIO as was done before we dropped
support for Python 2.4 and 2.5.
Closes #3180.
|
| |\ \
| |/
|/| |
DOC: Formatting fixes using regex
|
| | | |
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The new import `absolute_import` is added the `from __future__ import`
statement and The 2to3 `import` fixer is run to make the imports
compatible. There are several things that need to be dealt with to make
this work.
1) Files meant to be run as scripts run in a different environment than
files imported as part of a package, and so changes to those files need
to be skipped. The affected script files are:
* all setup.py files
* numpy/core/code_generators/generate_umath.py
* numpy/core/code_generators/generate_numpy_api.py
* numpy/core/code_generators/generate_ufunc_api.py
2) Some imported modules are not available as they are created during
the build process and consequently 2to3 is unable to handle them
correctly. Files that import those modules need a bit of extra work.
The affected files are:
* core/__init__.py,
* core/numeric.py,
* core/_internal.py,
* core/arrayprint.py,
* core/fromnumeric.py,
* numpy/__init__.py,
* lib/npyio.py,
* lib/function_base.py,
* fft/fftpack.py,
* random/__init__.py
Closes #3172
|
| |
|
|
|
|
|
|
| |
This should be harmless, as we already are division clean. However,
placement of this import takes some care. In the future a script
can be used to append new features without worry, at least until
such time as it exceeds a single line. Having that ability will
make it easier to deal with absolute imports and printing updates.
|
| |
|
|
| |
Example: except ValueError,msg: -> except ValueError as msg:
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
| |
of arr.data (required for Py3 compatibility)
The issue is that when passing a buffer object to Python's
io.BufferedWriter.write, it will try to obtain the buffer using PyBUF_ND
| PyBUF_C_CONTIGUOUS. This will fail for strided arrays -- seems to be
an issue in Python, as it probably should try to obtain a SIMPLE buffer.
|
| | |
|
| | |
|
| | |
|
| | |
|
