summaryrefslogtreecommitdiff
path: root/numpy/lib/npyio.py
Commit message (Collapse)AuthorAgeFilesLines
...
* | STY: Slight style changeCharles Harris2021-08-221-1/+1
| |
* | PERF: Optimize loadtxt usecols.Antony Lee2021-08-221-10/+15
|/ | | | | | 7-10% speedup in usecols benchmarks; it appears that even in the single-usecol case, avoiding the iteration over `usecols` more than compensates the cost of the extra function call to usecols_getter.
* MAINT: In loadtxt, inline read_data.Antony Lee2021-08-171-33/+18
| | | | | | No speed difference; the point is to avoid an unnecessary inner generator (which was previously defined quite far away from its point of use).
* ENH: Allow `np.fromregex` to accept `os.PathLike` implementationsBas van Beek2021-08-161-1/+5
|
* Merge pull request #19609 from anntzer/loadtxtstaticdecoderCharles Harris2021-08-111-26/+38
|\ | | | | PERF: In loadtxt, decide once and for all whether decoding is needed.
| * Move loadtxt bytes/str detection much earlier.Antony Lee2021-08-061-14/+21
| |
| * loadtxt: Preconstruct a (lineno, words) iterator to pass to read_data.Antony Lee2021-08-061-21/+20
| | | | | | | | | | Mostly to help later speedups, but also slightly optimizes `len(vals) == 0` into a bool check (in `filter`).
| * PERF: In loadtxt, decide once and for all whether decoding is needed.Antony Lee2021-08-061-6/+12
| | | | | | | | | | | | | | ... and use a single decoder function instead of repeatedly checking the input type (in `_decode_line`). ~5-8% speedup.
* | Merge pull request #19615 from rossbar/rm-deprecated-npyio-fnsCharles Harris2021-08-061-67/+2
|\ \ | | | | | | MAINT: Proposal to expire three deprecated functions in numpy.lib.npyio
| * | Rm numpy.lib.npyio.mafromtxt.Ross Barnowski2021-08-051-29/+1
| | |
| * | Rm numpy.lib.npyio.ndfromtxt.Ross Barnowski2021-08-051-29/+1
| | |
| * | Rm numpy.lib.npyio.loads.Ross Barnowski2021-08-051-10/+1
| | |
* | | Merge pull request #19620 from anntzer/loadtxtconvstrCharles Harris2021-08-061-5/+5
|\ \ \ | | | | | | | | PERF: Simplify some of loadtxt's standard converters.
| * | | PERF: Simplify some of loadtxt's standard converters.Antony Lee2021-08-061-5/+5
| | |/ | |/| | | | | | | | | | | | | | | | Standard converters only ever get called with str inputs (loadtxt performs the required decoding); saving a bunch of runtime typechecks (in `asstr`) results in a 15-20% speedup when loadtxt()ing the corresponding types.
* | | MAINT: Skip a type check in loadtxt when using user converters.Antony Lee2021-08-061-6/+4
|/ / | | | | | | | | | | loadtxt only ever calls converters with strs now, so the type check is unneeded. Skipping the type check may have a tiny performance benefit, but the main point is just code clarity.
* | PERF: Special-case single-converter in loadtxt.Antony Lee2021-08-061-42/+50
| | | | | | | | | | | | | | | | | | | | | | | | ~5-13% speedup: `[*map(conv, items)]` (single converter, which is quite common) is much faster than `[conv(val) for conv, val in zip(converters, vals)]`. `_loadtxt_floatconv` and `fencode` were lifted out so that every "instance" of them is the same, allowing checking for whether there's different converters in use (actually, it looks like two `floatconv`s returned by two separate calls to `_getconv` have the same identity, but we don't need to rely on that.
* | MAINT: In loadtxt, refactor detection of the number of columns. (#19616)Antony Lee2021-08-051-15/+12
|/ | | | | `for... else...` seems more idiomatic than manual iteration and catching StopIteration. Also rename the slightly cryptic `N` to a clearer `ncols`.
* PERF: Speed-up common case of loadtxt()ing non-hex floats. (#19598)Antony Lee2021-08-051-4/+9
| | | | | | | | | | | | | * PERF: Speed-up common case of loadtxt()ing non-hex floats. `python runtests.py --bench bench_io` reports a ~5-10% perf gain. * TST: Add tests to check fromhex not called unintentionally. Adds regression tests to check that the logic surrounding when the default floatconv applies the fromhex conversion does not change. Co-authored-by: Ross Barnowski <rossbar@berkeley.edu>
* Merge pull request #19608 from anntzer/loadtxtspecialpackerMatti Picus2021-08-041-11/+16
|\ | | | | PERF: Specialize loadtxt packer for uniform-dtype data.
| * Specialize loadtxt packer for uniform-dtype data.Antony Lee2021-08-031-11/+16
| | | | | | | | ~15-30% speedup.
* | Merge pull request #19593 from rossbar/loadtxt-supports-iterablesMatti Picus2021-08-041-9/+12
|\ \ | |/ |/| DOC,MAINT: Update wording surrounding `fname` parameter for loadtxt/genfromtxt
| * Undo change to exception type.Ross Barnowski2021-08-031-1/+1
| |
| * DOC: add compat release note.Ross Barnowski2021-08-031-2/+2
| |
| * DOC/MAINT: Update wording re: fname inputs.Ross Barnowski2021-08-021-10/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | Both loadtxt and genfromtxt can read from lists and generators of strings. The genfromtxt docstring captures this, but loadtxt was missing this info. Minor updates to docstrings and exceptions to: - make the wording more accurate. - homogenize the docs/exception messages for loadtxt/genfromtxt. - Make a ValueError in loadtxt a TypeError instead, which is more accurate and aligns with genfromtxt.
* | Merge pull request #19601 from anntzer/loadtxtcommentsCharles Harris2021-08-031-6/+4
|\ \ | | | | | | PERF: Speedup comments handling in loadtxt.
| * | PERF: Speedup comments handling in loadtxt.Antony Lee2021-08-021-6/+4
| |/ | | | | | | | | | | | | | | Regexes are quite slow; using str.split instead improves performance (likely, regexes could be better for somewhat artificial cases where there's a lot of different comment strings). `python runtests.py --bench bench_io` reports a ~5% perf gain.
* | PERF: Avoid using `@recursive`.Antony Lee2021-08-031-51/+58
|/ | | | | | | | | | | | | None of the functions decorated with `@recursive` actually need to close over internal variables, so they can be lifted to become toplevel recursive functions which avoid the need for an `@recursive` decorator. (No change was made to the logic of any of the functions.) As it turns out, the `@recursive` decorator adds a lot of overhead: `python runtests.py --bench bench_io` reports a ~15-20% perf gain(!) for `loadtxt` from this PR. (`_recursive_mask_or` seems less likely to matter performance-wise, but I also lifted it out for good measure... and just deleted the decorator's implementation.)
* DOC: clarify doc re: unsupported keys in savez.Antony Lee2021-07-241-3/+3
| | | | | | | The previous wording ("When saving dictionaries") could be interpreted as refering to calls like `savez("foo.npz", {"a": 1, "b": 2})` (literally "saving a dict to foo.npz"); reword to avoid this possibility of confusion.
* Merge pull request #19090 from default-303/LGTM_alertsCharles Harris2021-05-251-1/+0
|\ | | | | MAINT: removed unused imports listed in LGTM
| * MAINT: removed unused imports listed in LGTMdefault-3032021-05-241-1/+0
| |
* | DOC: Add note to savez about naming variables with keyword `file`.melissawm2021-05-241-0/+4
|/
* Fixes small typos in the genfromtext docstringMatt Hall2021-04-161-5/+5
| | | | | | - `comments`: Added a full stop after 'discarded'. - `names`: changed 'proceeded' to 'preceeded'. - `excludelist`: inserted 'with' because it sounds odd without it; the example makes it clear. - `replace_space`: should strictly be "variable names" or "variables' names". The latter seems fussy so I chose the former.
* Clarify savez documentation around naming of arrays in output fileStefan van der Walt2020-12-061-19/+21
|
* MAINT: Replace `contextlib_nullcontext` with `contextlib.nullcontext`Bas van Beek2020-12-051-3/+3
|
* MAINT: Minor touchups in npyio (#17796)Ross Barnowski2020-11-191-6/+2
| | | | | | | | | | | | | | | * Simplify logic for encoding kwarg in _decode_line. * Remove unnecessary else branch from split_line. * MAINT: rm else branch from loadtxt. * MAINT: re-nest encoding parsing. Co-Authored-By: mattip <matti.picus@gmail.com> * condense return statement. Co-authored-by: mattip <matti.picus@gmail.com>
* MAINT: Chaining exceptions in npyio.pyBijesh Mohan2020-09-161-5/+5
|
* ENH: Allow genfromtxt to unpack structured arrays (#16650)Andrew Eckart2020-09-111-5/+17
| | | | | | | | | | | | | * ENH: Allow genfromtxt to unpack structured arrays genfromtxt failed to transpose output when unpack=True and `dtype` was structured (or None). This patch resolves the issue by returning a list of arrays, as in `loadtxt`. Co-authored-by: Matti Picus <matti.picus@gmail.com> Co-authored-by: Eric Wieser <wieser.eric@gmail.com> Co-authored-by: Sebastian Berg <sebastian@sipsolutions.net> Co-authored-by: Ross Barnowski <rossbar@berkeley.edu>
* Merge pull request #17235 from hugovk/rm-old-codeCharles Harris2020-09-071-38/+8
|\ | | | | MAINT: Remove old PY_VERSION_HEX and sys.version_info code
| * MAINT: Remove old sys.version_info codeHugo2020-09-031-38/+8
| |
* | Merge pull request #17193 from eric-wieser/clean-compatSebastian Berg2020-09-031-1/+1
|\ \ | | | | | | MAINT: Remove some callers of functions in numpy.compat
| * | MAINT: Remove users of `numpy.compat.bytes`Eric Wieser2020-08-311-1/+1
| |/ | | | | | | Some more Python 2 cleanup.
* | MAINT, DOC: move informational files from numpy.doc.*.py to their *.rst ↵Matti Picus2020-09-021-1/+1
|/ | | | | | | | | counterparts (#17222) * DOC: redistribute docstring-only content from numpy/doc * DOC: post-transition clean-up * DOC, MAINT: reskip doctests, fix a few easy ones
* ENH: implement NEP-35's `like=` argument (gh-16935)Peter Andreas Entschev2020-08-281-3/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This PR adds the implementation of NEP-35's like= argument, allowing dispatch of array creation functions with __array_function__ based on a reference array. * ENH: Add like= kwarg via __array_function__ dispatcher to asarray * ENH: Add new function for __array_function__ dispatching from C This new function allows dispatching from C directly, while also implementing the new `like=` argument, requiring only minimal changes to existing array creation functions that need to add support for that argument. * ENH: Add like= support to numpy.array The implementation uses array_implement_c_array_function, thus introducing minimal complexity to the original _array_fromobject code. * BUG: Fix like= dispatcher for np.full * ENH: Remove np.asarray like= dispatcher via Python np.asarray can rely on np.array's C dispatcher instead. * TST: Add some tests for like= argument Tests comprise some of the functions that have been implemented already: * np.array (C dispatcher) * np.asarray (indirect C dispatcher via np.array) * np.full (Python dispatcher) * np.ones (Python dispatcher) * ENH: Remove like= argument during array_implement_array_function * ENH: Add like= kwarg to ones and full * BUG: prevent duplicate removal of `like=` argument * ENH: Make `like=` a keyword-only argument * ENH: Use PyUnicode_InternFromString in arrayfunction_override Replace PyUnicode_FromString by PyUnicode_InternFromString to cache "like" string. * ENH: Check for arrayfunction_override import errors Check and handle errors on importing NumPy's Python functions * BUG: Fix array_implement_c_array_function error handling * ENH: Handle exceptions with C implementation of `like=` * ENH: Add `like=` dispatch for all asarray functions Using Python dispatcher for all of them. Using the C dispatcher directly on the `np.array` call can result in incorrect behavior. Incorrect behavior may happen if the downstream library's implementation is different or if not all keyword arguments are supported. * ENH: Simplify handling of exceptions with `like=` * TST: Add test for exception handling with `like=` * ENH: Add support for `like=` to `np.empty` and `np.zeros` * TST: Add `like=` tests for `np.empty` and `np.zeros` * ENH: Add `like=` to remaining multiarraymodule.c functions Functions are: * np.arange * np.frombuffer * np.fromfile * np.fromiter * np.fromstring * TST: Add tests for multiarraymodule.c functions with like= Functions are: * np.arange * np.frombuffer * np.fromfile * np.fromiter * np.fromstring * ENH: Add `like=` support to more creation functions Support for the following functions is added: * np.eye * np.fromfunction * np.genfromtxt * np.identity * np.loadtxt * np.tri * TST: Add `like=` tests for multiple functions Tests for the following functions are added: * np.eye * np.fromfunction * np.genfromtxt * np.identity * np.loadtxt * np.tri * TST: Reduce code duplication in `like=` tests * DOC: Document `like=` in functions that support it Add documentations for the following functions: * np.array * np.arange * np.asarray * np.asanyarray * np.ascontiguousarray * np.asfortranarray * np.require * np.empty * np.full * np.ones * np.zeros * np.identity * np.eye * np.tri * np.frombuffer * np.fromfile * np.fromiter * np.fromstring * np.loadtxt * np.genfromtxt * ENH: Add `like=` to numpy/__init__.pyi stubs * BUG: Remove duplicate `like=` dispatching in as*array Functions `np.asanyarray`, `np.contiguousarray` and `np.fortranarray` were dispatching both via their definitions and `np.array` calls, the latter should be avoided. * BUG: Fix missing check in array_implement_array_function * BUG: Add missing keyword-only markers in stubs * BUG: Fix duplicate keyword-only marker in array stub * BUG: Fix syntax error in numpy/__init__.pyi * BUG: Fix more syntax errors in numpy/__init__.pyi * ENH: Intern arrayfunction_override strings in multiarraymodule * STY: Add missing brackets to arrayfunction_override.c * MAINT: Remove arrayfunction_override dict check for kwarg * TST: Assert that 'like' is not in TestArrayLike kwargs * MAINT: Rename array_implement_c_array_function(_creation) This is done to be more explicit as to its usage being intended for array creation functions only. * MAINT: Use NotImplemented to indicate fallback to default * TST: Test that results with `like=np.array` are correct * TST: Avoid duplicating MyArray code in TestArrayLike * TST: Don't delete temp file, it may cause issues with Windows * TST: Don't rely on eval in TestArrayLike * TST: Use lambda with StringIO in TestArrayLike * ENH: Avoid unnecessary Py_XDECREF in arrayfunction_override * TST: Make TestArrayLike more readable * ENH: Cleaner error handling in arrayfunction_override * ENH: Simplify array_implement_c_array_function_creation * STY: Add missing spaces to multiarraymodule.c * STY: C99 declaration style in arrayfunction_override.c * ENH: Simplify arrayfunction_override.c further Remove cleanup label from array_implementation_c_array_function, simplifying the code. Fix unitialized variable warning in array_implementation_array_function_internal. * DOC: Use string replacement for `like=` documentation Avoid repeating the full text for the `like=` argument by storing it as a variable and using `replace` on each docstring. * DOC: Update `like=` docstring * TST: Test like= with array not implementing __array_function__ * TST: Add missing asanyarray to TestArrayLike * ENH: Use helper function for like= dispatching Avoid dispatching like= from Python implementation functions to improve their performance. This is achieved by only calling a dispatcher function when like is passed by the users. * ENH: Rename array_function_dispatch kwarg to public_api * BUG: Add accidentally removed decorator for np.eye back * DOC: Add set_array_function_like_doc function The function keeps Python files cleaner and resolve errors when __doc__ is not defined due to PYTHONOPTIMIZE or -OO . * DOC: Add mention to like= kwarg being experimental * TST: Test like= with not implemented downstream function * DOC: Fix like= docstring reference to NEP 35. * ENH: Prevent silent errors if public_api is not callable * ENH: Make set_array_function_like_doc a decorator * ENH: Simplify `_*_with_like` functions * BUG: Fix multiple `like=` dispatching in `require` * MAINT: Remove now unused public_api from array_function_dispatch Co-authored-by: Sebastian Berg <sebastian@sipsolutions.net>
* MAINT: Avoid exception in NpzFile destructor if constructor raises ↵John Zwinck2020-07-211-2/+3
| | | | | | | | | | | | | | | | | BadZipFile (#15604) Previously if you gave an invalid zip file to NpzFile, zipfile_factory would raise BadZipFile and NpzFile.__exit__ would be called, which accessed members which had not yet been set, leading to a confusing second exception like this: zipfile.BadZipFile: File is not a zip file Exception ignored in: <function NpzFile.__del__ at 0x9b8ef0> Traceback (most recent call last): File "numpy/lib/npyio.py", line 230, in __del__ self.close() File "numpy/lib/npyio.py", line 221, in close if self.zip is not None: AttributeError: 'NpzFile' object has no attribute 'zip' This change makes __exit__ safe even when __init__ did not complete.
* DOC: Minor RST formatting. (#16792)Matthias Bussonnier2020-07-091-1/+1
| | | DOC: fixes to capitalization and header lines
* MAINT: Chain exceptions in npyio.py (gh16121)Kerem Hallaç2020-06-301-4/+6
| | | | | | this solution is related to the following issue #15986 Co-authored-by: Ross Barnowski <rossbar@berkeley.edu>
* MAINT: lib: A few PEP-8 fixes.Warren Weckesser2020-06-181-3/+6
|
* MAINT: lib: In loadtxt, move some code out of a try/finally block.Warren Weckesser2020-06-181-5/+8
| | | | | | In loadtxt, there is a try/finally block that ensures that the file is closed if it was opened in the function. Some code that did not need to be in that block was moved up, outside the try/finally block.
* MAINT: lib: In loadtxt, validate ndmin argument earlier.Warren Weckesser2020-06-181-3/+4
| | | | | | Validation of `ndmin` is moved to the beginning of the function, so we don't read the entire file only to raise an exception at the end because of a bad argument.
* MAINT: lib: Move some nested function definitions in loadtxt.Warren Weckesser2020-06-181-61/+70
| | | | | This change moves the nested function definitions in loadtxt to the top of the function body.
View Lorry Controller Status