MAINT: Move usecol handling to C and support more than integer cols

Of course to actually use that many columns you need A LOT of memory
right now.  Each field stores at least a UCS4 NUL character, but
the field is padded enough to require 16 bytes.
We always parse a full row, so that requires 20 bytes per field...
(i.e. 32 GiB RAM is not enough to test this :)).

1 files changed, 4 insertions, 17 deletions

diff --git a/numpy/lib/npyio.py b/numpy/lib/npyio.py
index 6e660dce1..f15f94580 100644
--- a/numpy/lib/npyio.py
+++ b/numpy/lib/npyio.py
@@ -909,25 +909,12 @@ def _read(fname, *, delimiter=',', comment='#', quote='"',
         dtype = np.dtype(object)
 
     if usecols is not None:
-        # Allow usecols to be a single int or a sequence of ints
+        # Allow usecols to be a single int or a sequence of ints, the C-code
+        # handles the rest
         try:
-            usecols_as_list = list(usecols)
+            usecols = list(usecols)
         except TypeError:
-            usecols_as_list = [usecols]
-        for col_idx in usecols_as_list:
-            try:
-                operator.index(col_idx)
-            except TypeError:
-                # Some unit tests for numpy.loadtxt require that the
-                # error message matches this format.
-                raise TypeError(
-                    "usecols must be an int or a sequence of ints but "
-                    "it contains at least one element of type %s" %
-                    type(col_idx),
-                    ) from None
-        # Fall back to existing code
-        usecols = np.array([operator.index(i) for i in usecols_as_list],
-                           dtype=np.int32)
+            usecols = [usecols]
 
     _ensure_ndmin_ndarray_check_param(ndmin)