diff options
| author | Charles Harris <charlesr.harris@gmail.com> | 2011-07-06 16:28:34 -0600 |
|---|---|---|
| committer | Charles Harris <charlesr.harris@gmail.com> | 2011-07-06 16:28:34 -0600 |
| commit | ddbe1d5301ad2c08ef97158d84d9856a9ac8b3cb (patch) | |
| tree | 02f41154add143cf369d4369ed13302dbb234738 | |
| parent | a84257505bfc5436affb3d62da0aac2038d6172e (diff) | |
| parent | 4703dd0d33b0455033ec9b5de526995d72b959cf (diff) | |
| download | numpy-ddbe1d5301ad2c08ef97158d84d9856a9ac8b3cb.tar.gz | |
Merge branch 'pull-99'
* pull-99: (24 commits)
TST: ufunc: Disable where= buffering test temporarily
DOC: c-api: Update to reflect array flag namespace change
NEP: missing-data: Add Peter to the Acknowledgments
NEP: missing-data: Add "Python API" and "C API" definitions
NEP: missing-data: Add numpy.ma to the glossary
NEP: missing-data: Add glossary of terms, try to clarify them better
NEP: missing-data: Fix copy/paste/edit typo for np.all example
NEP: missing-data: Incorporate Ben's feedback, add section on 'shared masks'
TST: umath: Add tests for casting output parameters
ENH: core: Deprecate some bad namespace-polluting macros
NEP: missingdata: Rename 'hasmask' and friends to 'hasnamask' and friends
NEP: missing-data: Add name to acknowledgments
NEP: missingdata: Various improvements
TST: ma: Feedback from Derek about Python 3 failures in ma tests
NEP: Trying to make the NEP's position on missing values and masks vs bit patterns more clear
DOC: Document the ufunc 'where=' parameter and the NpyAuxData C API mechanism
ENH: umath: Add tests, work out kinks in ufunc 'where=' parameter
ENH: umath: Write the function to call the masked inner loop
ENH: umath: Add parsing of a 'where=' parameter in the element-wise ufunc
ENH: umath: Implement the default type resolution for masked loops
...
28 files changed, 3707 insertions, 2948 deletions
diff --git a/doc/neps/missing-data.rst b/doc/neps/missing-data.rst index 6a4781f10..5ced54be2 100644 --- a/doc/neps/missing-data.rst +++ b/doc/neps/missing-data.rst @@ -13,7 +13,7 @@ Abstract ******** Users interested in dealing with missing data within NumPy are generally -pointed to the masked array subclass of the ndarray, generally known +pointed to the masked array subclass of the ndarray, known as 'numpy.ma'. This class has a number of users who depend strongly on its capabilities, but people who are accustomed to the deep integration of the missing data placeholder "NA" in the R project and others who @@ -21,19 +21,19 @@ find the programming interface challenging or inconsistent tend not to use it. This NEP proposes to integrate a mask-based missing data solution -into NumPy, with an additional NA bit pattern-based missing data solution -that can be implemented concurrently or later which would integrate seamlessly +into NumPy, with an additional bitpattern-based missing data solution +that can be implemented concurrently or later integrating seamlessly with the mask-based solution. -The mask-based solution and the NA bit pattern-based solutions in this +The mask-based solution and the bitpattern-based solutions in this proposal offer the exact same missing value abstraction, with several differences in performance, memory overhead, and flexibility. The mask-based solution is more flexible, supporting all behaviors of the -NA bit pattern-based solution, but leaving the hidden values untouched +bitpattern-based solution, but leaving the hidden values untouched whenever an element is masked. -The NA bit pattern-based solution requires less memory, is bit-level +The bitpattern-based solution requires less memory, is bit-level compatible with the 64-bit floating point representation used in R, but does not preserve the hidden values and in fact requires stealing at least one bit pattern from the underlying dtype to represent the missing @@ -42,22 +42,40 @@ value NA. Both solutions are generic in the sense that they can be used with custom data types very easily, with no effort in the case of the masked solution, and with the requirement that a bit pattern to sacrifice be -chosen in the case of the NA bit pattern solution. +chosen in the case of the bitpattern solution. ************************** Definition of Missing Data ************************** -Unknown Yet Existing Data -========================= - In order to be able to develop an intuition about what computation will be done by various NumPy functions, a consistent conceptual -model of what a missing element means must be applied. The approach -taken in the R project is to define a missing element as something which -does have a valid value, but that value is unknown. This proposal -adopts this behavior as as the default for all operations involving -missing values. +model of what a missing element means must be applied. +Ferreting out the behaviors people need or want when they are working +with "missing data" seems to be tricky, but I believe that it boils +down to two different ideas, each of which is internally self-consistent. + +One of them, the "unknown yet existing data" interpretation, can be applied +rigorously to all computations, while the other makes sense for +some statistical operations like standard deviation but not for +linear algebra operations like matrix product. +Thus, making "unknown yet existing data" be the default interpretation +is superior, providing a consistent model across all computations, +and for those operations where the other interpretation makes sense, +an optional parameter "skipna=" can be added. + +For people who want the other interpretation to be default, a mechanism +proposed elsewhere for customizing subclass ufunc behavior with a +_numpy_ufunc_ member function would allow a subclass with a different +default to be created. + +Unknown Yet Existing Data (NA) +============================== + +This is the approach taken in the R project, defining a missing element +as something which does have a valid value which isn't known, or is +NA (not available). This proposal adopts this behavior as as the +default for all operations involving missing values. In this interpretation, nearly any computation with a missing input produces a missing output. For example, 'sum(a)' would produce a missing value @@ -67,44 +85,126 @@ that is not NA, such as logical_and(NA, False) == False. Some more complex arithmetic operations, such as matrix products, are well defined with this interpretation, and the result should be -the same as is the missing values were NaNs. Actually implementing +the same as if the missing values were NaNs. Actually implementing such things to the theoretical limit is probably not worth it, and in many cases either raising an exception or returning all missing values may be preferred to doing precise calculations. -Care must be taken here when dealing with the values and the masks, -to preserve the semantics that masking a value never touches -the element's backing memory. -Data That Doesn't Exist -======================= +Data That Doesn't Exist Or Is Being Skipped (IGNORE) +==================================================== Another useful interpretation is that the missing elements should be treated as if they didn't exist in the array, and the operation should do its best to interpret what that means according to the data that's left. In this case, 'mean(a)' would compute the mean of just -the values that are unmasked, adjusting both the sum and count it -uses based on the mask. +the values that are available, adjusting both the sum and count it +uses based on which values are missing. To be consistent, the mean of +an array of all missing values must produce the same result as the +mean of a zero-sized array without missing value support. This kind of data can arise when conforming sparsely sampled data -into a regular sampling pattern, and is a useful interpretation so +into a regular sampling pattern, and is a useful interpretation to use when attempting to get best-guess answers for many statistical queries. In R, many functions take a parameter "na.rm=T" which means to treat the data as if the NA values are not part of the data set. This proposal -defines a standard parameter "skipmissing=True" for this same purpose. +defines a standard parameter "skipna=True" for this same purpose. + +******************************************** +Implementation Techniques For Missing Values +******************************************** + +In addition to there being two different interpretations of missing values, +there are two different commonly used implementation techniques for +missing values. While there are some differing default behaviors between +existing implementations of the techniques, I believe that the design +choices made in a new implementation must be made based on their merits, +not by rote copying of previous designs. + +Both masks and bitpatterns have different strong and weak points, +depending on the application context. This NEP thus proposes to implement +both. To enable the writing of generic "missing value" code which does +not have to worry about whether the arrays it is using have taken one +or the other approach, the missing value semantics will be identical +for the two implementations. + +Bit Patterns Signalling Missing Values (bitpattern) +=================================================== + +One or more patterns of bits, for example a NaN with +a particular payload, are chosen to represent the missing value +placeholder NA. + +A consequence of this approach is that assigning NA changes the bits +holding the value, so that value is gone. + +Additionally, for some types such as integers, a good and proper value +must be sacrificed to enable this functionality. + +Boolean Masks Signalling Missing Values (mask) +============================================== -Data That Is Being Temporarily Ignored -====================================== +A mask is a parallel array of booleans, either one byte per element or +one bit per element, allocated alongside the existing array data. In this +NEP, the convention is chosen that True means the element is valid +(unmasked), and False means the element is NA. -It can be useful to temporarily treat some array elements as if they -were NA, possibly in many different configurations. This is a common -use case for masks, and the mask-based implementation of missing values -supports this usage by having the strict requirement that the data -storage backing any missing array elements never be touched. +By taking care when writing any C algorithm that works with values +and masks together, it is possible to have the memory for a value +that is masked never be written to. This feature allows multiple +simultaneous views of the same data with different choices of what +is missing, a feature requested by many people on the mailing list. -In general, this can be done by first creating a view, then either adding -a mask if there isn't one yet, or having the view create its own copy of -the mask instead of retaining a view of the original's mask. +This approach places no limitations on the values of the underlying +data type, it may take on any binary pattern without affecting the +NA behavior. + +***************** +Glossary of Terms +***************** + +Because the above discussions of the different concepts and their +relationships are tricky to understand, here are more succinct +definitions of the terms used in this NEP. + +NA (Not Available) + A placeholder for a value which is unknown to computations. That + value may be temporarily hidden with a mask, may have been lost + due to hard drive corruption, or gone for any number of reasons. + For sums and products this means to produce NA if any of the inputs + are NA. This is the same as NA in the R project. + +IGNORE (Skip/Ignore) + A placeholder which should be treated by computations as if no value does + or could exist there. For sums, this means act as if the value + were zero, and for products, this means act as if the value were one. + It's as if the array were compressed in some fashion to not include + that element. + +bitpattern + A technique for implementing either NA or IGNORE, where a particular + set of bit patterns are chosen from all the possible bit patterns of the + value's data type to signal that the element is NA or IGNORE. + +mask + A technique for implementing either NA or IGNORE, where a + boolean or enum array parallel to the data array is used to signal + which elements are NA or IGNORE. + +numpy.ma + The existing implementation of a particular form of masked arrays, + which is part of the NumPy codebase. + +Python API + All the interface mechanisms that are exposed to Python code + for using missing values in NumPy. This API is designed to be + Pythonic and fit into the way NumPy works as much as possible. + +C API + All the implementation mechanisms exposed for CPython extensions + written in C that want to support NumPy missing value support. + This API is designed to be as natural as possible in C, and + is usually prioritizes flexibility and high performance. ******************************** Missing Values as Seen in Python @@ -117,38 +217,40 @@ NumPy will gain a global singleton called numpy.NA, similar to None, but with semantics reflecting its status as a missing value. In particular, trying to treat it as a boolean will raise an exception, and comparisons with it will produce numpy.NA instead of True or False. These basics are -adopted from the behavior of the NA value in the R project. +adopted from the behavior of the NA value in the R project. To dig +deeper into the ideas, http://en.wikipedia.org/wiki/Ternary_logic#Kleene_logic +provides a starting point. For example,:: - >>> np.array([1.0, 2.0, np.NA, 7.0], masked=True) - array([1., 2., NA, 7.], masked=True) + >>> np.array([1.0, 2.0, np.NA, 7.0], namasked=True) + array([1., 2., NA, 7.], namasked=True) >>> np.array([1.0, 2.0, np.NA, 7.0], dtype='NA[f8]') array([1., 2., NA, 7.], dtype='NA[<f8]') produce arrays with values [1.0, 2.0, <inaccessible>, 7.0] / mask [Unmasked, Unmasked, Masked, Unmasked], and -values [1.0, 2.0, <NA bit pattern>, 7.0] respectively. +values [1.0, 2.0, <NA bitpattern>, 7.0] respectively. It may be worth overloading the np.NA __call__ method to accept a dtype, returning a zero-dimensional array with a missing value of that dtype. Without doing this, NA printouts would look like:: - >>> np.sum(np.array([1.0, 2.0, np.NA, 7.0], masked=True)) - array(NA, dtype='float64', masked=True) + >>> np.sum(np.array([1.0, 2.0, np.NA, 7.0], namasked=True)) + array(NA, dtype='float64', namasked=True) >>> np.sum(np.array([1.0, 2.0, np.NA, 7.0], dtype='NA[f8]')) array(NA, dtype='NA[<f8]') but with this, they could be printed as:: - >>> np.sum(np.array([1.0, 2.0, np.NA, 7.0], masked=True)) + >>> np.sum(np.array([1.0, 2.0, np.NA, 7.0], namasked=True)) NA('float64') >>> np.sum(np.array([1.0, 2.0, np.NA, 7.0], dtype='NA[f8]')) NA('NA[<f8]') Assigning a value to an array always causes that element to not be NA, transparently unmasking it if necessary. Assigning numpy.NA to the array -masks that element or assigns the NA bit pattern for the particular dtype. +masks that element or assigns the NA bitpattern for the particular dtype. In the mask-based implementation, the storage behind a missing value may never be accessed in any way, other than to unmask it by assigning its value. @@ -156,9 +258,9 @@ While numpy.NA works to mask values, it does not itself have a dtype. This means that returning the numpy.NA singleton from an operation like 'arr[0]' would be throwing away the dtype, which is still valuable to retain, so 'arr[0]' will return a zero-dimensional -array either with its value masked, or containing the NA bit pattern +array either with its value masked, or containing the NA bitpattern for the array's dtype. To test if the value is missing, the function -"np.ismissing(arr[0])" will be provided. One of the key reasons for the +"np.isna(arr[0])" will be provided. One of the key reasons for the NumPy scalars is to allow their values into dictionaries. Having a missing value as the key in a dictionary is a bad idea, so the NumPy scalars will not support missing values in any form. @@ -170,24 +272,29 @@ from another view which doesn't have them masked. For example:: >>> a = np.array([1,2]) >>> b = a.view() - >>> b.flags.hasmask = True + >>> b.flags.hasnamask = True >>> b - array([1,2], masked=True) + array([1,2], namasked=True) >>> b[0] = np.NA >>> b - array([NA,2], masked=True) + array([NA,2], namasked=True) >>> a array([1,2]) >>> # The underlying number 1 value in 'a[0]' was untouched Copying values between the mask-based implementation and the -NA bit pattern implementation will transparently do the correct thing, -turning the NA bit pattern into a masked value, or a masked value -into the NA bit pattern where appropriate. The one exception is -if a valid value in a masked array happens to have the NA bit pattern, +bitpattern implementation will transparently do the correct thing, +turning the bitpattern into a masked value, or a masked value +into the bitpattern where appropriate. The one exception is +if a valid value in a masked array happens to have the NA bitpattern, copying this value to the NA form of the dtype will cause it to become NA as well. +When operations are done between arrays with NA dtypes and masked arrays, +the result will be masked arrays. This is because in some cases the +NA dtypes cannot represent all the values in the masked array, so +going to masked arrays is the only way to preserve all aspects of the data. + If np.NA or masked values are copied to an array without support for missing values enabled, an exception will be raised. Adding a mask to the target array would be problematic, because then having a mask @@ -197,14 +304,14 @@ performance in unexpected ways. By default, the string "NA" will be used to represent missing values in str and repr outputs. A global configuration will allow this to be changed. The array2string function will also gain a -'maskedstr=' parameter so this could be changed to "<missing>" or +'nastr=' parameter so this could be changed to "<missing>" or other values people may desire. For floating point numbers, Inf and NaN are separate concepts from missing values. If a division by zero occurs in an array with default missing value support, an unmasked Inf or NaN will be produced. To mask those values, a further 'a[np.logical_not(a.isfinite(a)] = np.NA' -can achieve that. For the NA bit pattern approach, the parameterized +can achieve that. For the bitpattern approach, the parameterized dtype('NA[f8,InfNan]') described in a later section can be used to get these semantics without the extra manipulation. @@ -229,10 +336,12 @@ Additionally, exposing the mask directly would preclude a potential space optimization, where a bit-level instead of a byte-level mask is used to get a factor of eight memory usage improvement. -To access the mask values, there are two functions provided, -'np.ismissing' and 'np.isavail', which test for NA or available values -respectively. These functions work equivalently for masked arrays -and NA bit pattern dtypes. +To access a mask directly, there are two functions provided. They +work equivalently for both arrays with masks and NA bit +patterns, so they are specified in terms of NA and available values +instead of masked and unmasked values. The functions are +'np.isna' and 'np.isavail', which test for NA or available values +respectively. Creating Masked Arrays ====================== @@ -240,10 +349,10 @@ Creating Masked Arrays There are two flags which indicate and control the nature of the mask used in masked arrays. -First is 'arr.flags.hasmask', which is True for all masked arrays and +First is 'arr.flags.hasnamask', which is True for all masked arrays and may be set to True to add a mask to an array which does not have one. -Second is 'arr.flags.ownmask', which is True if the array owns the +Second is 'arr.flags.ownnamask', which is True if the array owns the memory to the mask, and False if the array has no mask, or has a view into the mask of another array. If this is set to False in a masked array, the array will create a copy of the mask so that further modifications @@ -274,25 +383,25 @@ New ndarray Methods New functions added to the numpy namespace are:: - np.ismissing(arr) + np.isna(arr) Returns a boolean array with True whereever the array is masked - or matches the NA bit pattern, and False elsewhere + or matches the NA bitpattern, and False elsewhere np.isavail(arr) Returns a boolean array with False whereever the array is masked - or matches the NA bit pattern, and True elsewhere + or matches the NA bitpattern, and True elsewhere New functions added to the ndarray are:: arr.copy(..., replacena=None) Modification to the copy function which replaces NA values, - either masked or with the NA bit pattern, with the 'replacena=' + either masked or with the NA bitpattern, with the 'replacena=' parameter suppled. When 'replacena' isn't None, the copied array is unmasked and has the 'NA' part stripped from the parameterized type ('NA[f8]' becomes just 'f8'). - arr.view(masked=True) - This is a shortcut for 'a = arr.view(); a.flags.hasmask=True'. + arr.view(namasked=True) + This is a shortcut for 'a = arr.view(); a.flags.hasnamask=True'. Element-wise UFuncs With Missing Values ======================================= @@ -301,18 +410,20 @@ As part of the implementation, ufuncs and other operations will have to be extended to support masked computation. Because this is a useful feature in general, even outside the context of a masked array, in addition to working with masked arrays ufuncs -will take an optional 'mask=' parameter which allows the use -of boolean arrays to choose where a computation should be done. -This functions similar to a "where" clause on the ufunc.:: +will take an optional 'where=' parameter which allows the use +of boolean arrays to choose where a computation should be done.:: - >>> np.add(a, b, out=b, mask=(a > threshold)) + >>> np.add(a, b, out=b, where=(a > threshold)) -A benefit of having this 'mask=' parameter is that it provides a way +A benefit of having this 'where=' parameter is that it provides a way to temporarily treat an object with a mask without ever creating a -masked array object. +masked array object. In the example above, this would only do the +add for the array elements with True in the 'where' clause, and neither +'a' nor 'b' need to be masked arrays. -If the 'out' parameter isn't specified, use of the 'mask=' parameter -will produce an array with a mask as the result. +If the 'out' parameter isn't specified, use of the 'where=' parameter +will produce an array with a mask as the result, with missing values +for everywhere the 'where' clause had the value False. For boolean operations, the R project special cases logical_and and logical_or so that logical_and(NA, False) is False, and @@ -348,56 +459,49 @@ will also use the unmasked value counts for their calculations if Some examples:: - >>> a = np.array([1., 3., np.NA, 7.], masked=True) + >>> a = np.array([1., 3., np.NA, 7.], namasked=True) >>> np.sum(a) array(NA, dtype='<f8', masked=True) >>> np.sum(a, skipna=True) 11.0 >>> np.mean(a) - array(NA, dtype='<f8', masked=True) - >>> np.mean(a) + NA('<f8') + >>> np.mean(a, skipna=True) 3.6666666666666665 - >>> a = np.array([np.NA, np.NA], dtype='f8', masked=True) + + >>> a = np.array([np.NA, np.NA], dtype='f8', namasked=True) >>> np.sum(a, skipna=True) 0.0 >>> np.max(a, skipna=True) - array(NA, dtype='<f8', masked=True) - -PEP 3118 -======== - -PEP 3118 doesn't have any mask mechanism, so arrays with masks will -not be accessible through this interface. Similarly, it doesn't support -the specification of dtypes with NA bit patterns, so the parameterized NA -dtypes will also not be accessible through this interface. - -If NumPy did allow access through PEP 3118, this would circumvent the -missing value abstraction in a very damaging way. Other libraries would -try to use masked arrays, and silently get access to the data without -also getting access to the mask or being aware of the missing value -abstraction the mask and data together are following. - -Unresolved Design Questions + array(NA, dtype='<f8', namasked=True) + >>> np.mean(a) + NA('<f8') + >>> np.mean(a, skipna=True) + /home/mwiebe/virtualenvs/dev/lib/python2.7/site-packages/numpy/core/fromnumeric.py:2374: RuntimeWarning: invalid value encountered in double_scalars + return mean(axis, dtype, out) + nan + +The functions 'np.any' and 'np.all' require some special consideration, +just as logical_and and logical_or do. Maybe the best way to describe +their behavior is through a series of examples:: + + >>> np.any(np.array([False, False, False], namasked=True)) + False + >>> np.any(np.array([False, NA, False], namasked=True)) + NA + >>> np.any(np.array([False, NA, True], namasked=True)) + True + + >>> np.all(np.array([True, True, True], namasked=True)) + True + >>> np.all(np.array([True, NA, True], namasked=True)) + NA + >>> np.all(np.array([False, NA, True], namasked=True)) + False + +Parameterized NA Data Types =========================== -The existing masked array implementation has a "hardmask" feature, -which prevents values from ever being unmasked by assigning a value. -This would be an internal array flag, named something like -'arr.flags.hardmask'. - -If the hardmask feature is implemented, boolean indexing could -return a hardmasked array instead of a flattened array with the -arbitrary choice of C-ordering as it currently does. While this -improves the abstraction of the array significantly, it is not -a compatible change. - -********************************** -Alternative Designs Without a Mask -********************************** - -Parameterized Data Type With NA Signal Values -============================================= - A masked array isn't the only way to deal with missing data, and some systems deal with the problem by defining a special "NA" value, for data which is missing. This is distinct from NaN floating point @@ -422,7 +526,7 @@ This allows one to avoid the need to write special case code for each ufunc and for each na* dtype, something that is hard to avoid when building a separate independent dtype implementation for each na* dtype. -Reliable conversions with the NA bit pattern preserved across primitive +Reliable conversions with the NA bitpattern preserved across primitive types requires consideration as well. Even in the simple case of double -> float, where this is supported by hardware, the NA value will get lost because the NaN payload is typically not preserved. @@ -485,6 +589,119 @@ cannot hold values, but will conform to the input types in functions like maps to [('a', 'NA[f4]'), ('b', 'NA[i4]')]. Thus, to view the memory of an 'f8' array 'arr' with 'NA[f8]', you can say arr.view(dtype='NA'). +PEP 3118 +======== + +PEP 3118 doesn't have any mask mechanism, so arrays with masks will +not be accessible through this interface. Similarly, it doesn't support +the specification of dtypes with NA or IGNORE bitpatterns, so the +parameterized NA dtypes will also not be accessible through this interface. + +If NumPy did allow access through PEP 3118, this would circumvent the +missing value abstraction in a very damaging way. Other libraries would +try to use masked arrays, and silently get access to the data without +also getting access to the mask or being aware of the missing value +abstraction the mask and data together are following. + +Cython +====== + +Cython uses PEP 3118 to work with NumPy arrays, so currently it will +simply refuse to work with them as described in the "PEP 3118" section. + +In order to properly support NumPy missing values, Cython will need to +be modified in some fashion to add this support. Likely the best way +to do this will be to include it with supporting np.nditer, which +is most likely going to have an enhancement to make writing missing +value algorithms easier. + +Hard Masks +========== + +The numpy.ma implementation has a "hardmask" feature, +which prevents values from ever being unmasked by assigning a value. +This would be an internal array flag, named something like +'arr.flags.hardmask'. + +If the hardmask feature is implemented, boolean indexing could +return a hardmasked array instead of a flattened array with the +arbitrary choice of C-ordering as it currently does. While this +improves the abstraction of the array significantly, it is not +a compatible change. + +Shared Masks +============ + +One feature of numpy.ma is called 'shared masks'. + +http://docs.scipy.org/doc/numpy/reference/maskedarray.baseclass.html#numpy.ma.MaskedArray.sharedmask + +This feature cannot be supported by a masked implementation of +missing values without directly violating the missing value abstraction. +If the same mask memory is shared between two arrays 'a' and 'b', assigning +a value to a masked element in 'a' will simultaneously unmask the +element with matching index in 'b'. Because this isn't at the same time +assigning a valid value to that element in 'b', this has violated the +abstraction. For this reason, shared masks will not be supported +by the mask-based missing value implementation. + +This is slightly different from what happens when taking a view +of an array with masked missing value support, where a view of +both the mask and the data are taken simultaneously. The result +is two views which share the same mask memory and the same data memory, +which still preserves the missing value abstraction. + +************************ +C Implementation Details +************************ + +The first version to implement is the array masks, because it is +the more general approach. The mask itself is an array, but since +it is intended to never be directly accessible from Python, it won't +be a full ndarray itself. The mask always has the same shape as +the array it's attached to, so it doesn't need its own shape. For +an array with a struct dtype, however, the mask will have a different +dtype than just a straight bool, so it does need its own dtype. +This gives us the following additions to the PyArrayObject:: + + /* + * Descriptor for the mask dtype. + * If no mask: NULL + * If mask : bool/structured dtype of bools + */ + PyArray_Descr *maskdescr; + /* + * Raw data buffer for mask. If the array has the flag + * NPY_ARRAY_OWNNAMASK enabled, it owns this memory and + * must call PyArray_free on it when destroyed. + */ + char *maskdata; + /* + * Just like dimensions and strides point into the same memory + * buffer, we now just make the buffer 3x the nd instead of 2x + * and use the same buffer. + */ + npy_intp *maskstrides; + +There are 2 (or 3) flags which must be added to the array flags:: + + NPY_ARRAY_HASNAMASK + NPY_ARRAY_OWNNAMASK + /* To possibly add in a later revision */ + NPY_ARRAY_HARDNAMASK + +****************************** +C API Access: Masked Iteration +****************************** + +TODO: Describe details about how the nditer will be extended to allow +functions to do masked iteration, transparently working with both +NA dtypes or masked arrays in one implementation. + +******************** +Rejected Alternative +******************** + Parameterized Data Type Which Adds Additional Memory for the NA Flag ==================================================================== @@ -526,5 +743,12 @@ the discussion are:: LluÃs Olivier Delalleau Alan G Isaac + E. Antero Tammi + Jason Grout + Dag Sverre Seljebotn + Joe Harrington + Gary Strangman + Chris Jordan-Squire + Peter I apologize if I missed anyone. diff --git a/doc/source/reference/c-api.array.rst b/doc/source/reference/c-api.array.rst index f34176a00..02aa03ff0 100644 --- a/doc/source/reference/c-api.array.rst +++ b/doc/source/reference/c-api.array.rst @@ -154,7 +154,7 @@ From scratch provided. If *strides* is ``NULL``, then the array strides are computed as C-style contiguous (default) or Fortran-style contiguous (*flags* is nonzero for *data* = ``NULL`` or *flags* & - :cdata:`NPY_F_CONTIGUOUS` is nonzero non-NULL *data*). Any provided + :cdata:`NPY_ARRAY_F_CONTIGUOUS` is nonzero non-NULL *data*). Any provided *dims* and *strides* are copied into newly allocated dimension and strides arrays for the new array object. @@ -297,108 +297,112 @@ From other objects the array is constructed that way. Almost always this parameter is ``NULL``. - .. cvar:: NPY_C_CONTIGUOUS + In versions 1.6 and earlier of NumPy, the following flags + did not have the _ARRAY_ macro namespace in them. That form + of the constant names is deprecated in 1.7. + + .. cvar:: NPY_ARRAY_C_CONTIGUOUS Make sure the returned array is C-style contiguous - .. cvar:: NPY_F_CONTIGUOUS + .. cvar:: NPY_ARRAY_F_CONTIGUOUS Make sure the returned array is Fortran-style contiguous. - .. cvar:: NPY_ALIGNED + .. cvar:: NPY_ARRAY_ALIGNED Make sure the returned array is aligned on proper boundaries for its data type. An aligned array has the data pointer and every strides factor as a multiple of the alignment factor for the data-type- descriptor. - .. cvar:: NPY_WRITEABLE + .. cvar:: NPY_ARRAY_WRITEABLE Make sure the returned array can be written to. - .. cvar:: NPY_ENSURECOPY + .. cvar:: NPY_ARRAY_ENSURECOPY Make sure a copy is made of *op*. If this flag is not present, data is not copied if it can be avoided. - .. cvar:: NPY_ENSUREARRAY + .. cvar:: NPY_ARRAY_ENSUREARRAY Make sure the result is a base-class ndarray or bigndarray. By default, if *op* is an instance of a subclass of the bigndarray, an instance of that same subclass is returned. If this flag is set, an ndarray object will be returned instead. - .. cvar:: NPY_FORCECAST + .. cvar:: NPY_ARRAY_FORCECAST Force a cast to the output type even if it cannot be done safely. Without this flag, a data cast will occur only if it can be done safely, otherwise an error is reaised. - .. cvar:: NPY_UPDATEIFCOPY + .. cvar:: NPY_ARRAY_UPDATEIFCOPY If *op* is already an array, but does not satisfy the requirements, then a copy is made (which will satisfy the - requirements). If this flag is present and a copy (of an - object that is already an array) must be made, then the - corresponding :cdata:`NPY_UPDATEIFCOPY` flag is set in the returned + requirements). If this flag is present and a copy (of an object + that is already an array) must be made, then the corresponding + :cdata:`NPY_ARRAY_UPDATEIFCOPY` flag is set in the returned copy and *op* is made to be read-only. When the returned copy is deleted (presumably after your calculations are complete), its contents will be copied back into *op* and the *op* array - will be made writeable again. If *op* is not writeable to - begin with, then an error is raised. If *op* is not already an - array, then this flag has no effect. + will be made writeable again. If *op* is not writeable to begin + with, then an error is raised. If *op* is not already an array, + then this flag has no effect. - .. cvar:: NPY_BEHAVED + .. cvar:: NPY_ARRAY_BEHAVED - :cdata:`NPY_ALIGNED` \| :cdata:`NPY_WRITEABLE` + :cdata:`NPY_ARRAY_ALIGNED` \| :cdata:`NPY_ARRAY_WRITEABLE` - .. cvar:: NPY_CARRAY + .. cvar:: NPY_ARRAY_CARRAY - :cdata:`NPY_C_CONTIGUOUS` \| :cdata:`NPY_BEHAVED` + :cdata:`NPY_ARRAY_C_CONTIGUOUS` \| :cdata:`NPY_ARRAY_BEHAVED` - .. cvar:: NPY_CARRAY_RO + .. cvar:: NPY_ARRAY_CARRAY_RO - :cdata:`NPY_C_CONTIGUOUS` \| :cdata:`NPY_ALIGNED` + :cdata:`NPY_ARRAY_C_CONTIGUOUS` \| :cdata:`NPY_ARRAY_ALIGNED` - .. cvar:: NPY_FARRAY + .. cvar:: NPY_ARRAY_FARRAY - :cdata:`NPY_F_CONTIGUOUS` \| :cdata:`NPY_BEHAVED` + :cdata:`NPY_ARRAY_F_CONTIGUOUS` \| :cdata:`NPY_ARRAY_BEHAVED` - .. cvar:: NPY_FARRAY_RO + .. cvar:: NPY_ARRAY_FARRAY_RO - :cdata:`NPY_F_CONTIGUOUS` \| :cdata:`NPY_ALIGNED` + :cdata:`NPY_ARRAY_F_CONTIGUOUS` \| :cdata:`NPY_ARRAY_ALIGNED` - .. cvar:: NPY_DEFAULT + .. cvar:: NPY_ARRAY_DEFAULT - :cdata:`NPY_CARRAY` + :cdata:`NPY_ARRAY_CARRAY` - .. cvar:: NPY_IN_ARRAY + .. cvar:: NPY_ARRAY_IN_ARRAY - :cdata:`NPY_CONTIGUOUS` \| :cdata:`NPY_ALIGNED` + :cdata:`NPY_ARRAY_CONTIGUOUS` \| :cdata:`NPY_ARRAY_ALIGNED` - .. cvar:: NPY_IN_FARRAY + .. cvar:: NPY_ARRAY_IN_FARRAY - :cdata:`NPY_F_CONTIGUOUS` \| :cdata:`NPY_ALIGNED` + :cdata:`NPY_ARRAY_F_CONTIGUOUS` \| :cdata:`NPY_ARRAY_ALIGNED` .. cvar:: NPY_OUT_ARRAY - :cdata:`NPY_C_CONTIGUOUS` \| :cdata:`NPY_WRITEABLE` \| - :cdata:`NPY_ALIGNED` + :cdata:`NPY_ARRAY_C_CONTIGUOUS` \| :cdata:`NPY_ARRAY_WRITEABLE` \| + :cdata:`NPY_ARRAY_ALIGNED` - .. cvar:: NPY_OUT_FARRAY + .. cvar:: NPY_ARRAY_OUT_FARRAY - :cdata:`NPY_F_CONTIGUOUS` \| :cdata:`NPY_WRITEABLE` \| - :cdata:`NPY_ALIGNED` + :cdata:`NPY_ARRAY_F_CONTIGUOUS` \| :cdata:`NPY_ARRAY_WRITEABLE` \| + :cdata:`NPY_ARRAY_ALIGNED` - .. cvar:: NPY_INOUT_ARRAY + .. cvar:: NPY_ARRAY_INOUT_ARRAY - :cdata:`NPY_C_CONTIGUOUS` \| :cdata:`NPY_WRITEABLE` \| - :cdata:`NPY_ALIGNED` \| :cdata:`NPY_UPDATEIFCOPY` + :cdata:`NPY_ARRAY_C_CONTIGUOUS` \| :cdata:`NPY_ARRAY_WRITEABLE` \| + :cdata:`NPY_ARRAY_ALIGNED` \| :cdata:`NPY_ARRAY_UPDATEIFCOPY` - .. cvar:: NPY_INOUT_FARRAY + .. cvar:: NPY_ARRAY_INOUT_FARRAY - :cdata:`NPY_F_CONTIGUOUS` \| :cdata:`NPY_WRITEABLE` \| - :cdata:`NPY_ALIGNED` \| :cdata:`NPY_UPDATEIFCOPY` + :cdata:`NPY_ARRAY_F_CONTIGUOUS` \| :cdata:`NPY_ARRAY_WRITEABLE` \| + :cdata:`NPY_ARRAY_ALIGNED` \| :cdata:`NPY_ARRAY_UPDATEIFCOPY` .. cfunction:: int PyArray_GetArrayParamsFromObject(PyObject* op, PyArray_Descr* requested_dtype, npy_bool writeable, PyArray_Descr** out_dtype, int* out_ndim, npy_intp* out_dims, PyArrayObject** out_arr, PyObject* context) @@ -421,7 +425,7 @@ From other objects If writing to the value in 'op' is desired, set the boolean 'writeable' to 1. This raises an error when 'op' is a scalar, list of lists, or other non-writeable 'op'. This differs from passing - NPY_WRITEABLE to PyArray_FromAny, where the writeable array may + NPY_ARRAY_WRITEABLE to PyArray_FromAny, where the writeable array may be a copy of the input. When success (0 return value) is returned, either out_arr @@ -447,7 +451,7 @@ From other objects // Could make custom strides here too arr = PyArray_NewFromDescr(&PyArray_Type, dtype, ndim, dims, NULL, - fortran ? NPY_F_CONTIGUOUS : 0, + fortran ? NPY_ARRAY_F_CONTIGUOUS : 0, NULL); if (arr == NULL) { return NULL; @@ -466,12 +470,12 @@ From other objects .. cfunction:: PyObject* PyArray_CheckFromAny(PyObject* op, PyArray_Descr* dtype, int min_depth, int max_depth, int requirements, PyObject* context) Nearly identical to :cfunc:`PyArray_FromAny` (...) except - *requirements* can contain :cdata:`NPY_NOTSWAPPED` (over-riding the - specification in *dtype*) and :cdata:`NPY_ELEMENTSTRIDES` which + *requirements* can contain :cdata:`NPY_ARRAY_NOTSWAPPED` (over-riding the + specification in *dtype*) and :cdata:`NPY_ARRAY_ELEMENTSTRIDES` which indicates that the array should be aligned in the sense that the strides are multiples of the element size. -.. cvar:: NPY_NOTSWAPPED +.. cvar:: NPY_ARRAY_NOTSWAPPED Make sure the returned array has a data-type descriptor that is in machine byte-order, over-riding any specification in the *dtype* @@ -482,11 +486,11 @@ From other objects not in machine byte- order), then a new data-type descriptor is created and used with its byte-order field set to native. -.. cvar:: NPY_BEHAVED_NS +.. cvar:: NPY_ARRAY_BEHAVED_NS - :cdata:`NPY_ALIGNED` \| :cdata:`NPY_WRITEABLE` \| :cdata:`NPY_NOTSWAPPED` + :cdata:`NPY_ARRAY_ALIGNED` \| :cdata:`NPY_ARRAY_WRITEABLE` \| :cdata:`NPY_ARRAY_NOTSWAPPED` -.. cvar:: NPY_ELEMENTSTRIDES +.. cvar:: NPY_ARRAY_ELEMENTSTRIDES Make sure the returned array has strides that are multiples of the element size. @@ -545,7 +549,7 @@ From other objects This function **steals a reference** to ``op`` and makes sure that ``op`` is a base-class ndarray. It special cases array scalars, but otherwise calls :cfunc:`PyArray_FromAny` ( ``op``, NULL, 0, 0, - :cdata:`NPY_ENSUREARRAY`). + :cdata:`NPY_ARRAY_ENSUREARRAY`). .. cfunction:: PyObject* PyArray_FromString(char* string, npy_intp slen, PyArray_Descr* dtype, npy_intp num, char* sep) @@ -579,7 +583,7 @@ From other objects object, ``buf``, that exports the (single-segment) buffer protocol (or has an attribute __buffer\__ that returns an object that exports the buffer protocol). A writeable buffer will be tried - first followed by a read- only buffer. The :cdata:`NPY_WRITEABLE` + first followed by a read- only buffer. The :cdata:`NPY_ARRAY_WRITEABLE` flag of the returned array will reflect which one was successful. The data is assumed to start at ``offset`` bytes from the start of the memory location for the object. The type of the @@ -625,11 +629,11 @@ From other objects Similar to :cfunc:`PyArray_FROM_O` except it can take an argument of *requirements* indicating properties the resulting array must have. Available requirements that can be enforced are - :cdata:`NPY_CONTIGUOUS`, :cdata:`NPY_F_CONTIGUOUS`, - :cdata:`NPY_ALIGNED`, :cdata:`NPY_WRITEABLE`, - :cdata:`NPY_NOTSWAPPED`, :cdata:`NPY_ENSURECOPY`, - :cdata:`NPY_UPDATEIFCOPY`, :cdata:`NPY_FORCECAST`, and - :cdata:`NPY_ENSUREARRAY`. Standard combinations of flags can also + :cdata:`NPY_ARRAY_C_CONTIGUOUS`, :cdata:`NPY_ARRAY_F_CONTIGUOUS`, + :cdata:`NPY_ARRAY_ALIGNED`, :cdata:`NPY_ARRAY_WRITEABLE`, + :cdata:`NPY_ARRAY_NOTSWAPPED`, :cdata:`NPY_ARRAY_ENSURECOPY`, + :cdata:`NPY_ARRAY_UPDATEIFCOPY`, :cdata:`NPY_ARRAY_FORCECAST`, and + :cdata:`NPY_ARRAY_ENSUREARRAY`. Standard combinations of flags can also be used: .. cfunction:: PyObject* PyArray_FROM_OT(PyObject* obj, int typenum) @@ -648,7 +652,7 @@ From other objects specified using a typenumber. :cfunc:`PyArray_DescrFromType` (*typenum*) is passed directly to :cfunc:`PyArray_FromAny`. This macro also adds :cdata:`NPY_DEFAULT` to requirements if - :cdata:`NPY_ENSURECOPY` is passed in as requirements. + :cdata:`NPY_ARRAY_ENSURECOPY` is passed in as requirements. .. cfunction:: PyObject *PyArray_CheckAxis(PyObject* obj, int* axis, int requirements) @@ -1187,11 +1191,11 @@ determine the bit-position of the flag. Python exposes a nice attribute- based interface as well as a dictionary-like interface for getting (and, if appropriate, setting) these flags. -Memory areas of all kinds can be pointed to by an ndarray, -necessitating these flags. If you get an arbitrary ``PyArrayObject`` -in C-code, you need to be aware of the flags that are set. If you -need to guarantee a certain kind of array (like :cdata:`NPY_C_CONTIGUOUS` and -:cdata:`NPY_BEHAVED`), then pass these requirements into the +Memory areas of all kinds can be pointed to by an ndarray, necessitating +these flags. If you get an arbitrary ``PyArrayObject`` in C-code, you +need to be aware of the flags that are set. If you need to guarantee +a certain kind of array (like :cdata:`NPY_ARRAY_C_CONTIGUOUS` and +:cdata:`NPY_ARRAY_BEHAVED`), then pass these requirements into the PyArray_FromAny function. @@ -1206,38 +1210,38 @@ might not be writeable. It might be in Fortan-contiguous order. The array flags are used to indicate what can be said about data associated with an array. -.. cvar:: NPY_C_CONTIGUOUS +.. cvar:: NPY_ARRAY_C_CONTIGUOUS The data area is in C-style contiguous order (last index varies the fastest). -.. cvar:: NPY_F_CONTIGUOUS +.. cvar:: NPY_ARRAY_F_CONTIGUOUS The data area is in Fortran-style contiguous order (first index varies the fastest). -Notice that contiguous 1-d arrays are always both Fortran -contiguous and C contiguous. Both of these flags can be checked and -are convenience flags only as whether or not an array is -:cdata:`NPY_C_CONTIGUOUS` or :cdata:`NPY_F_CONTIGUOUS` can be determined by the -``strides``, ``dimensions``, and ``itemsize`` attributes. +Notice that contiguous 1-d arrays are always both Fortran contiguous +and C contiguous. Both of these flags can be checked and are convenience +flags only as whether or not an array is :cdata:`NPY_ARRAY_C_CONTIGUOUS` +or :cdata:`NPY_ARRAY_F_CONTIGUOUS` can be determined by the ``strides``, +``dimensions``, and ``itemsize`` attributes. -.. cvar:: NPY_OWNDATA +.. cvar:: NPY_ARRAY_OWNDATA The data area is owned by this array. -.. cvar:: NPY_ALIGNED +.. cvar:: NPY_ARRAY_ALIGNED The data area is aligned appropriately (for all strides). -.. cvar:: NPY_WRITEABLE +.. cvar:: NPY_ARRAY_WRITEABLE The data area can be written to. Notice that the above 3 flags are are defined so that a new, well- behaved array has these flags defined as true. -.. cvar:: NPY_UPDATEIFCOPY +.. cvar:: NPY_ARRAY_UPDATEIFCOPY The data area represents a (well-behaved) copy whose information should be transferred back to the original when this array is deleted. @@ -1250,47 +1254,47 @@ are convenience flags only as whether or not an array is array (which is set read_only). When the array with this flag set is deallocated, it will copy its contents back to the "misbehaved" array (casting if necessary) and will reset the "misbehaved" array - to :cdata:`NPY_WRITEABLE`. If the "misbehaved" array was not - :cdata:`NPY_WRITEABLE` to begin with then :cfunc:`PyArray_FromAny` - would have returned an error because :cdata:`NPY_UPDATEIFCOPY` + to :cdata:`NPY_ARRAY_WRITEABLE`. If the "misbehaved" array was not + :cdata:`NPY_ARRAY_WRITEABLE` to begin with then :cfunc:`PyArray_FromAny` + would have returned an error because :cdata:`NPY_ARRAY_UPDATEIFCOPY` would not have been possible. -:cfunc:`PyArray_UpdateFlags` (obj, flags) will update the -``obj->flags`` for ``flags`` which can be any of -:cdata:`NPY_C_CONTIGUOUS`, :cdata:`NPY_F_CONTIGUOUS`, :cdata:`NPY_ALIGNED`, -or :cdata:`NPY_WRITEABLE`. +:cfunc:`PyArray_UpdateFlags` (obj, flags) will update the ``obj->flags`` +for ``flags`` which can be any of :cdata:`NPY_ARRAY_C_CONTIGUOUS`, +:cdata:`NPY_ARRAY_F_CONTIGUOUS`, :cdata:`NPY_ARRAY_ALIGNED`, or +:cdata:`NPY_ARRAY_WRITEABLE`. Combinations of array flags ^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. cvar:: NPY_BEHAVED +.. cvar:: NPY_ARRAY_BEHAVED - :cdata:`NPY_ALIGNED` \| :cdata:`NPY_WRITEABLE` + :cdata:`NPY_ARRAY_ALIGNED` \| :cdata:`NPY_ARRAY_WRITEABLE` -.. cvar:: NPY_CARRAY +.. cvar:: NPY_ARRAY_CARRAY - :cdata:`NPY_C_CONTIGUOUS` \| :cdata:`NPY_BEHAVED` + :cdata:`NPY_ARRAY_C_CONTIGUOUS` \| :cdata:`NPY_ARRAY_BEHAVED` -.. cvar:: NPY_CARRAY_RO +.. cvar:: NPY_ARRAY_CARRAY_RO - :cdata:`NPY_C_CONTIGUOUS` \| :cdata:`NPY_ALIGNED` + :cdata:`NPY_ARRAY_C_CONTIGUOUS` \| :cdata:`NPY_ARRAY_ALIGNED` -.. cvar:: NPY_FARRAY +.. cvar:: NPY_ARRAY_FARRAY - :cdata:`NPY_F_CONTIGUOUS` \| :cdata:`NPY_BEHAVED` + :cdata:`NPY_ARRAY_F_CONTIGUOUS` \| :cdata:`NPY_ARRAY_BEHAVED` -.. cvar:: NPY_FARRAY_RO +.. cvar:: NPY_ARRAY_FARRAY_RO - :cdata:`NPY_F_CONTIGUOUS` \| :cdata:`NPY_ALIGNED` + :cdata:`NPY_ARRAY_F_CONTIGUOUS` \| :cdata:`NPY_ARRAY_ALIGNED` -.. cvar:: NPY_DEFAULT +.. cvar:: NPY_ARRAY_DEFAULT - :cdata:`NPY_CARRAY` + :cdata:`NPY_ARRAY_CARRAY` -.. cvar:: NPY_UPDATE_ALL +.. cvar:: NPY_ARRAY_UPDATE_ALL - :cdata:`NPY_C_CONTIGUOUS` \| :cdata:`NPY_F_CONTIGUOUS` \| :cdata:`NPY_ALIGNED` + :cdata:`NPY_ARRAY_C_CONTIGUOUS` \| :cdata:`NPY_ARRAY_F_CONTIGUOUS` \| :cdata:`NPY_ARRAY_ALIGNED` Flag-like constants @@ -1299,28 +1303,28 @@ Flag-like constants These constants are used in :cfunc:`PyArray_FromAny` (and its macro forms) to specify desired properties of the new array. -.. cvar:: NPY_FORCECAST +.. cvar:: NPY_ARRAY_FORCECAST Cast to the desired type, even if it can't be done without losing information. -.. cvar:: NPY_ENSURECOPY +.. cvar:: NPY_ARRAY_ENSURECOPY Make sure the resulting array is a copy of the original. -.. cvar:: NPY_ENSUREARRAY +.. cvar:: NPY_ARRAY_ENSUREARRAY Make sure the resulting object is an actual ndarray (or bigndarray), and not a sub-class. -.. cvar:: NPY_NOTSWAPPED +.. cvar:: NPY_ARRAY_NOTSWAPPED Only used in :cfunc:`PyArray_CheckFromAny` to over-ride the byteorder of the data-type object passed in. -.. cvar:: NPY_BEHAVED_NS +.. cvar:: NPY_ARRAY_BEHAVED_NS - :cdata:`NPY_ALIGNED` \| :cdata:`NPY_WRITEABLE` \| :cdata:`NPY_NOTSWAPPED` + :cdata:`NPY_ARRAY_ALIGNED` \| :cdata:`NPY_ARRAY_WRITEABLE` \| :cdata:`NPY_ARRAY_NOTSWAPPED` Flag checking @@ -1334,9 +1338,9 @@ For all of these macros *arr* must be an instance of a (subclass of) The first parameter, arr, must be an ndarray or subclass. The parameter, *flags*, should be an integer consisting of bitwise combinations of the possible flags an array can have: - :cdata:`NPY_C_CONTIGUOUS`, :cdata:`NPY_F_CONTIGUOUS`, - :cdata:`NPY_OWNDATA`, :cdata:`NPY_ALIGNED`, - :cdata:`NPY_WRITEABLE`, :cdata:`NPY_UPDATEIFCOPY`. + :cdata:`NPY_ARRAY_C_CONTIGUOUS`, :cdata:`NPY_ARRAY_F_CONTIGUOUS`, + :cdata:`NPY_ARRAY_OWNDATA`, :cdata:`NPY_ARRAY_ALIGNED`, + :cdata:`NPY_ARRAY_WRITEABLE`, :cdata:`NPY_ARRAY_UPDATEIFCOPY`. .. cfunction:: PyArray_ISCONTIGUOUS(arr) @@ -1392,8 +1396,8 @@ For all of these macros *arr* must be an instance of a (subclass of) .. cfunction:: void PyArray_UpdateFlags(PyArrayObject* arr, int flagmask) - The :cdata:`NPY_C_CONTIGUOUS`, :cdata:`NPY_ALIGNED`, and - :cdata:`NPY_F_CONTIGUOUS` array flags can be "calculated" from the + The :cdata:`NPY_ARRAY_C_CONTIGUOUS`, :cdata:`NPY_ARRAY_ALIGNED`, and + :cdata:`NPY_ARRAY_F_CONTIGUOUS` array flags can be "calculated" from the array object itself. This routine updates one or more of these flags of *arr* as specified in *flagmask* by performing the required calculation. @@ -2025,6 +2029,97 @@ Other functions 1 if the lists are identical; otherwise, return 0. +Auxiliary Data With Object Semantics +------------------------------------ + +.. versionadded:: 1.7.0 + +.. ctype:: NpyAuxData + +When working with more complex dtypes which are composed of other dtypes, +such as the struct dtype, creating inner loops that manipulate the dtypes +requires carrying along additional data. NumPy supports this idea +through a struct :ctype:`NpyAuxData`, mandating a few conventions so that +it is possible to do this. + +Defining an :ctype:`NpyAuxData` is similar to defining a class in C++, +but the object semantics have to be tracked manually since the API is in C. +Here's an example for a function which doubles up an element using +an element copier function as a primitive.:: + + typedef struct { + NpyAuxData base; + ElementCopier_Func *func; + NpyAuxData *funcdata; + } eldoubler_aux_data; + + void free_element_doubler_aux_data(NpyAuxData *data) + { + eldoubler_aux_data *d = (eldoubler_aux_data *)data; + /* Free the memory owned by this auxadata */ + NPY_AUXDATA_FREE(d->funcdata); + PyArray_free(d); + } + + NpyAuxData *clone_element_doubler_aux_data(NpyAuxData *data) + { + eldoubler_aux_data *ret = PyArray_malloc(sizeof(eldoubler_aux_data)); + if (ret == NULL) { + return NULL; + } + + /* Raw copy of all data */ + memcpy(ret, data, sizeof(eldoubler_aux_data)); + + /* Fix up the owned auxdata so we have our own copy */ + ret->funcdata = NPY_AUXDATA_CLONE(ret->funcdata); + if (ret->funcdata == NULL) { + PyArray_free(ret); + return NULL; + } + + return (NpyAuxData *)ret; + } + + NpyAuxData *create_element_doubler_aux_data( + ElementCopier_Func *func, + NpyAuxData *funcdata) + { + eldoubler_aux_data *ret = PyArray_malloc(sizeof(eldoubler_aux_data)); + if (ret == NULL) { + PyErr_NoMemory(); + return NULL; + } + memset(&ret, 0, sizeof(eldoubler_aux_data)); + ret->base->free = &free_element_doubler_aux_data; + ret->base->clone = &clone_element_doubler_aux_data; + ret->func = func; + ret->funcdata = funcdata; + + return (NpyAuxData *)ret; + } + +.. ctype:: NpyAuxData_FreeFunc + + The function pointer type for NpyAuxData free functions. + +.. ctype:: NpyAuxData_CloneFunc + + The function pointer type for NpyAuxData clone functions. These + functions should never set the Python exception on error, because + they may be called from a multi-threaded context. + +.. cfunction:: NPY_AUXDATA_FREE(auxdata) + + A macro which calls the auxdata's free function appropriately, + does nothing if auxdata is NULL. + +.. cfunction:: NPY_AUXDATA_CLONE(auxdata) + + A macro which calls the auxdata's clone function appropriately, + returning a deep copy of the auxiliary data. + + Array Iterators --------------- @@ -2516,7 +2611,7 @@ to. to hold on to the memory be sure to INCREF the base member. The chunk of memory is pointed to by *buf* ->ptr member and has length *buf* ->len. The flags member of *buf* is :cdata:`NPY_BEHAVED_RO` with - the :cdata:`NPY_WRITEABLE` flag set if *obj* has a writeable buffer + the :cdata:`NPY_ARRAY_WRITEABLE` flag set if *obj* has a writeable buffer interface. .. cfunction:: int PyArray_AxisConverter(PyObject \* obj, int* axis) @@ -3006,11 +3101,11 @@ Miscellaneous Macros .. cfunction:: PyArray_XDECREF_ERR(PyObject \*obj) - DECREF's an array object which may have the :cdata:`NPY_UPDATEIFCOPY` + DECREF's an array object which may have the :cdata:`NPY_ARRAY_UPDATEIFCOPY` flag set without causing the contents to be copied back into the - original array. Resets the :cdata:`NPY_WRITEABLE` flag on the base + original array. Resets the :cdata:`NPY_ARRAY_WRITEABLE` flag on the base object. This is useful for recovering from an error condition when - :cdata:`NPY_UPDATEIFCOPY` is used. + :cdata:`NPY_ARRAY_UPDATEIFCOPY` is used. Enumerated Types diff --git a/doc/source/reference/c-api.types-and-structures.rst b/doc/source/reference/c-api.types-and-structures.rst index 770aac995..8c813faf5 100644 --- a/doc/source/reference/c-api.types-and-structures.rst +++ b/doc/source/reference/c-api.types-and-structures.rst @@ -129,14 +129,14 @@ PyArray_Type .. cmember:: PyObject *PyArrayObject.base - This member is used to hold a pointer to another Python object - that is related to this array. There are two use cases: 1) If - this array does not own its own memory, then base points to the - Python object that owns it (perhaps another array object), 2) - If this array has the :cdata:`NPY_UPDATEIFCOPY` flag set, then this - array is a working copy of a "misbehaved" array. As soon as - this array is deleted, the array pointed to by base will be - updated with the contents of this array. + This member is used to hold a pointer to another Python object that + is related to this array. There are two use cases: 1) If this array + does not own its own memory, then base points to the Python object + that owns it (perhaps another array object), 2) If this array has + the :cdata:`NPY_ARRAY_UPDATEIFCOPY` flag set, then this array is + a working copy of a "misbehaved" array. As soon as this array is + deleted, the array pointed to by base will be updated with the + contents of this array. .. cmember:: PyArray_Descr *PyArrayObject.descr @@ -151,9 +151,10 @@ PyArray_Type .. cmember:: int PyArrayObject.flags Flags indicating how the memory pointed to by data is to be - interpreted. Possible flags are :cdata:`NPY_C_CONTIGUOUS`, - :cdata:`NPY_F_CONTIGUOUS`, :cdata:`NPY_OWNDATA`, :cdata:`NPY_ALIGNED`, - :cdata:`NPY_WRITEABLE`, and :cdata:`NPY_UPDATEIFCOPY`. + interpreted. Possible flags are :cdata:`NPY_ARRAY_C_CONTIGUOUS`, + :cdata:`NPY_ARRAY_F_CONTIGUOUS`, :cdata:`NPY_ARRAY_OWNDATA`, + :cdata:`NPY_ARRAY_ALIGNED`, :cdata:`NPY_ARRAY_WRITEABLE`, and + :cdata:`NPY_ARRAY_UPDATEIFCOPY`. .. cmember:: PyObject *PyArrayObject.weakreflist @@ -850,8 +851,8 @@ PyArrayIter_Type .. cmember:: Bool PyArrayIterObject.contiguous This flag is true if the underlying array is - :cdata:`NPY_C_CONTIGUOUS`. It is used to simplify calculations when - possible. + :cdata:`NPY_ARRAY_C_CONTIGUOUS`. It is used to simplify + calculations when possible. How to use an array iterator on a C-level is explained more fully in @@ -1057,8 +1058,8 @@ PyArray_Chunk .. cmember:: int PyArray_Chunk.flags - Any data flags (*e.g.* :cdata:`NPY_WRITEABLE` ) that should be used - to interpret the memory. + Any data flags (*e.g.* :cdata:`NPY_ARRAY_WRITEABLE` ) that should + be used to interpret the memory. PyArrayInterface @@ -1117,12 +1118,12 @@ PyArrayInterface .. cmember:: int PyArrayInterface.flags - Any of the bits :cdata:`NPY_C_CONTIGUOUS` (1), - :cdata:`NPY_F_CONTIGUOUS` (2), :cdata:`NPY_ALIGNED` (0x100), - :cdata:`NPY_NOTSWAPPED` (0x200), or :cdata:`NPY_WRITEABLE` + Any of the bits :cdata:`NPY_ARRAY_C_CONTIGUOUS` (1), + :cdata:`NPY_ARRAY_F_CONTIGUOUS` (2), :cdata:`NPY_ARRAY_ALIGNED` (0x100), + :cdata:`NPY_ARRAY_NOTSWAPPED` (0x200), or :cdata:`NPY_ARRAY_WRITEABLE` (0x400) to indicate something about the data. The - :cdata:`NPY_ALIGNED`, :cdata:`NPY_C_CONTIGUOUS`, and - :cdata:`NPY_F_CONTIGUOUS` flags can actually be determined from + :cdata:`NPY_ARRAY_ALIGNED`, :cdata:`NPY_ARRAY_C_CONTIGUOUS`, and + :cdata:`NPY_ARRAY_F_CONTIGUOUS` flags can actually be determined from the other parameters. The flag :cdata:`NPY_ARR_HAS_DESCR` (0x800) can also be set to indicate to objects consuming the version 3 array interface that the descr member of the diff --git a/doc/source/reference/internals.code-explanations.rst b/doc/source/reference/internals.code-explanations.rst index cceb1a60d..580661cb3 100644 --- a/doc/source/reference/internals.code-explanations.rst +++ b/doc/source/reference/internals.code-explanations.rst @@ -41,20 +41,20 @@ called a rank-0 array), then the strides and dimensions variables are NULL. Besides the structural information contained in the strides and -dimensions members of the :ctype:`PyArrayObject`, the flags contain important -information about how the data may be accessed. In particular, the -:cdata:`NPY_ALIGNED` flag is set when the memory is on a suitable boundary -according to the data-type array. Even if you have a contiguous chunk -of memory, you cannot just assume it is safe to dereference a data- -type-specific pointer to an element. Only if the :cdata:`NPY_ALIGNED` flag is -set is this a safe operation (on some platforms it will work but on -others, like Solaris, it will cause a bus error). The :cdata:`NPY_WRITEABLE` -should also be ensured if you plan on writing to the memory area of -the array. It is also possible to obtain a pointer to an unwriteable -memory area. Sometimes, writing to the memory area when the -:cdata:`NPY_WRITEABLE` flag is not set will just be rude. Other times it can -cause program crashes ( *e.g.* a data-area that is a read-only -memory-mapped file). +dimensions members of the :ctype:`PyArrayObject`, the flags contain +important information about how the data may be accessed. In particular, +the :cdata:`NPY_ARRAY_ALIGNED` flag is set when the memory is on a +suitable boundary according to the data-type array. Even if you have +a contiguous chunk of memory, you cannot just assume it is safe to +dereference a data- type-specific pointer to an element. Only if the +:cdata:`NPY_ARRAY_ALIGNED` flag is set is this a safe operation (on +some platforms it will work but on others, like Solaris, it will cause +a bus error). The :cdata:`NPY_ARRAY_WRITEABLE` should also be ensured +if you plan on writing to the memory area of the array. It is also +possible to obtain a pointer to an unwriteable memory area. Sometimes, +writing to the memory area when the :cdata:`NPY_ARRAY_WRITEABLE` flag is not +set will just be rude. Other times it can cause program crashes ( *e.g.* +a data-area that is a read-only memory-mapped file). Data-type encapsulation diff --git a/doc/source/reference/ufuncs.rst b/doc/source/reference/ufuncs.rst index 0e7da347e..295d52ef4 100644 --- a/doc/source/reference/ufuncs.rst +++ b/doc/source/reference/ufuncs.rst @@ -292,6 +292,14 @@ advanced usage and will not typically be used. The first output can provided as either a positional or a keyword parameter. +*where* + + .. versionadded:: 1.7 + + Accepts a boolean array which is broadcast together with the operands. + Values of True indicate to calculate the ufunc at that position, values + of False indicate to leave the value in the output alone. + *casting* .. versionadded:: 1.6 diff --git a/doc/source/user/c-info.how-to-extend.rst b/doc/source/user/c-info.how-to-extend.rst index 6c5e25aff..2d10c36a3 100644 --- a/doc/source/user/c-info.how-to-extend.rst +++ b/doc/source/user/c-info.how-to-extend.rst @@ -432,64 +432,65 @@ writeable). The syntax is converting the Python object into an array that is more "well-behaved" for your specific usage. - The requirements flag allows specification of what kind of array is - acceptable. If the object passed in does not satisfy this requirements - then a copy is made so that thre returned object will satisfy the - requirements. these ndarray can use a very generic pointer to memory. - This flag allows specification of the desired properties of the - returned array object. All of the flags are explained in the detailed - API chapter. The flags most commonly needed are :cdata:`NPY_IN_ARRAY`, - :cdata:`NPY_OUT_ARRAY`, and :cdata:`NPY_INOUT_ARRAY`: - - .. cvar:: NPY_IN_ARRAY - - Equivalent to :cdata:`NPY_CONTIGUOUS` \| - :cdata:`NPY_ALIGNED`. This combination of flags is useful + The requirements flag allows specification of what kind of + array is acceptable. If the object passed in does not satisfy + this requirements then a copy is made so that thre returned + object will satisfy the requirements. these ndarray can use a + very generic pointer to memory. This flag allows specification + of the desired properties of the returned array object. All + of the flags are explained in the detailed API chapter. The + flags most commonly needed are :cdata:`NPY_ARRAY_IN_ARRAY`, + :cdata:`NPY_OUT_ARRAY`, and :cdata:`NPY_ARRAY_INOUT_ARRAY`: + + .. cvar:: NPY_ARRAY_IN_ARRAY + + Equivalent to :cdata:`NPY_ARRAY_C_CONTIGUOUS` \| + :cdata:`NPY_ARRAY_ALIGNED`. This combination of flags is useful for arrays that must be in C-contiguous order and aligned. These kinds of arrays are usually input arrays for some algorithm. - .. cvar:: NPY_OUT_ARRAY + .. cvar:: NPY_ARRAY_OUT_ARRAY - Equivalent to :cdata:`NPY_CONTIGUOUS` \| - :cdata:`NPY_ALIGNED` \| :cdata:`NPY_WRITEABLE`. This + Equivalent to :cdata:`NPY_ARRAY_C_CONTIGUOUS` \| + :cdata:`NPY_ARRAY_ALIGNED` \| :cdata:`NPY_ARRAY_WRITEABLE`. This combination of flags is useful to specify an array that is in C-contiguous order, is aligned, and can be written to as well. Such an array is usually returned as output (although normally such output arrays are created from scratch). - .. cvar:: NPY_INOUT_ARRAY + .. cvar:: NPY_ARRAY_INOUT_ARRAY - Equivalent to :cdata:`NPY_CONTIGUOUS` \| - :cdata:`NPY_ALIGNED` \| :cdata:`NPY_WRITEABLE` \| - :cdata:`NPY_UPDATEIFCOPY`. This combination of flags is + Equivalent to :cdata:`NPY_ARRAY_C_CONTIGUOUS` \| + :cdata:`NPY_ARRAY_ALIGNED` \| :cdata:`NPY_ARRAY_WRITEABLE` \| + :cdata:`NPY_ARRAY_UPDATEIFCOPY`. This combination of flags is useful to specify an array that will be used for both input and output. If a copy is needed, then when the temporary is deleted (by your use of :cfunc:`Py_DECREF` at the end of the interface routine), the temporary array will be copied back into the original array passed in. Use - of the :cdata:`UPDATEIFCOPY` flag requires that the input + of the :cdata:`NPY_ARRAY_UPDATEIFCOPY` flag requires that the input object is already an array (because other objects cannot be automatically updated in this fashion). If an error occurs use :cfunc:`PyArray_DECREF_ERR` (obj) on an array - with the :cdata:`NPY_UPDATEIFCOPY` flag set. This will + with the :cdata:`NPY_ARRAY_UPDATEIFCOPY` flag set. This will delete the array without causing the contents to be copied back into the original array. Other useful flags that can be OR'd as additional requirements are: - .. cvar:: NPY_FORCECAST + .. cvar:: NPY_ARRAY_FORCECAST Cast to the desired type, even if it can't be done without losing information. - .. cvar:: NPY_ENSURECOPY + .. cvar:: NPY_ARRAY_ENSURECOPY Make sure the resulting array is a copy of the original. - .. cvar:: NPY_ENSUREARRAY + .. cvar:: NPY_ARRAY_ENSUREARRAY Make sure the resulting object is an actual ndarray and not a sub- class. @@ -499,7 +500,7 @@ writeable). The syntax is Whether or not an array is byte-swapped is determined by the data-type of the array. Native byte-order arrays are always requested by :cfunc:`PyArray_FROM_OTF` and so there is no need for - a :cdata:`NPY_NOTSWAPPED` flag in the requirements argument. There + a :cdata:`NPY_ARRAY_NOTSWAPPED` flag in the requirements argument. There is also no way to get a byte-swapped array from this routine. diff --git a/numpy/core/SConscript b/numpy/core/SConscript index 55330a609..ca630254e 100644 --- a/numpy/core/SConscript +++ b/numpy/core/SConscript @@ -488,6 +488,7 @@ env.DistutilsPythonExtension('multiarray_tests', source=multiarray_tests_src) #------------------- if ENABLE_SEPARATE_COMPILATION: umathmodule_src.extend([pjoin('src', 'umath', 'ufunc_object.c')]) + umathmodule_src.extend([pjoin('src', 'umath', 'ufunc_type_resolution.c')]) umathmodule_src.extend(umath_loops_src) else: umathmodule_src = [pjoin('src', 'umath', 'umathmodule_onefile.c')] diff --git a/numpy/core/code_generators/genapi.py b/numpy/core/code_generators/genapi.py index f69424d31..b4d651b1d 100644 --- a/numpy/core/code_generators/genapi.py +++ b/numpy/core/code_generators/genapi.py @@ -51,6 +51,7 @@ API_FILES = [join('multiarray', 'methods.c'), join('multiarray', 'nditer_pywrap.c'), join('multiarray', 'einsum.c.src'), join('umath', 'ufunc_object.c'), + join('umath', 'ufunc_type_resolution.c'), join('umath', 'loops.c.src'), ] THIS_DIR = os.path.dirname(__file__) diff --git a/numpy/core/code_generators/numpy_api.py b/numpy/core/code_generators/numpy_api.py index a789ae683..b8cde7b3b 100644 --- a/numpy/core/code_generators/numpy_api.py +++ b/numpy/core/code_generators/numpy_api.py @@ -364,6 +364,7 @@ ufunc_funcs_api = { # End 1.6 API 'PyUFunc_DefaultTypeResolution': 39, 'PyUFunc_ValidateCasting': 40, + 'PyUFunc_DefaultTypeResolutionMasked': 41, } # List of all the dicts which define the C API diff --git a/numpy/core/include/numpy/ndarraytypes.h b/numpy/core/include/numpy/ndarraytypes.h index 0d8f35397..bf8af1661 100644 --- a/numpy/core/include/numpy/ndarraytypes.h +++ b/numpy/core/include/numpy/ndarraytypes.h @@ -1400,6 +1400,40 @@ PyArrayNeighborhoodIter_Next2D(PyArrayNeighborhoodIterObject* iter); #define PyDataType_ISNOTSWAPPED(d) PyArray_ISNBO(((PyArray_Descr *)(d))->byteorder) #define PyDataType_ISBYTESWAPPED(d) (!PyDataType_ISNOTSWAPPED(d)) +/************************************************************ + * NumPy Auxiliary Data for inner loops, sort functions, etc. + ************************************************************/ + +/* + * When creating an auxiliary data struct, this should always appear + * as the first member, like this: + * + * typedef struct { + * NpyAuxData base; + * double constant; + * } constant_multiplier_aux_data; + */ +typedef struct NpyAuxData_tag NpyAuxData; + +/* Function pointers for freeing or cloning auxiliary data */ +typedef void (NpyAuxData_FreeFunc) (NpyAuxData *); +typedef NpyAuxData *(NpyAuxData_CloneFunc) (NpyAuxData *); + +struct NpyAuxData_tag { + NpyAuxData_FreeFunc *free; + NpyAuxData_CloneFunc *clone; + /* To allow for a bit of expansion without breaking the ABI */ + void *reserved[2]; +}; + +/* Macros to use for freeing and cloning auxiliary data */ +#define NPY_AUXDATA_FREE(auxdata) \ + if ((auxdata) == NULL) \ + ; \ + else \ + ((auxdata)->free(auxdata)) +#define NPY_AUXDATA_CLONE(auxdata) \ + ((auxdata)->clone(auxdata)) /* * This is the form of the struct that's returned pointed by the diff --git a/numpy/core/include/numpy/ufuncobject.h b/numpy/core/include/numpy/ufuncobject.h index b4354b58c..c7096371d 100644 --- a/numpy/core/include/numpy/ufuncobject.h +++ b/numpy/core/include/numpy/ufuncobject.h @@ -7,7 +7,23 @@ extern "C" { #endif -typedef void (*PyUFuncGenericFunction) (char **, npy_intp *, npy_intp *, void *); +/* The most generic inner loop for a standard element-wise ufunc */ +typedef void (*PyUFuncGenericFunction) + (char **args, + npy_intp *dimensions, + npy_intp *steps, + void *innerloopdata); + +/* + * The most generic inner loop for a masked standard element-wise ufunc. + * The mask data and step is at args[narg] and steps[narg], after all + * the operands. + */ +typedef void (*PyUFuncGenericMaskedFunction) + (char **args, + npy_intp *dimensions, + npy_intp *steps, + NpyAuxData *innerloopdata); /* Forward declaration for the type resolution function */ struct _tagPyUFuncObject; @@ -18,6 +34,12 @@ struct _tagPyUFuncObject; * This function should validate that the casting rule is being followed, * and fail if it is not. * + * For backwards compatibility, the regular type resolution function does not + * support auxiliary data with object semantics. The type resolution call + * which returns a masked generic function returns a standard NpyAuxData + * object, for which the NPY_AUXDATA_FREE and NPY_AUXDATA_CLONE macros + * work. + * * ufunc: The ufunc object. * casting: The 'casting' parameter provided to the ufunc. * operands: An array of length (ufunc->nin + ufunc->nout), @@ -43,6 +65,14 @@ typedef int (PyUFunc_TypeResolutionFunc)( PyArray_Descr **out_dtypes, PyUFuncGenericFunction *out_innerloop, void **out_innerloopdata); +typedef int (PyUFunc_TypeResolutionMaskedFunc)( + struct _tagPyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericMaskedFunction *out_innerloop, + NpyAuxData **out_innerloopdata); typedef struct _tagPyUFuncObject { PyObject_HEAD @@ -112,6 +142,12 @@ typedef struct _tagPyUFuncObject { * have a different set of rules. */ PyUFunc_TypeResolutionFunc *type_resolution_function; + /* + * A function which resolves the types and returns an inner loop. + * This is used by the regular ufunc when it requires using + * a mask to select which elements to compute. + */ + PyUFunc_TypeResolutionMaskedFunc *type_resolution_masked_function; } PyUFuncObject; #include "arrayobject.h" diff --git a/numpy/core/setup.py b/numpy/core/setup.py index b0984dc5c..80f84d29f 100644 --- a/numpy/core/setup.py +++ b/numpy/core/setup.py @@ -822,7 +822,8 @@ def configuration(parent_package='',top_path=None): join('src', 'umath', 'umathmodule.c.src'), join('src', 'umath', 'funcs.inc.src'), join('src', 'umath', 'loops.c.src'), - join('src', 'umath', 'ufunc_object.c')] + join('src', 'umath', 'ufunc_object.c'), + join('src', 'umath', 'ufunc_type_resolution.c')] umath_deps = [ generate_umath_py, diff --git a/numpy/core/src/multiarray/convert.c b/numpy/core/src/multiarray/convert.c index e53166036..57ad8c22f 100644 --- a/numpy/core/src/multiarray/convert.c +++ b/numpy/core/src/multiarray/convert.c @@ -376,7 +376,7 @@ NPY_NO_EXPORT int PyArray_FillWithZero(PyArrayObject *a) { PyArray_StridedTransferFn *stransfer = NULL; - void *transferdata = NULL; + NpyAuxData *transferdata = NULL; PyArray_Descr *dtype = PyArray_DESCR(a); NpyIter *iter; @@ -456,7 +456,7 @@ PyArray_FillWithZero(PyArrayObject *a) NPY_END_THREADS; } - PyArray_FreeStridedTransferData(transferdata); + NPY_AUXDATA_FREE(transferdata); NpyIter_Deallocate(iter); return 0; diff --git a/numpy/core/src/multiarray/ctors.c b/numpy/core/src/multiarray/ctors.c index 6f1005dda..deaa152d7 100644 --- a/numpy/core/src/multiarray/ctors.c +++ b/numpy/core/src/multiarray/ctors.c @@ -2449,7 +2449,7 @@ PyArray_CopyAnyIntoOrdered(PyArrayObject *dst, PyArrayObject *src, NPY_ORDER order) { PyArray_StridedTransferFn *stransfer = NULL; - void *transferdata = NULL; + NpyAuxData *transferdata = NULL; NpyIter *dst_iter, *src_iter; NpyIter_IterNextFunc *dst_iternext, *src_iternext; @@ -2617,7 +2617,7 @@ PyArray_CopyAnyIntoOrdered(PyArrayObject *dst, PyArrayObject *src, NPY_END_THREADS; } - PyArray_FreeStridedTransferData(transferdata); + NPY_AUXDATA_FREE(transferdata); NpyIter_Deallocate(dst_iter); NpyIter_Deallocate(src_iter); @@ -2651,7 +2651,7 @@ NPY_NO_EXPORT int PyArray_CopyInto(PyArrayObject *dst, PyArrayObject *src) { PyArray_StridedTransferFn *stransfer = NULL; - void *transferdata = NULL; + NpyAuxData *transferdata = NULL; NPY_BEGIN_THREADS_DEF; if (!PyArray_ISWRITEABLE(dst)) { @@ -2706,7 +2706,7 @@ PyArray_CopyInto(PyArrayObject *dst, PyArrayObject *src) NPY_END_THREADS; } - PyArray_FreeStridedTransferData(transferdata); + NPY_AUXDATA_FREE(transferdata); return PyErr_Occurred() ? -1 : 0; } @@ -2791,7 +2791,7 @@ PyArray_CopyInto(PyArrayObject *dst, PyArrayObject *src) } } - PyArray_FreeStridedTransferData(transferdata); + NPY_AUXDATA_FREE(transferdata); NpyIter_Deallocate(iter); return PyErr_Occurred() ? -1 : 0; diff --git a/numpy/core/src/multiarray/dtype_transfer.c b/numpy/core/src/multiarray/dtype_transfer.c index b7febe8c2..123505d75 100644 --- a/numpy/core/src/multiarray/dtype_transfer.c +++ b/numpy/core/src/multiarray/dtype_transfer.c @@ -57,7 +57,7 @@ get_decsrcref_transfer_function(int aligned, npy_intp src_stride, PyArray_Descr *src_dtype, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api); /* @@ -70,7 +70,7 @@ get_setdstzero_transfer_function(int aligned, npy_intp dst_stride, PyArray_Descr *dst_dtype, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api); /* @@ -81,7 +81,7 @@ get_setdstzero_transfer_function(int aligned, NPY_NO_EXPORT int get_bool_setdstone_transfer_function(npy_intp dst_stride, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *NPY_UNUSED(out_needs_api)); /*************************** COPY REFERENCES *******************************/ @@ -91,7 +91,7 @@ static void _strided_to_strided_move_references(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { PyObject *src_ref = NULL, *dst_ref = NULL; while (N > 0) { @@ -119,7 +119,7 @@ static void _strided_to_strided_copy_references(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { PyObject *src_ref = NULL, *dst_ref = NULL; while (N > 0) { @@ -143,18 +143,14 @@ _strided_to_strided_copy_references(char *dst, npy_intp dst_stride, /************************** ZERO-PADDED COPY ******************************/ -typedef void (*free_strided_transfer_data)(void *); -typedef void *(*copy_strided_transfer_data)(void *); - /* Does a zero-padded copy */ typedef struct { - free_strided_transfer_data freefunc; - copy_strided_transfer_data copyfunc; + NpyAuxData base; npy_intp dst_itemsize; } _strided_zero_pad_data; /* zero-padded data copy function */ -void *_strided_zero_pad_data_copy(void *data) +NpyAuxData *_strided_zero_pad_data_clone(NpyAuxData *data) { _strided_zero_pad_data *newdata = (_strided_zero_pad_data *)PyArray_malloc( @@ -165,7 +161,7 @@ void *_strided_zero_pad_data_copy(void *data) memcpy(newdata, data, sizeof(_strided_zero_pad_data)); - return newdata; + return (NpyAuxData *)newdata; } /* @@ -176,7 +172,7 @@ static void _strided_to_strided_zero_pad_copy(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { _strided_zero_pad_data *d = (_strided_zero_pad_data *)data; npy_intp dst_itemsize = d->dst_itemsize; @@ -199,7 +195,7 @@ static void _strided_to_strided_truncate_copy(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { _strided_zero_pad_data *d = (_strided_zero_pad_data *)data; npy_intp dst_itemsize = d->dst_itemsize; @@ -217,7 +213,7 @@ PyArray_GetStridedZeroPadCopyFn(int aligned, npy_intp src_stride, npy_intp dst_stride, npy_intp src_itemsize, npy_intp dst_itemsize, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata) + NpyAuxData **out_transferdata) { if (src_itemsize == dst_itemsize) { *out_stransfer = PyArray_GetStridedCopyFn(aligned, src_stride, @@ -233,8 +229,8 @@ PyArray_GetStridedZeroPadCopyFn(int aligned, return NPY_FAIL; } d->dst_itemsize = dst_itemsize; - d->freefunc = &PyArray_free; - d->copyfunc = &_strided_zero_pad_data_copy; + d->base.free = (NpyAuxData_FreeFunc *)&PyArray_free; + d->base.clone = &_strided_zero_pad_data_clone; if (src_itemsize < dst_itemsize) { *out_stransfer = &_strided_to_strided_zero_pad_copy; @@ -243,7 +239,7 @@ PyArray_GetStridedZeroPadCopyFn(int aligned, *out_stransfer = &_strided_to_strided_truncate_copy; } - *out_transferdata = d; + *out_transferdata = (NpyAuxData *)d; return NPY_SUCCEED; } } @@ -252,27 +248,26 @@ PyArray_GetStridedZeroPadCopyFn(int aligned, /* Wraps a transfer function + data in alignment code */ typedef struct { - free_strided_transfer_data freefunc; - copy_strided_transfer_data copyfunc; + NpyAuxData base; PyArray_StridedTransferFn *wrapped, *tobuffer, *frombuffer; - void *wrappeddata, *todata, *fromdata; + NpyAuxData *wrappeddata, *todata, *fromdata; npy_intp src_itemsize, dst_itemsize; char *bufferin, *bufferout; } _align_wrap_data; /* transfer data free function */ -void _align_wrap_data_free(void *data) +void _align_wrap_data_free(NpyAuxData *data) { _align_wrap_data *d = (_align_wrap_data *)data; - PyArray_FreeStridedTransferData(d->wrappeddata); - PyArray_FreeStridedTransferData(d->todata); - PyArray_FreeStridedTransferData(d->fromdata); + NPY_AUXDATA_FREE(d->wrappeddata); + NPY_AUXDATA_FREE(d->todata); + NPY_AUXDATA_FREE(d->fromdata); PyArray_free(data); } /* transfer data copy function */ -void *_align_wrap_data_copy(void *data) +NpyAuxData *_align_wrap_data_clone(NpyAuxData *data) { _align_wrap_data *d = (_align_wrap_data *)data; _align_wrap_data *newdata; @@ -295,39 +290,38 @@ void *_align_wrap_data_copy(void *data) newdata->bufferout = newdata->bufferin + NPY_LOWLEVEL_BUFFER_BLOCKSIZE*newdata->src_itemsize; if (newdata->wrappeddata != NULL) { - newdata->wrappeddata = - PyArray_CopyStridedTransferData(d->wrappeddata); + newdata->wrappeddata = NPY_AUXDATA_CLONE(d->wrappeddata); if (newdata->wrappeddata == NULL) { PyArray_free(newdata); return NULL; } } if (newdata->todata != NULL) { - newdata->todata = PyArray_CopyStridedTransferData(d->todata); + newdata->todata = NPY_AUXDATA_CLONE(d->todata); if (newdata->todata == NULL) { - PyArray_FreeStridedTransferData(newdata->wrappeddata); + NPY_AUXDATA_FREE(newdata->wrappeddata); PyArray_free(newdata); return NULL; } } if (newdata->fromdata != NULL) { - newdata->fromdata = PyArray_CopyStridedTransferData(d->fromdata); + newdata->fromdata = NPY_AUXDATA_CLONE(d->fromdata); if (newdata->fromdata == NULL) { - PyArray_FreeStridedTransferData(newdata->wrappeddata); - PyArray_FreeStridedTransferData(newdata->todata); + NPY_AUXDATA_FREE(newdata->wrappeddata); + NPY_AUXDATA_FREE(newdata->todata); PyArray_free(newdata); return NULL; } } - return (void *)newdata; + return (NpyAuxData *)newdata; } static void _strided_to_strided_contig_align_wrap(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { _align_wrap_data *d = (_align_wrap_data *)data; PyArray_StridedTransferFn *wrapped = d->wrapped, @@ -335,7 +329,7 @@ _strided_to_strided_contig_align_wrap(char *dst, npy_intp dst_stride, *frombuffer = d->frombuffer; npy_intp inner_src_itemsize = d->src_itemsize, dst_itemsize = d->dst_itemsize; - void *wrappeddata = d->wrappeddata, + NpyAuxData *wrappeddata = d->wrappeddata, *todata = d->todata, *fromdata = d->fromdata; char *bufferin = d->bufferin, *bufferout = d->bufferout; @@ -371,7 +365,7 @@ static void _strided_to_strided_contig_align_wrap_init_dest(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { _align_wrap_data *d = (_align_wrap_data *)data; PyArray_StridedTransferFn *wrapped = d->wrapped, @@ -379,7 +373,7 @@ _strided_to_strided_contig_align_wrap_init_dest(char *dst, npy_intp dst_stride, *frombuffer = d->frombuffer; npy_intp inner_src_itemsize = d->src_itemsize, dst_itemsize = d->dst_itemsize; - void *wrappeddata = d->wrappeddata, + NpyAuxData *wrappeddata = d->wrappeddata, *todata = d->todata, *fromdata = d->fromdata; char *bufferin = d->bufferin, *bufferout = d->bufferout; @@ -431,12 +425,12 @@ _strided_to_strided_contig_align_wrap_init_dest(char *dst, npy_intp dst_stride, NPY_NO_EXPORT int wrap_aligned_contig_transfer_function( npy_intp src_itemsize, npy_intp dst_itemsize, - PyArray_StridedTransferFn *tobuffer, void *todata, - PyArray_StridedTransferFn *frombuffer, void *fromdata, - PyArray_StridedTransferFn *wrapped, void *wrappeddata, + PyArray_StridedTransferFn *tobuffer, NpyAuxData *todata, + PyArray_StridedTransferFn *frombuffer, NpyAuxData *fromdata, + PyArray_StridedTransferFn *wrapped, NpyAuxData *wrappeddata, int init_dest, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata) + NpyAuxData **out_transferdata) { _align_wrap_data *data; npy_intp basedatasize, datasize; @@ -454,8 +448,8 @@ wrap_aligned_contig_transfer_function( PyErr_NoMemory(); return NPY_FAIL; } - data->freefunc = &_align_wrap_data_free; - data->copyfunc = &_align_wrap_data_copy; + data->base.free = &_align_wrap_data_free; + data->base.clone = &_align_wrap_data_clone; data->tobuffer = tobuffer; data->todata = todata; data->frombuffer = frombuffer; @@ -475,7 +469,7 @@ wrap_aligned_contig_transfer_function( else { *out_stransfer = &_strided_to_strided_contig_align_wrap; } - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; return NPY_SUCCEED; } @@ -483,15 +477,14 @@ wrap_aligned_contig_transfer_function( /*************************** WRAP DTYPE COPY/SWAP *************************/ /* Wraps the dtype copy swap function */ typedef struct { - free_strided_transfer_data freefunc; - copy_strided_transfer_data copyfunc; + NpyAuxData base; PyArray_CopySwapNFunc *copyswapn; int swap; PyArrayObject *arr; } _wrap_copy_swap_data; /* wrap copy swap data free function */ -void _wrap_copy_swap_data_free(void *data) +void _wrap_copy_swap_data_free(NpyAuxData *data) { _wrap_copy_swap_data *d = (_wrap_copy_swap_data *)data; Py_DECREF(d->arr); @@ -499,7 +492,7 @@ void _wrap_copy_swap_data_free(void *data) } /* wrap copy swap data copy function */ -void *_wrap_copy_swap_data_copy(void *data) +NpyAuxData *_wrap_copy_swap_data_clone(NpyAuxData *data) { _wrap_copy_swap_data *newdata = (_wrap_copy_swap_data *)PyArray_malloc(sizeof(_wrap_copy_swap_data)); @@ -510,14 +503,14 @@ void *_wrap_copy_swap_data_copy(void *data) memcpy(newdata, data, sizeof(_wrap_copy_swap_data)); Py_INCREF(newdata->arr); - return (void *)newdata; + return (NpyAuxData *)newdata; } static void _strided_to_strided_wrap_copy_swap(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *data) + NpyAuxData *data) { _wrap_copy_swap_data *d = (_wrap_copy_swap_data *)data; @@ -531,7 +524,7 @@ wrap_copy_swap_function(int aligned, PyArray_Descr *dtype, int should_swap, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata) + NpyAuxData **out_transferdata) { _wrap_copy_swap_data *data; npy_intp shape = 1; @@ -545,8 +538,8 @@ wrap_copy_swap_function(int aligned, return NPY_FAIL; } - data->freefunc = &_wrap_copy_swap_data_free; - data->copyfunc = &_wrap_copy_swap_data_copy; + data->base.free = &_wrap_copy_swap_data_free; + data->base.clone = &_wrap_copy_swap_data_clone; data->copyswapn = dtype->f->copyswapn; data->swap = should_swap; @@ -563,7 +556,7 @@ wrap_copy_swap_function(int aligned, } *out_stransfer = &_strided_to_strided_wrap_copy_swap; - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; return NPY_SUCCEED; } @@ -572,14 +565,13 @@ wrap_copy_swap_function(int aligned, /* Does a simple aligned cast */ typedef struct { - free_strided_transfer_data freefunc; - copy_strided_transfer_data copyfunc; + NpyAuxData base; PyArray_VectorUnaryFunc *castfunc; PyArrayObject *aip, *aop; } _strided_cast_data; /* strided cast data free function */ -void _strided_cast_data_free(void *data) +void _strided_cast_data_free(NpyAuxData *data) { _strided_cast_data *d = (_strided_cast_data *)data; Py_DECREF(d->aip); @@ -588,7 +580,7 @@ void _strided_cast_data_free(void *data) } /* strided cast data copy function */ -void *_strided_cast_data_copy(void *data) +NpyAuxData *_strided_cast_data_clone(NpyAuxData *data) { _strided_cast_data *newdata = (_strided_cast_data *)PyArray_malloc(sizeof(_strided_cast_data)); @@ -600,14 +592,14 @@ void *_strided_cast_data_copy(void *data) Py_INCREF(newdata->aip); Py_INCREF(newdata->aop); - return (void *)newdata; + return (NpyAuxData *)newdata; } static void _aligned_strided_to_strided_cast(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { _strided_cast_data *d = (_strided_cast_data *)data; PyArray_VectorUnaryFunc *castfunc = d->castfunc; @@ -626,7 +618,7 @@ static void _aligned_strided_to_strided_cast_decref_src(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { _strided_cast_data *d = (_strided_cast_data *)data; PyArray_VectorUnaryFunc *castfunc = d->castfunc; @@ -651,7 +643,7 @@ static void _aligned_contig_to_contig_cast(char *dst, npy_intp NPY_UNUSED(dst_stride), char *src, npy_intp NPY_UNUSED(src_stride), npy_intp N, npy_intp NPY_UNUSED(itemsize), - void *data) + NpyAuxData *data) { _strided_cast_data *d = (_strided_cast_data *)data; @@ -663,7 +655,7 @@ get_nbo_cast_numeric_transfer_function(int aligned, npy_intp src_stride, npy_intp dst_stride, int src_type_num, int dst_type_num, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata) + NpyAuxData **out_transferdata) { /* Emit a warning if complex imaginary is being cast away */ if (PyTypeNum_ISCOMPLEX(src_type_num) && @@ -709,8 +701,7 @@ get_nbo_cast_numeric_transfer_function(int aligned, * datetime->ascii, or ascii->datetime cast */ typedef struct { - free_strided_transfer_data freefunc; - copy_strided_transfer_data copyfunc; + NpyAuxData base; /* The conversion fraction */ npy_int64 num, denom; /* For the datetime -> string conversion, the dst string length */ @@ -730,7 +721,7 @@ typedef struct { } _strided_datetime_cast_data; /* strided datetime cast data free function */ -void _strided_datetime_cast_data_free(void *data) +void _strided_datetime_cast_data_free(NpyAuxData *data) { _strided_datetime_cast_data *d = (_strided_datetime_cast_data *)data; if (d->tmp_buffer != NULL) { @@ -740,7 +731,7 @@ void _strided_datetime_cast_data_free(void *data) } /* strided datetime cast data copy function */ -void *_strided_datetime_cast_data_copy(void *data) +NpyAuxData *_strided_datetime_cast_data_clone(NpyAuxData *data) { _strided_datetime_cast_data *newdata = (_strided_datetime_cast_data *)PyArray_malloc( @@ -758,14 +749,14 @@ void *_strided_datetime_cast_data_copy(void *data) } } - return (void *)newdata; + return (NpyAuxData *)newdata; } static void _strided_to_strided_datetime_general_cast(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { _strided_datetime_cast_data *d = (_strided_datetime_cast_data *)data; npy_int64 dt; @@ -797,7 +788,7 @@ static void _strided_to_strided_datetime_cast(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { _strided_datetime_cast_data *d = (_strided_datetime_cast_data *)data; npy_int64 num = d->num, denom = d->denom; @@ -829,7 +820,7 @@ _aligned_strided_to_strided_datetime_cast(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { _strided_datetime_cast_data *d = (_strided_datetime_cast_data *)data; npy_int64 num = d->num, denom = d->denom; @@ -860,7 +851,7 @@ static void _strided_to_strided_datetime_to_string(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *data) + NpyAuxData *data) { _strided_datetime_cast_data *d = (_strided_datetime_cast_data *)data; npy_intp dst_itemsize = d->dst_itemsize; @@ -897,7 +888,7 @@ static void _strided_to_strided_string_to_datetime(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { _strided_datetime_cast_data *d = (_strided_datetime_cast_data *)data; npy_int64 dt; @@ -952,7 +943,7 @@ get_nbo_cast_datetime_transfer_function(int aligned, npy_intp src_stride, npy_intp dst_stride, PyArray_Descr *src_dtype, PyArray_Descr *dst_dtype, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata) + NpyAuxData **out_transferdata) { PyArray_DatetimeMetaData *src_meta, *dst_meta; npy_int64 num = 0, denom = 0; @@ -982,8 +973,8 @@ get_nbo_cast_datetime_transfer_function(int aligned, *out_transferdata = NULL; return NPY_FAIL; } - data->freefunc = &_strided_datetime_cast_data_free; - data->copyfunc = &_strided_datetime_cast_data_copy; + data->base.free = &_strided_datetime_cast_data_free; + data->base.clone = &_strided_datetime_cast_data_clone; data->num = num; data->denom = denom; data->tmp_buffer = NULL; @@ -1008,7 +999,7 @@ get_nbo_cast_datetime_transfer_function(int aligned, else { *out_stransfer = &_strided_to_strided_datetime_cast; } - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; #if NPY_DT_DBG_TRACING printf("Dtype transfer from "); @@ -1028,7 +1019,7 @@ get_nbo_datetime_to_string_transfer_function(int aligned, npy_intp src_stride, npy_intp dst_stride, PyArray_Descr *src_dtype, PyArray_Descr *dst_dtype, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata) + NpyAuxData **out_transferdata) { PyArray_DatetimeMetaData *src_meta; _strided_datetime_cast_data *data; @@ -1047,15 +1038,15 @@ get_nbo_datetime_to_string_transfer_function(int aligned, *out_transferdata = NULL; return NPY_FAIL; } - data->freefunc = &_strided_datetime_cast_data_free; - data->copyfunc = &_strided_datetime_cast_data_copy; + data->base.free = &_strided_datetime_cast_data_free; + data->base.clone = &_strided_datetime_cast_data_clone; data->dst_itemsize = dst_dtype->elsize; data->tmp_buffer = NULL; memcpy(&data->src_meta, src_meta, sizeof(data->src_meta)); *out_stransfer = &_strided_to_strided_datetime_to_string; - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; #if NPY_DT_DBG_TRACING printf("Dtype transfer from "); @@ -1073,10 +1064,10 @@ get_datetime_to_unicode_transfer_function(int aligned, npy_intp src_stride, npy_intp dst_stride, PyArray_Descr *src_dtype, PyArray_Descr *dst_dtype, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api) { - void *castdata = NULL, *todata = NULL, *fromdata = NULL; + NpyAuxData *castdata = NULL, *todata = NULL, *fromdata = NULL; PyArray_StridedTransferFn *caststransfer, *tobuffer, *frombuffer; PyArray_Descr *str_dtype; @@ -1102,7 +1093,7 @@ get_datetime_to_unicode_transfer_function(int aligned, src_dtype, str_dtype, &caststransfer, &castdata) != NPY_SUCCEED) { Py_DECREF(str_dtype); - PyArray_FreeStridedTransferData(todata); + NPY_AUXDATA_FREE(todata); return NPY_FAIL; } @@ -1114,8 +1105,8 @@ get_datetime_to_unicode_transfer_function(int aligned, &frombuffer, &fromdata, out_needs_api) != NPY_SUCCEED) { Py_DECREF(str_dtype); - PyArray_FreeStridedTransferData(todata); - PyArray_FreeStridedTransferData(castdata); + NPY_AUXDATA_FREE(todata); + NPY_AUXDATA_FREE(castdata); return NPY_FAIL; } @@ -1127,9 +1118,9 @@ get_datetime_to_unicode_transfer_function(int aligned, caststransfer, castdata, PyDataType_FLAGCHK(str_dtype, NPY_NEEDS_INIT), out_stransfer, out_transferdata) != NPY_SUCCEED) { - PyArray_FreeStridedTransferData(castdata); - PyArray_FreeStridedTransferData(todata); - PyArray_FreeStridedTransferData(fromdata); + NPY_AUXDATA_FREE(castdata); + NPY_AUXDATA_FREE(todata); + NPY_AUXDATA_FREE(fromdata); return NPY_FAIL; } @@ -1143,7 +1134,7 @@ get_nbo_string_to_datetime_transfer_function(int aligned, npy_intp src_stride, npy_intp dst_stride, PyArray_Descr *src_dtype, PyArray_Descr *dst_dtype, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata) + NpyAuxData **out_transferdata) { PyArray_DatetimeMetaData *dst_meta; _strided_datetime_cast_data *data; @@ -1162,8 +1153,8 @@ get_nbo_string_to_datetime_transfer_function(int aligned, *out_transferdata = NULL; return NPY_FAIL; } - data->freefunc = &_strided_datetime_cast_data_free; - data->copyfunc = &_strided_datetime_cast_data_copy; + data->base.free = &_strided_datetime_cast_data_free; + data->base.clone = &_strided_datetime_cast_data_clone; data->src_itemsize = src_dtype->elsize; data->tmp_buffer = PyArray_malloc(data->src_itemsize + 1); if (data->tmp_buffer == NULL) { @@ -1177,7 +1168,7 @@ get_nbo_string_to_datetime_transfer_function(int aligned, memcpy(&data->dst_meta, dst_meta, sizeof(data->dst_meta)); *out_stransfer = &_strided_to_strided_string_to_datetime; - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; #if NPY_DT_DBG_TRACING printf("Dtype transfer from "); @@ -1195,10 +1186,10 @@ get_unicode_to_datetime_transfer_function(int aligned, npy_intp src_stride, npy_intp dst_stride, PyArray_Descr *src_dtype, PyArray_Descr *dst_dtype, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api) { - void *castdata = NULL, *todata = NULL, *fromdata = NULL; + NpyAuxData *castdata = NULL, *todata = NULL, *fromdata = NULL; PyArray_StridedTransferFn *caststransfer, *tobuffer, *frombuffer; PyArray_Descr *str_dtype; @@ -1226,7 +1217,7 @@ get_unicode_to_datetime_transfer_function(int aligned, str_dtype, dst_dtype, &caststransfer, &castdata) != NPY_SUCCEED) { Py_DECREF(str_dtype); - PyArray_FreeStridedTransferData(todata); + NPY_AUXDATA_FREE(todata); return NPY_FAIL; } @@ -1236,8 +1227,8 @@ get_unicode_to_datetime_transfer_function(int aligned, dst_dtype, &frombuffer, &fromdata) != NPY_SUCCEED) { Py_DECREF(str_dtype); - PyArray_FreeStridedTransferData(todata); - PyArray_FreeStridedTransferData(castdata); + NPY_AUXDATA_FREE(todata); + NPY_AUXDATA_FREE(castdata); return NPY_FAIL; } @@ -1250,9 +1241,9 @@ get_unicode_to_datetime_transfer_function(int aligned, PyDataType_FLAGCHK(dst_dtype, NPY_NEEDS_INIT), out_stransfer, out_transferdata) != NPY_SUCCEED) { Py_DECREF(str_dtype); - PyArray_FreeStridedTransferData(castdata); - PyArray_FreeStridedTransferData(todata); - PyArray_FreeStridedTransferData(fromdata); + NPY_AUXDATA_FREE(castdata); + NPY_AUXDATA_FREE(todata); + NPY_AUXDATA_FREE(fromdata); return NPY_FAIL; } @@ -1267,7 +1258,7 @@ get_nbo_cast_transfer_function(int aligned, PyArray_Descr *src_dtype, PyArray_Descr *dst_dtype, int move_references, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api, int *out_needs_wrap) { @@ -1393,8 +1384,8 @@ get_nbo_cast_transfer_function(int aligned, *out_transferdata = NULL; return NPY_FAIL; } - data->freefunc = &_strided_cast_data_free; - data->copyfunc = &_strided_cast_data_copy; + data->base.free = &_strided_cast_data_free; + data->base.clone = &_strided_cast_data_clone; data->castfunc = castfunc; /* * TODO: This is a hack so the cast functions have an array. @@ -1463,7 +1454,7 @@ get_nbo_cast_transfer_function(int aligned, *out_stransfer = _aligned_strided_to_strided_cast; } } - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; return NPY_SUCCEED; } @@ -1474,11 +1465,11 @@ get_cast_transfer_function(int aligned, PyArray_Descr *src_dtype, PyArray_Descr *dst_dtype, int move_references, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api) { PyArray_StridedTransferFn *caststransfer; - void *castdata, *todata = NULL, *fromdata = NULL; + NpyAuxData *castdata, *todata = NULL, *fromdata = NULL; int needs_wrap = 0; npy_intp src_itemsize = src_dtype->elsize, dst_itemsize = dst_dtype->elsize; @@ -1530,9 +1521,9 @@ get_cast_transfer_function(int aligned, &frombuffer, &fromdata); if (frombuffer == NULL || tobuffer == NULL) { - PyArray_FreeStridedTransferData(castdata); - PyArray_FreeStridedTransferData(todata); - PyArray_FreeStridedTransferData(fromdata); + NPY_AUXDATA_FREE(castdata); + NPY_AUXDATA_FREE(todata); + NPY_AUXDATA_FREE(fromdata); return NPY_FAIL; } @@ -1546,9 +1537,9 @@ get_cast_transfer_function(int aligned, caststransfer, castdata, PyDataType_FLAGCHK(dst_dtype, NPY_NEEDS_INIT), out_stransfer, out_transferdata) != NPY_SUCCEED) { - PyArray_FreeStridedTransferData(castdata); - PyArray_FreeStridedTransferData(todata); - PyArray_FreeStridedTransferData(fromdata); + NPY_AUXDATA_FREE(castdata); + NPY_AUXDATA_FREE(todata); + NPY_AUXDATA_FREE(fromdata); return NPY_FAIL; } @@ -1560,27 +1551,26 @@ get_cast_transfer_function(int aligned, /* Copies 1 element to N contiguous elements */ typedef struct { - free_strided_transfer_data freefunc; - copy_strided_transfer_data copyfunc; + NpyAuxData base; PyArray_StridedTransferFn *stransfer; - void *data; + NpyAuxData *data; npy_intp N, dst_itemsize; /* If this is non-NULL the source type has references needing a decref */ PyArray_StridedTransferFn *stransfer_finish_src; - void *data_finish_src; + NpyAuxData *data_finish_src; } _one_to_n_data; /* transfer data free function */ -void _one_to_n_data_free(void *data) +void _one_to_n_data_free(NpyAuxData *data) { _one_to_n_data *d = (_one_to_n_data *)data; - PyArray_FreeStridedTransferData(d->data); - PyArray_FreeStridedTransferData(d->data_finish_src); + NPY_AUXDATA_FREE(d->data); + NPY_AUXDATA_FREE(d->data_finish_src); PyArray_free(data); } /* transfer data copy function */ -void *_one_to_n_data_copy(void *data) +NpyAuxData *_one_to_n_data_clone(NpyAuxData *data) { _one_to_n_data *d = (_one_to_n_data *)data; _one_to_n_data *newdata; @@ -1592,34 +1582,33 @@ void *_one_to_n_data_copy(void *data) } memcpy(newdata, data, sizeof(_one_to_n_data)); if (d->data != NULL) { - newdata->data = PyArray_CopyStridedTransferData(d->data); + newdata->data = NPY_AUXDATA_CLONE(d->data); if (newdata->data == NULL) { PyArray_free(newdata); return NULL; } } if (d->data_finish_src != NULL) { - newdata->data_finish_src = - PyArray_CopyStridedTransferData(d->data_finish_src); + newdata->data_finish_src = NPY_AUXDATA_CLONE(d->data_finish_src); if (newdata->data_finish_src == NULL) { - PyArray_FreeStridedTransferData(newdata->data); + NPY_AUXDATA_FREE(newdata->data); PyArray_free(newdata); return NULL; } } - return (void *)newdata; + return (NpyAuxData *)newdata; } static void _strided_to_strided_one_to_n(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { _one_to_n_data *d = (_one_to_n_data *)data; PyArray_StridedTransferFn *subtransfer = d->stransfer; - void *subdata = d->data; + NpyAuxData *subdata = d->data; npy_intp subN = d->N, dst_itemsize = d->dst_itemsize; while (N > 0) { @@ -1638,12 +1627,12 @@ static void _strided_to_strided_one_to_n_with_finish(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { _one_to_n_data *d = (_one_to_n_data *)data; PyArray_StridedTransferFn *subtransfer = d->stransfer, *stransfer_finish_src = d->stransfer_finish_src; - void *subdata = d->data, *data_finish_src = data_finish_src; + NpyAuxData *subdata = d->data, *data_finish_src = data_finish_src; npy_intp subN = d->N, dst_itemsize = d->dst_itemsize; while (N > 0) { @@ -1673,13 +1662,13 @@ _strided_to_strided_one_to_n_with_finish(char *dst, npy_intp dst_stride, static int wrap_transfer_function_one_to_n( PyArray_StridedTransferFn *stransfer_inner, - void *data_inner, + NpyAuxData *data_inner, PyArray_StridedTransferFn *stransfer_finish_src, - void *data_finish_src, + NpyAuxData *data_finish_src, npy_intp dst_itemsize, npy_intp N, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata) + NpyAuxData **out_transferdata) { _one_to_n_data *data; @@ -1690,8 +1679,8 @@ wrap_transfer_function_one_to_n( return NPY_FAIL; } - data->freefunc = &_one_to_n_data_free; - data->copyfunc = &_one_to_n_data_copy; + data->base.free = &_one_to_n_data_free; + data->base.clone = &_one_to_n_data_clone; data->stransfer = stransfer_inner; data->data = data_inner; data->stransfer_finish_src = stransfer_finish_src; @@ -1705,7 +1694,7 @@ wrap_transfer_function_one_to_n( else { *out_stransfer = &_strided_to_strided_one_to_n_with_finish; } - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; return NPY_SUCCEED; } @@ -1717,11 +1706,11 @@ get_one_to_n_transfer_function(int aligned, int move_references, npy_intp N, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api) { PyArray_StridedTransferFn *stransfer, *stransfer_finish_src = NULL; - void *data, *data_finish_src = NULL; + NpyAuxData *data, *data_finish_src = NULL; /* * move_references is set to 0, handled in the wrapping transfer fn, @@ -1746,7 +1735,7 @@ get_one_to_n_transfer_function(int aligned, &stransfer_finish_src, &data_finish_src, out_needs_api) != NPY_SUCCEED) { - PyArray_FreeStridedTransferData(data); + NPY_AUXDATA_FREE(data); return NPY_FAIL; } } @@ -1756,8 +1745,8 @@ get_one_to_n_transfer_function(int aligned, dst_dtype->elsize, N, out_stransfer, out_transferdata) != NPY_SUCCEED) { - PyArray_FreeStridedTransferData(data); - PyArray_FreeStridedTransferData(data_finish_src); + NPY_AUXDATA_FREE(data); + NPY_AUXDATA_FREE(data_finish_src); return NPY_FAIL; } @@ -1768,23 +1757,22 @@ get_one_to_n_transfer_function(int aligned, /* Copies N contiguous elements to N contiguous elements */ typedef struct { - free_strided_transfer_data freefunc; - copy_strided_transfer_data copyfunc; + NpyAuxData base; PyArray_StridedTransferFn *stransfer; - void *data; + NpyAuxData *data; npy_intp N, src_itemsize, dst_itemsize; } _n_to_n_data; /* transfer data free function */ -void _n_to_n_data_free(void *data) +void _n_to_n_data_free(NpyAuxData *data) { _n_to_n_data *d = (_n_to_n_data *)data; - PyArray_FreeStridedTransferData(d->data); + NPY_AUXDATA_FREE(d->data); PyArray_free(data); } /* transfer data copy function */ -void *_n_to_n_data_copy(void *data) +NpyAuxData *_n_to_n_data_clone(NpyAuxData *data) { _n_to_n_data *d = (_n_to_n_data *)data; _n_to_n_data *newdata; @@ -1796,25 +1784,25 @@ void *_n_to_n_data_copy(void *data) } memcpy(newdata, data, sizeof(_n_to_n_data)); if (newdata->data != NULL) { - newdata->data = PyArray_CopyStridedTransferData(d->data); + newdata->data = NPY_AUXDATA_CLONE(d->data); if (newdata->data == NULL) { PyArray_free(newdata); return NULL; } } - return (void *)newdata; + return (NpyAuxData *)newdata; } static void _strided_to_strided_n_to_n(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *data) + NpyAuxData *data) { _n_to_n_data *d = (_n_to_n_data *)data; PyArray_StridedTransferFn *subtransfer = d->stransfer; - void *subdata = d->data; + NpyAuxData *subdata = d->data; npy_intp subN = d->N, src_subitemsize = d->src_itemsize, dst_subitemsize = d->dst_itemsize; @@ -1834,11 +1822,11 @@ static void _contig_to_contig_n_to_n(char *dst, npy_intp NPY_UNUSED(dst_stride), char *src, npy_intp NPY_UNUSED(src_stride), npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *data) + NpyAuxData *data) { _n_to_n_data *d = (_n_to_n_data *)data; PyArray_StridedTransferFn *subtransfer = d->stransfer; - void *subdata = d->data; + NpyAuxData *subdata = d->data; npy_intp subN = d->N, src_subitemsize = d->src_itemsize, dst_subitemsize = d->dst_itemsize; @@ -1855,12 +1843,12 @@ _contig_to_contig_n_to_n(char *dst, npy_intp NPY_UNUSED(dst_stride), static int wrap_transfer_function_n_to_n( PyArray_StridedTransferFn *stransfer_inner, - void *data_inner, + NpyAuxData *data_inner, npy_intp src_stride, npy_intp dst_stride, npy_intp src_itemsize, npy_intp dst_itemsize, npy_intp N, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata) + NpyAuxData **out_transferdata) { _n_to_n_data *data; @@ -1870,8 +1858,8 @@ wrap_transfer_function_n_to_n( return NPY_FAIL; } - data->freefunc = &_n_to_n_data_free; - data->copyfunc = &_n_to_n_data_copy; + data->base.free = &_n_to_n_data_free; + data->base.clone = &_n_to_n_data_clone; data->stransfer = stransfer_inner; data->data = data_inner; data->N = N; @@ -1889,7 +1877,7 @@ wrap_transfer_function_n_to_n( else { *out_stransfer = &_strided_to_strided_n_to_n; } - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; return NPY_SUCCEED; } @@ -1901,11 +1889,11 @@ get_n_to_n_transfer_function(int aligned, int move_references, npy_intp N, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api) { PyArray_StridedTransferFn *stransfer; - void *data; + NpyAuxData *data; /* * src_stride and dst_stride are set to contiguous, because @@ -1926,7 +1914,7 @@ get_n_to_n_transfer_function(int aligned, N, out_stransfer, out_transferdata) != NPY_SUCCEED) { - PyArray_FreeStridedTransferData(data); + NPY_AUXDATA_FREE(data); return NPY_FAIL; } @@ -1941,32 +1929,31 @@ typedef struct { /* Copies element with subarray broadcasting */ typedef struct { - free_strided_transfer_data freefunc; - copy_strided_transfer_data copyfunc; + NpyAuxData base; PyArray_StridedTransferFn *stransfer; - void *data; + NpyAuxData *data; npy_intp src_N, dst_N, src_itemsize, dst_itemsize; PyArray_StridedTransferFn *stransfer_decsrcref; - void *data_decsrcref; + NpyAuxData *data_decsrcref; PyArray_StridedTransferFn *stransfer_decdstref; - void *data_decdstref; + NpyAuxData *data_decdstref; /* This gets a run-length encoded representation of the transfer */ npy_intp run_count; _subarray_broadcast_offsetrun offsetruns; } _subarray_broadcast_data; /* transfer data free function */ -void _subarray_broadcast_data_free(void *data) +void _subarray_broadcast_data_free(NpyAuxData *data) { _subarray_broadcast_data *d = (_subarray_broadcast_data *)data; - PyArray_FreeStridedTransferData(d->data); - PyArray_FreeStridedTransferData(d->data_decsrcref); - PyArray_FreeStridedTransferData(d->data_decdstref); + NPY_AUXDATA_FREE(d->data); + NPY_AUXDATA_FREE(d->data_decsrcref); + NPY_AUXDATA_FREE(d->data_decdstref); PyArray_free(data); } /* transfer data copy function */ -void *_subarray_broadcast_data_copy( void *data) +NpyAuxData *_subarray_broadcast_data_clone( NpyAuxData *data) { _subarray_broadcast_data *d = (_subarray_broadcast_data *)data; _subarray_broadcast_data *newdata; @@ -1982,44 +1969,42 @@ void *_subarray_broadcast_data_copy( void *data) } memcpy(newdata, data, structsize); if (d->data != NULL) { - newdata->data = PyArray_CopyStridedTransferData(d->data); + newdata->data = NPY_AUXDATA_CLONE(d->data); if (newdata->data == NULL) { PyArray_free(newdata); return NULL; } } if (d->data_decsrcref != NULL) { - newdata->data_decsrcref = - PyArray_CopyStridedTransferData(d->data_decsrcref); + newdata->data_decsrcref = NPY_AUXDATA_CLONE(d->data_decsrcref); if (newdata->data_decsrcref == NULL) { - PyArray_FreeStridedTransferData(newdata->data); + NPY_AUXDATA_FREE(newdata->data); PyArray_free(newdata); return NULL; } } if (d->data_decdstref != NULL) { - newdata->data_decdstref = - PyArray_CopyStridedTransferData(d->data_decdstref); + newdata->data_decdstref = NPY_AUXDATA_CLONE(d->data_decdstref); if (newdata->data_decdstref == NULL) { - PyArray_FreeStridedTransferData(newdata->data); - PyArray_FreeStridedTransferData(newdata->data_decsrcref); + NPY_AUXDATA_FREE(newdata->data); + NPY_AUXDATA_FREE(newdata->data_decsrcref); PyArray_free(newdata); return NULL; } } - return newdata; + return (NpyAuxData *)newdata; } static void _strided_to_strided_subarray_broadcast(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *data) + NpyAuxData *data) { _subarray_broadcast_data *d = (_subarray_broadcast_data *)data; PyArray_StridedTransferFn *subtransfer = d->stransfer; - void *subdata = d->data; + NpyAuxData *subdata = d->data; npy_intp run, run_count = d->run_count, src_subitemsize = d->src_itemsize, dst_subitemsize = d->dst_itemsize; @@ -2056,15 +2041,15 @@ static void _strided_to_strided_subarray_broadcast_withrefs(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *data) + NpyAuxData *data) { _subarray_broadcast_data *d = (_subarray_broadcast_data *)data; PyArray_StridedTransferFn *subtransfer = d->stransfer; - void *subdata = d->data; + NpyAuxData *subdata = d->data; PyArray_StridedTransferFn *stransfer_decsrcref = d->stransfer_decsrcref; - void *data_decsrcref = d->data_decsrcref; + NpyAuxData *data_decsrcref = d->data_decsrcref; PyArray_StridedTransferFn *stransfer_decdstref = d->stransfer_decdstref; - void *data_decdstref = d->data_decdstref; + NpyAuxData *data_decdstref = d->data_decdstref; npy_intp run, run_count = d->run_count, src_subitemsize = d->src_itemsize, dst_subitemsize = d->dst_itemsize, @@ -2117,7 +2102,7 @@ get_subarray_broadcast_transfer_function(int aligned, PyArray_Dims src_shape, PyArray_Dims dst_shape, int move_references, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api) { _subarray_broadcast_data *data; @@ -2149,8 +2134,8 @@ get_subarray_broadcast_transfer_function(int aligned, PyArray_free(data); return NPY_FAIL; } - data->freefunc = &_subarray_broadcast_data_free; - data->copyfunc = &_subarray_broadcast_data_copy; + data->base.free = &_subarray_broadcast_data_free; + data->base.clone = &_subarray_broadcast_data_clone; data->src_N = src_size; data->dst_N = dst_size; data->src_itemsize = src_dtype->elsize; @@ -2165,7 +2150,7 @@ get_subarray_broadcast_transfer_function(int aligned, &data->stransfer_decsrcref, &data->data_decsrcref, out_needs_api) != NPY_SUCCEED) { - PyArray_FreeStridedTransferData(data->data); + NPY_AUXDATA_FREE(data->data); PyArray_free(data); return NPY_FAIL; } @@ -2184,8 +2169,8 @@ get_subarray_broadcast_transfer_function(int aligned, &data->stransfer_decdstref, &data->data_decdstref, out_needs_api) != NPY_SUCCEED) { - PyArray_FreeStridedTransferData(data->data); - PyArray_FreeStridedTransferData(data->data_decsrcref); + NPY_AUXDATA_FREE(data->data); + NPY_AUXDATA_FREE(data->data_decsrcref); PyArray_free(data); return NPY_FAIL; } @@ -2289,7 +2274,7 @@ get_subarray_broadcast_transfer_function(int aligned, else { *out_stransfer = &_strided_to_strided_subarray_broadcast_withrefs; } - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; return NPY_SUCCEED; } @@ -2304,7 +2289,7 @@ get_subarray_transfer_function(int aligned, PyArray_Descr *src_dtype, PyArray_Descr *dst_dtype, int move_references, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api) { PyArray_Dims src_shape = {NULL, -1}, dst_shape = {NULL, -1}; @@ -2401,19 +2386,18 @@ get_subarray_transfer_function(int aligned, typedef struct { npy_intp src_offset, dst_offset, src_itemsize; PyArray_StridedTransferFn *stransfer; - void *data; + NpyAuxData *data; } _single_field_transfer; typedef struct { - free_strided_transfer_data freefunc; - copy_strided_transfer_data copyfunc; + NpyAuxData base; npy_intp field_count; _single_field_transfer fields; } _field_transfer_data; /* transfer data free function */ -void _field_transfer_data_free(void *data) +void _field_transfer_data_free(NpyAuxData *data) { _field_transfer_data *d = (_field_transfer_data *)data; npy_intp i, field_count; @@ -2423,13 +2407,13 @@ void _field_transfer_data_free(void *data) fields = &d->fields; for (i = 0; i < field_count; ++i) { - PyArray_FreeStridedTransferData(fields[i].data); + NPY_AUXDATA_FREE(fields[i].data); } PyArray_free(d); } /* transfer data copy function */ -void *_field_transfer_data_copy(void *data) +NpyAuxData *_field_transfer_data_clone(NpyAuxData *data) { _field_transfer_data *d = (_field_transfer_data *)data; _field_transfer_data *newdata; @@ -2450,11 +2434,10 @@ void *_field_transfer_data_copy(void *data) newfields = &newdata->fields; for (i = 0; i < field_count; ++i) { if (fields[i].data != NULL) { - newfields[i].data = - PyArray_CopyStridedTransferData(fields[i].data); + newfields[i].data = NPY_AUXDATA_CLONE(fields[i].data); if (newfields[i].data == NULL) { for (i = i-1; i >= 0; --i) { - PyArray_FreeStridedTransferData(newfields[i].data); + NPY_AUXDATA_FREE(newfields[i].data); } PyArray_free(newdata); return NULL; @@ -2463,14 +2446,14 @@ void *_field_transfer_data_copy(void *data) } - return (void *)newdata; + return (NpyAuxData *)newdata; } static void _strided_to_strided_field_transfer(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *data) + NpyAuxData *data) { _field_transfer_data *d = (_field_transfer_data *)data; npy_intp i, field_count = d->field_count; @@ -2514,7 +2497,7 @@ get_fields_transfer_function(int aligned, PyArray_Descr *src_dtype, PyArray_Descr *dst_dtype, int move_references, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api) { PyObject *names, *key, *tup, *title; @@ -2538,8 +2521,8 @@ get_fields_transfer_function(int aligned, PyErr_NoMemory(); return NPY_FAIL; } - data->freefunc = &_field_transfer_data_free; - data->copyfunc = &_field_transfer_data_copy; + data->base.free = &_field_transfer_data_free; + data->base.clone = &_field_transfer_data_clone; fields = &data->fields; for (i = 0; i < names_size; ++i) { @@ -2558,7 +2541,7 @@ get_fields_transfer_function(int aligned, &fields[i].data, out_needs_api) != NPY_SUCCEED) { for (i = i-1; i >= 0; --i) { - PyArray_FreeStridedTransferData(fields[i].data); + NPY_AUXDATA_FREE(fields[i].data); } PyArray_free(data); return NPY_FAIL; @@ -2580,7 +2563,7 @@ get_fields_transfer_function(int aligned, &fields[field_count].data, out_needs_api) != NPY_SUCCEED) { for (i = 0; i < field_count; ++i) { - PyArray_FreeStridedTransferData(fields[i].data); + NPY_AUXDATA_FREE(fields[i].data); } PyArray_free(data); return NPY_FAIL; @@ -2593,7 +2576,7 @@ get_fields_transfer_function(int aligned, data->field_count = field_count; *out_stransfer = &_strided_to_strided_field_transfer; - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; return NPY_SUCCEED; } @@ -2620,8 +2603,8 @@ get_fields_transfer_function(int aligned, PyErr_NoMemory(); return NPY_FAIL; } - data->freefunc = &_field_transfer_data_free; - data->copyfunc = &_field_transfer_data_copy; + data->base.free = &_field_transfer_data_free; + data->base.clone = &_field_transfer_data_clone; fields = &data->fields; key = PyTuple_GET_ITEM(names, 0); @@ -2659,7 +2642,7 @@ get_fields_transfer_function(int aligned, &fields[field_count].stransfer, &fields[field_count].data, out_needs_api) != NPY_SUCCEED) { - PyArray_FreeStridedTransferData(fields[0].data); + NPY_AUXDATA_FREE(fields[0].data); PyArray_free(data); return NPY_FAIL; } @@ -2708,7 +2691,7 @@ get_fields_transfer_function(int aligned, &fields[field_count].data, out_needs_api) != NPY_SUCCEED) { for (i = field_count-1; i >= 0; --i) { - PyArray_FreeStridedTransferData(fields[i].data); + NPY_AUXDATA_FREE(fields[i].data); } PyArray_free(data); return NPY_FAIL; @@ -2724,7 +2707,7 @@ get_fields_transfer_function(int aligned, data->field_count = field_count; *out_stransfer = &_strided_to_strided_field_transfer; - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; return NPY_SUCCEED; } @@ -2759,8 +2742,8 @@ get_fields_transfer_function(int aligned, Py_XDECREF(used_names_dict); return NPY_FAIL; } - data->freefunc = &_field_transfer_data_free; - data->copyfunc = &_field_transfer_data_copy; + data->base.free = &_field_transfer_data_free; + data->base.clone = &_field_transfer_data_clone; fields = &data->fields; for (i = 0; i < names_size; ++i) { @@ -2769,7 +2752,7 @@ get_fields_transfer_function(int aligned, if (!PyArg_ParseTuple(tup, "Oi|O", &dst_fld_dtype, &dst_offset, &title)) { for (i = i-1; i >= 0; --i) { - PyArray_FreeStridedTransferData(fields[i].data); + NPY_AUXDATA_FREE(fields[i].data); } PyArray_free(data); Py_XDECREF(used_names_dict); @@ -2780,7 +2763,7 @@ get_fields_transfer_function(int aligned, if (!PyArg_ParseTuple(tup, "Oi|O", &src_fld_dtype, &src_offset, &title)) { for (i = i-1; i >= 0; --i) { - PyArray_FreeStridedTransferData(fields[i].data); + NPY_AUXDATA_FREE(fields[i].data); } PyArray_free(data); Py_XDECREF(used_names_dict); @@ -2794,7 +2777,7 @@ get_fields_transfer_function(int aligned, &fields[i].data, out_needs_api) != NPY_SUCCEED) { for (i = i-1; i >= 0; --i) { - PyArray_FreeStridedTransferData(fields[i].data); + NPY_AUXDATA_FREE(fields[i].data); } PyArray_free(data); Py_XDECREF(used_names_dict); @@ -2816,7 +2799,7 @@ get_fields_transfer_function(int aligned, &fields[i].data, out_needs_api) != NPY_SUCCEED) { for (i = i-1; i >= 0; --i) { - PyArray_FreeStridedTransferData(fields[i].data); + NPY_AUXDATA_FREE(fields[i].data); } PyArray_free(data); Py_XDECREF(used_names_dict); @@ -2841,7 +2824,7 @@ get_fields_transfer_function(int aligned, if (!PyArg_ParseTuple(tup, "Oi|O", &src_fld_dtype, &src_offset, &title)) { for (i = field_count-1; i >= 0; --i) { - PyArray_FreeStridedTransferData(fields[i].data); + NPY_AUXDATA_FREE(fields[i].data); } PyArray_free(data); Py_XDECREF(used_names_dict); @@ -2855,7 +2838,7 @@ get_fields_transfer_function(int aligned, &fields[field_count].data, out_needs_api) != NPY_SUCCEED) { for (i = field_count-1; i >= 0; --i) { - PyArray_FreeStridedTransferData(fields[i].data); + NPY_AUXDATA_FREE(fields[i].data); } PyArray_free(data); return NPY_FAIL; @@ -2875,7 +2858,7 @@ get_fields_transfer_function(int aligned, data->field_count = field_count; *out_stransfer = &_strided_to_strided_field_transfer; - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; return NPY_SUCCEED; } @@ -2886,7 +2869,7 @@ get_decsrcref_fields_transfer_function(int aligned, npy_intp src_stride, PyArray_Descr *src_dtype, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api) { PyObject *names, *key, *tup, *title; @@ -2908,8 +2891,8 @@ get_decsrcref_fields_transfer_function(int aligned, PyErr_NoMemory(); return NPY_FAIL; } - data->freefunc = &_field_transfer_data_free; - data->copyfunc = &_field_transfer_data_copy; + data->base.free = &_field_transfer_data_free; + data->base.clone = &_field_transfer_data_clone; fields = &data->fields; field_count = 0; @@ -2932,7 +2915,7 @@ get_decsrcref_fields_transfer_function(int aligned, &fields[field_count].data, out_needs_api) != NPY_SUCCEED) { for (i = field_count-1; i >= 0; --i) { - PyArray_FreeStridedTransferData(fields[i].data); + NPY_AUXDATA_FREE(fields[i].data); } PyArray_free(data); return NPY_FAIL; @@ -2947,7 +2930,7 @@ get_decsrcref_fields_transfer_function(int aligned, data->field_count = field_count; *out_stransfer = &_strided_to_strided_field_transfer; - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; return NPY_SUCCEED; } @@ -2957,7 +2940,7 @@ get_setdestzero_fields_transfer_function(int aligned, npy_intp dst_stride, PyArray_Descr *dst_dtype, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api) { PyObject *names, *key, *tup, *title; @@ -2979,8 +2962,8 @@ get_setdestzero_fields_transfer_function(int aligned, PyErr_NoMemory(); return NPY_FAIL; } - data->freefunc = &_field_transfer_data_free; - data->copyfunc = &_field_transfer_data_copy; + data->base.free = &_field_transfer_data_free; + data->base.clone = &_field_transfer_data_clone; fields = &data->fields; for (i = 0; i < names_size; ++i) { @@ -2998,7 +2981,7 @@ get_setdestzero_fields_transfer_function(int aligned, &fields[i].data, out_needs_api) != NPY_SUCCEED) { for (i = i-1; i >= 0; --i) { - PyArray_FreeStridedTransferData(fields[i].data); + NPY_AUXDATA_FREE(fields[i].data); } PyArray_free(data); return NPY_FAIL; @@ -3011,7 +2994,7 @@ get_setdestzero_fields_transfer_function(int aligned, data->field_count = field_count; *out_stransfer = &_strided_to_strided_field_transfer; - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; return NPY_SUCCEED; } @@ -3023,7 +3006,7 @@ _null_to_strided_set_bool_one(char *dst, npy_intp dst_stride, char *NPY_UNUSED(src), npy_intp NPY_UNUSED(src_stride), npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *NPY_UNUSED(data)) + NpyAuxData *NPY_UNUSED(data)) { /* bool type is one byte, so can just use the char */ @@ -3040,7 +3023,7 @@ _null_to_contig_set_bool_one(char *dst, npy_intp NPY_UNUSED(dst_stride), char *NPY_UNUSED(src), npy_intp NPY_UNUSED(src_stride), npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *NPY_UNUSED(data)) + NpyAuxData *NPY_UNUSED(data)) { /* bool type is one byte, so can just use the char */ @@ -3051,7 +3034,7 @@ _null_to_contig_set_bool_one(char *dst, NPY_NO_EXPORT int get_bool_setdstone_transfer_function(npy_intp dst_stride, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *NPY_UNUSED(out_needs_api)) { if (dst_stride == 1) { @@ -3069,13 +3052,12 @@ get_bool_setdstone_transfer_function(npy_intp dst_stride, /* Sets dest to zero */ typedef struct { - free_strided_transfer_data freefunc; - copy_strided_transfer_data copyfunc; + NpyAuxData base; npy_intp dst_itemsize; } _dst_memset_zero_data; /* zero-padded data copy function */ -void *_dst_memset_zero_data_copy(void *data) +NpyAuxData *_dst_memset_zero_data_clone(NpyAuxData *data) { _dst_memset_zero_data *newdata = (_dst_memset_zero_data *)PyArray_malloc( @@ -3086,7 +3068,7 @@ void *_dst_memset_zero_data_copy(void *data) memcpy(newdata, data, sizeof(_dst_memset_zero_data)); - return newdata; + return (NpyAuxData *)newdata; } static void @@ -3094,7 +3076,7 @@ _null_to_strided_memset_zero(char *dst, npy_intp dst_stride, char *NPY_UNUSED(src), npy_intp NPY_UNUSED(src_stride), npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *data) + NpyAuxData *data) { _dst_memset_zero_data *d = (_dst_memset_zero_data *)data; npy_intp dst_itemsize = d->dst_itemsize; @@ -3111,7 +3093,7 @@ _null_to_contig_memset_zero(char *dst, npy_intp dst_stride, char *NPY_UNUSED(src), npy_intp NPY_UNUSED(src_stride), npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *data) + NpyAuxData *data) { _dst_memset_zero_data *d = (_dst_memset_zero_data *)data; npy_intp dst_itemsize = d->dst_itemsize; @@ -3124,7 +3106,7 @@ _null_to_strided_reference_setzero(char *dst, npy_intp dst_stride, char *NPY_UNUSED(src), npy_intp NPY_UNUSED(src_stride), npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *NPY_UNUSED(data)) + NpyAuxData *NPY_UNUSED(data)) { PyObject *dst_ref = NULL; @@ -3149,7 +3131,7 @@ get_setdstzero_transfer_function(int aligned, npy_intp dst_stride, PyArray_Descr *dst_dtype, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api) { _dst_memset_zero_data *data; @@ -3163,8 +3145,8 @@ get_setdstzero_transfer_function(int aligned, return NPY_FAIL; } - data->freefunc = &PyArray_free; - data->copyfunc = &_dst_memset_zero_data_copy; + data->base.free = (NpyAuxData_FreeFunc *)(&PyArray_free); + data->base.clone = &_dst_memset_zero_data_clone; data->dst_itemsize = dst_dtype->elsize; if (dst_stride == data->dst_itemsize) { @@ -3173,7 +3155,7 @@ get_setdstzero_transfer_function(int aligned, else { *out_stransfer = &_null_to_strided_memset_zero; } - *out_transferdata = data; + *out_transferdata = (NpyAuxData *)data; } /* If it's exactly one reference, use the decref function */ else if (dst_dtype->type_num == NPY_OBJECT) { @@ -3189,7 +3171,7 @@ get_setdstzero_transfer_function(int aligned, PyArray_Dims dst_shape = {NULL, -1}; npy_intp dst_size = 1; PyArray_StridedTransferFn *contig_stransfer; - void *contig_data; + NpyAuxData *contig_data; if (out_needs_api) { *out_needs_api = 1; @@ -3218,7 +3200,7 @@ get_setdstzero_transfer_function(int aligned, 0, dst_dtype->subarray->base->elsize, dst_size, out_stransfer, out_transferdata) != NPY_SUCCEED) { - PyArray_FreeStridedTransferData(contig_data); + NPY_AUXDATA_FREE(contig_data); return NPY_FAIL; } } @@ -3244,7 +3226,7 @@ _dec_src_ref_nop(char *NPY_UNUSED(dst), char *NPY_UNUSED(src), npy_intp NPY_UNUSED(src_stride), npy_intp NPY_UNUSED(N), npy_intp NPY_UNUSED(src_itemsize), - void *NPY_UNUSED(data)) + NpyAuxData *NPY_UNUSED(data)) { /* NOP */ } @@ -3255,7 +3237,7 @@ _strided_to_null_dec_src_ref_reference(char *NPY_UNUSED(dst), char *src, npy_intp src_stride, npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *NPY_UNUSED(data)) + NpyAuxData *NPY_UNUSED(data)) { PyObject *src_ref = NULL; while (N > 0) { @@ -3276,7 +3258,7 @@ get_decsrcref_transfer_function(int aligned, npy_intp src_stride, PyArray_Descr *src_dtype, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api) { /* If there are no references, it's a nop */ @@ -3302,7 +3284,7 @@ get_decsrcref_transfer_function(int aligned, PyArray_Dims src_shape = {NULL, -1}; npy_intp src_size = 1; PyArray_StridedTransferFn *stransfer; - void *data; + NpyAuxData *data; if (out_needs_api) { *out_needs_api = 1; @@ -3331,7 +3313,7 @@ get_decsrcref_transfer_function(int aligned, src_dtype->subarray->base->elsize, 0, src_size, out_stransfer, out_transferdata) != NPY_SUCCEED) { - PyArray_FreeStridedTransferData(data); + NPY_AUXDATA_FREE(data); return NPY_FAIL; } @@ -3358,7 +3340,7 @@ PyArray_GetDTypeCopySwapFn(int aligned, npy_intp src_stride, npy_intp dst_stride, PyArray_Descr *dtype, PyArray_StridedTransferFn **outstransfer, - void **outtransferdata) + NpyAuxData **outtransferdata) { npy_intp itemsize = dtype->elsize; @@ -3404,7 +3386,7 @@ PyArray_GetDTypeTransferFunction(int aligned, PyArray_Descr *src_dtype, PyArray_Descr *dst_dtype, int move_references, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api) { npy_intp src_itemsize, dst_itemsize; @@ -3628,7 +3610,7 @@ PyArray_CastRawArrays(npy_intp count, int move_references) { PyArray_StridedTransferFn *stransfer = NULL; - void *transferdata = NULL; + NpyAuxData *transferdata = NULL; int aligned = 1, needs_api = 0; /* Make sure the copy is reasonable */ @@ -3662,7 +3644,7 @@ PyArray_CastRawArrays(npy_intp count, src_dtype->elsize, transferdata); /* Cleanup */ - PyArray_FreeStridedTransferData(transferdata); + NPY_AUXDATA_FREE(transferdata); /* If needs_api was set to 1, it may have raised a Python exception */ return (needs_api && PyErr_Occurred()) ? NPY_FAIL : NPY_SUCCEED; diff --git a/numpy/core/src/multiarray/lowlevel_strided_loops.c.src b/numpy/core/src/multiarray/lowlevel_strided_loops.c.src index 4c6f3dada..ab1918e0e 100644 --- a/numpy/core/src/multiarray/lowlevel_strided_loops.c.src +++ b/numpy/core/src/multiarray/lowlevel_strided_loops.c.src @@ -126,7 +126,7 @@ static void @prefix@_@oper@_size@elsize@(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *NPY_UNUSED(data)) + NpyAuxData *NPY_UNUSED(data)) { /*printf("fn @prefix@_@oper@_size@elsize@\n");*/ while (N > 0) { @@ -186,7 +186,7 @@ static void npy_intp dst_stride, char *src, npy_intp NPY_UNUSED(src_stride), npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *NPY_UNUSED(data)) + NpyAuxData *NPY_UNUSED(data)) { #if @elsize@ != 16 @type@ temp = @swap@@elsize@(*((@type@ *)src)); @@ -231,7 +231,7 @@ static void _strided_to_strided(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *NPY_UNUSED(data)) + NpyAuxData *NPY_UNUSED(data)) { while (N > 0) { memcpy(dst, src, src_itemsize); @@ -245,7 +245,7 @@ static void _swap_strided_to_strided(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *NPY_UNUSED(data)) + NpyAuxData *NPY_UNUSED(data)) { char *a, *b, c; @@ -270,7 +270,7 @@ static void _swap_pair_strided_to_strided(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *NPY_UNUSED(data)) + NpyAuxData *NPY_UNUSED(data)) { char *a, *b, c; npy_intp itemsize_half = src_itemsize / 2; @@ -305,7 +305,7 @@ static void _contig_to_contig(char *dst, npy_intp NPY_UNUSED(dst_stride), char *src, npy_intp NPY_UNUSED(src_stride), npy_intp N, npy_intp src_itemsize, - void *NPY_UNUSED(data)) + NpyAuxData *NPY_UNUSED(data)) { memcpy(dst, src, src_itemsize*N); } @@ -763,7 +763,7 @@ static void char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp NPY_UNUSED(src_itemsize), - void *NPY_UNUSED(data)) + NpyAuxData *NPY_UNUSED(data)) { #if @is_complex1@ _TYPE1 src_value[2]; @@ -919,32 +919,6 @@ PyArray_GetStridedNumericCastFn(int aligned, npy_intp src_stride, } -/************** STRIDED TRANSFER FUNCTION MEMORY MANAGEMENT **************/ - -typedef void (*_npy_stridedtransfer_dealloc)(void *); -NPY_NO_EXPORT void -PyArray_FreeStridedTransferData(void *transferdata) -{ - if (transferdata != NULL) { - _npy_stridedtransfer_dealloc dealloc = - *((_npy_stridedtransfer_dealloc *)transferdata); - dealloc(transferdata); - } -} - -typedef void *(*_npy_stridedtransfer_copy)(void *); -NPY_NO_EXPORT void * -PyArray_CopyStridedTransferData(void *transferdata) -{ - if (transferdata != NULL) { - _npy_stridedtransfer_copy copy = - *((_npy_stridedtransfer_copy *)transferdata + 1); - return copy(transferdata); - } - - return NULL; -} - /****************** PRIMITIVE FLAT TO/FROM NDIM FUNCTIONS ******************/ NPY_NO_EXPORT npy_intp @@ -955,7 +929,7 @@ PyArray_TransferNDimToStrided(npy_intp ndim, npy_intp *shape, npy_intp shape_inc, npy_intp count, npy_intp src_itemsize, PyArray_StridedTransferFn *stransfer, - void *data) + NpyAuxData *data) { npy_intp i, M, N, coord0, shape0, src_stride0, coord1, shape1, src_stride1; @@ -1073,7 +1047,7 @@ PyArray_TransferStridedToNDim(npy_intp ndim, npy_intp *shape, npy_intp shape_inc, npy_intp count, npy_intp src_itemsize, PyArray_StridedTransferFn *stransfer, - void *data) + NpyAuxData *data) { npy_intp i, M, N, coord0, shape0, dst_stride0, coord1, shape1, dst_stride1; diff --git a/numpy/core/src/multiarray/nditer.c.src b/numpy/core/src/multiarray/nditer.c.src index fd106ec73..d8758bdfa 100644 --- a/numpy/core/src/multiarray/nditer.c.src +++ b/numpy/core/src/multiarray/nditer.c.src @@ -232,11 +232,11 @@ struct NpyIter_BD { (&(bufferdata)->bd_flexdata + 3*(nop))) #define NBF_READTRANSFERFN(bufferdata) ((PyArray_StridedTransferFn **) \ (&(bufferdata)->bd_flexdata + 4*(nop))) -#define NBF_READTRANSFERDATA(bufferdata) ((void **) \ +#define NBF_READTRANSFERDATA(bufferdata) ((NpyAuxData **) \ (&(bufferdata)->bd_flexdata + 5*(nop))) #define NBF_WRITETRANSFERFN(bufferdata) ((PyArray_StridedTransferFn **) \ (&(bufferdata)->bd_flexdata + 6*(nop))) -#define NBF_WRITETRANSFERDATA(bufferdata) ((void **) \ +#define NBF_WRITETRANSFERDATA(bufferdata) ((NpyAuxData **) \ (&(bufferdata)->bd_flexdata + 7*(nop))) #define NBF_BUFFERS(bufferdata) ((char **) \ (&(bufferdata)->bd_flexdata + 8*(nop))) @@ -841,7 +841,7 @@ NpyIter_Copy(NpyIter *iter) NpyIter_BufferData *bufferdata; npy_intp buffersize, itemsize; char **buffers; - void **readtransferdata, **writetransferdata; + NpyAuxData **readtransferdata, **writetransferdata; bufferdata = NIT_BUFFERDATA(newiter); buffers = NBF_BUFFERS(bufferdata); @@ -869,7 +869,7 @@ NpyIter_Copy(NpyIter *iter) } else { readtransferdata[iop] = - PyArray_CopyStridedTransferData(readtransferdata[iop]); + NPY_AUXDATA_CLONE(readtransferdata[iop]); if (readtransferdata[iop] == NULL) { out_of_memory = 1; } @@ -882,7 +882,7 @@ NpyIter_Copy(NpyIter *iter) } else { writetransferdata[iop] = - PyArray_CopyStridedTransferData(writetransferdata[iop]); + NPY_AUXDATA_CLONE(writetransferdata[iop]); if (writetransferdata[iop] == NULL) { out_of_memory = 1; } @@ -925,7 +925,7 @@ NpyIter_Deallocate(NpyIter *iter) if (itflags&NPY_ITFLAG_BUFFER) { NpyIter_BufferData *bufferdata = NIT_BUFFERDATA(iter); char **buffers; - void **transferdata; + NpyAuxData **transferdata; /* buffers */ buffers = NBF_BUFFERS(bufferdata); @@ -938,14 +938,14 @@ NpyIter_Deallocate(NpyIter *iter) transferdata = NBF_READTRANSFERDATA(bufferdata); for(iop = 0; iop < nop; ++iop, ++transferdata) { if (*transferdata) { - PyArray_FreeStridedTransferData(*transferdata); + NPY_AUXDATA_FREE(*transferdata); } } /* write bufferdata */ transferdata = NBF_WRITETRANSFERDATA(bufferdata); for(iop = 0; iop < nop; ++iop, ++transferdata) { if (*transferdata) { - PyArray_FreeStridedTransferData(*transferdata); + NPY_AUXDATA_FREE(*transferdata); } } } @@ -5057,11 +5057,11 @@ npyiter_allocate_transfer_functions(NpyIter *iter) npy_intp *strides = NAD_STRIDES(axisdata), op_stride; PyArray_StridedTransferFn **readtransferfn = NBF_READTRANSFERFN(bufferdata), **writetransferfn = NBF_WRITETRANSFERFN(bufferdata); - void **readtransferdata = NBF_READTRANSFERDATA(bufferdata), - **writetransferdata = NBF_WRITETRANSFERDATA(bufferdata); + NpyAuxData **readtransferdata = NBF_READTRANSFERDATA(bufferdata), + **writetransferdata = NBF_WRITETRANSFERDATA(bufferdata); PyArray_StridedTransferFn *stransfer = NULL; - void *transferdata = NULL; + NpyAuxData *transferdata = NULL; int needs_api = 0; for (iop = 0; iop < nop; ++iop) { @@ -5155,11 +5155,11 @@ npyiter_allocate_transfer_functions(NpyIter *iter) fail: for (i = 0; i < iop; ++i) { if (readtransferdata[iop] != NULL) { - PyArray_FreeStridedTransferData(readtransferdata[iop]); + NPY_AUXDATA_FREE(readtransferdata[iop]); readtransferdata[iop] = NULL; } if (writetransferdata[iop] != NULL) { - PyArray_FreeStridedTransferData(writetransferdata[iop]); + NPY_AUXDATA_FREE(writetransferdata[iop]); writetransferdata[iop] = NULL; } } @@ -5336,7 +5336,7 @@ npyiter_copy_from_buffers(NpyIter *iter) npy_intp *reduce_outerstrides = NULL; PyArray_StridedTransferFn *stransfer = NULL; - void *transferdata = NULL; + NpyAuxData *transferdata = NULL; npy_intp axisdata_incr = NIT_AXISDATA_SIZEOF(itflags, ndim, nop) / NPY_SIZEOF_INTP; @@ -5503,7 +5503,7 @@ npyiter_copy_to_buffers(NpyIter *iter, char **prev_dataptrs) char **reduce_outerptrs = NULL; PyArray_StridedTransferFn *stransfer = NULL; - void *transferdata = NULL; + NpyAuxData *transferdata = NULL; /* * Have to get this flag before npyiter_checkreducesize sets diff --git a/numpy/core/src/private/lowlevel_strided_loops.h b/numpy/core/src/private/lowlevel_strided_loops.h index fddeb7c21..a1f183e50 100644 --- a/numpy/core/src/private/lowlevel_strided_loops.h +++ b/numpy/core/src/private/lowlevel_strided_loops.h @@ -16,31 +16,15 @@ * Examples of transfer functions are a straight copy, a byte-swap, * and a casting operation, * - * The 'transferdata' parameter is slightly special, and must always contain - * pointer to deallocation and copying routines at its beginning. The function - * PyArray_FreeStridedTransferData should be used to deallocate such - * pointers, and calls the first function pointer, while the function - * PyArray_CopyStridedTransferData should be used to copy it. + * The 'transferdata' parameter is slightly special, following a + * generic auxiliary data pattern defined in ndarraytypes.h + * Use NPY_AUXDATA_CLONE and NPY_AUXDATA_FREE to deal with this data. * */ typedef void (PyArray_StridedTransferFn)(char *dst, npy_intp dst_stride, char *src, npy_intp src_stride, npy_intp N, npy_intp src_itemsize, - void *transferdata); - -/* - * Deallocates a PyArray_StridedTransferFunction data object. See - * the comment with the function typedef for more details. - */ -NPY_NO_EXPORT void -PyArray_FreeStridedTransferData(void *transferdata); - -/* - * Copies a PyArray_StridedTransferFunction data object. See - * the comment with the function typedef for more details. - */ -NPY_NO_EXPORT void * -PyArray_CopyStridedTransferData(void *transferdata); + NpyAuxData *transferdata); /* * Gives back a function pointer to a specialized function for copying @@ -109,7 +93,7 @@ PyArray_GetStridedZeroPadCopyFn(int aligned, npy_intp src_stride, npy_intp dst_stride, npy_intp src_itemsize, npy_intp dst_itemsize, PyArray_StridedTransferFn **outstransfer, - void **outtransferdata); + NpyAuxData **outtransferdata); /* * For casts between built-in numeric types, @@ -133,7 +117,7 @@ PyArray_GetDTypeCopySwapFn(int aligned, npy_intp src_stride, npy_intp dst_stride, PyArray_Descr *dtype, PyArray_StridedTransferFn **outstransfer, - void **outtransferdata); + NpyAuxData **outtransferdata); /* * If it's possible, gives back a transfer function which casts and/or @@ -186,7 +170,7 @@ PyArray_GetDTypeTransferFunction(int aligned, PyArray_Descr *src_dtype, PyArray_Descr *dst_dtype, int move_references, PyArray_StridedTransferFn **out_stransfer, - void **out_transferdata, + NpyAuxData **out_transferdata, int *out_needs_api); /* @@ -253,7 +237,7 @@ PyArray_TransferNDimToStrided(npy_intp ndim, npy_intp *shape, npy_intp shape_inc, npy_intp count, npy_intp src_itemsize, PyArray_StridedTransferFn *stransfer, - void *transferdata); + NpyAuxData *transferdata); NPY_NO_EXPORT npy_intp PyArray_TransferStridedToNDim(npy_intp ndim, @@ -263,7 +247,7 @@ PyArray_TransferStridedToNDim(npy_intp ndim, npy_intp *shape, npy_intp shape_inc, npy_intp count, npy_intp src_itemsize, PyArray_StridedTransferFn *stransfer, - void *transferdata); + NpyAuxData *transferdata); /* * TRIVIAL ITERATION diff --git a/numpy/core/src/umath/loops.c.src b/numpy/core/src/umath/loops.c.src index 8f86391ff..e85b24c42 100644 --- a/numpy/core/src/umath/loops.c.src +++ b/numpy/core/src/umath/loops.c.src @@ -33,37 +33,37 @@ #define OUTPUT_LOOP\ char *op1 = args[1];\ - intp os1 = steps[1];\ - intp n = dimensions[0];\ - intp i;\ + npy_intp os1 = steps[1];\ + npy_intp n = dimensions[0];\ + npy_intp i;\ for(i = 0; i < n; i++, op1 += os1) #define UNARY_LOOP\ char *ip1 = args[0], *op1 = args[1];\ - intp is1 = steps[0], os1 = steps[1];\ - intp n = dimensions[0];\ - intp i;\ + npy_intp is1 = steps[0], os1 = steps[1];\ + npy_intp n = dimensions[0];\ + npy_intp i;\ for(i = 0; i < n; i++, ip1 += is1, op1 += os1) #define UNARY_LOOP_TWO_OUT\ char *ip1 = args[0], *op1 = args[1], *op2 = args[2];\ - intp is1 = steps[0], os1 = steps[1], os2 = steps[2];\ - intp n = dimensions[0];\ - intp i;\ + npy_intp is1 = steps[0], os1 = steps[1], os2 = steps[2];\ + npy_intp n = dimensions[0];\ + npy_intp i;\ for(i = 0; i < n; i++, ip1 += is1, op1 += os1, op2 += os2) #define BINARY_LOOP\ char *ip1 = args[0], *ip2 = args[1], *op1 = args[2];\ - intp is1 = steps[0], is2 = steps[1], os1 = steps[2];\ - intp n = dimensions[0];\ - intp i;\ + npy_intp is1 = steps[0], is2 = steps[1], os1 = steps[2];\ + npy_intp n = dimensions[0];\ + npy_intp i;\ for(i = 0; i < n; i++, ip1 += is1, ip2 += is2, op1 += os1) #define BINARY_REDUCE_LOOP_INNER\ char *ip2 = args[1]; \ - intp is2 = steps[1]; \ - intp n = dimensions[0]; \ - intp i; \ + npy_intp is2 = steps[1]; \ + npy_intp n = dimensions[0]; \ + npy_intp i; \ for(i = 0; i < n; i++, ip2 += is2) #define BINARY_REDUCE_LOOP(TYPE)\ @@ -73,9 +73,9 @@ #define BINARY_LOOP_TWO_OUT\ char *ip1 = args[0], *ip2 = args[1], *op1 = args[2], *op2 = args[3];\ - intp is1 = steps[0], is2 = steps[1], os1 = steps[2], os2 = steps[3];\ - intp n = dimensions[0];\ - intp i;\ + npy_intp is1 = steps[0], is2 = steps[1], os1 = steps[2], os2 = steps[3];\ + npy_intp n = dimensions[0];\ + npy_intp i;\ for(i = 0; i < n; i++, ip1 += is1, ip2 += is2, op1 += os1, op2 += os2) /****************************************************************************** @@ -95,7 +95,7 @@ typedef longdouble longdoubleBinaryFunc(longdouble x, longdouble y); /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_e_e(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_e_e(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { halfUnaryFunc *f = (halfUnaryFunc *)func; UNARY_LOOP { @@ -106,7 +106,7 @@ PyUFunc_e_e(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_e_e_As_f_f(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_e_e_As_f_f(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { floatUnaryFunc *f = (floatUnaryFunc *)func; UNARY_LOOP { @@ -117,7 +117,7 @@ PyUFunc_e_e_As_f_f(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_e_e_As_d_d(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_e_e_As_d_d(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { doubleUnaryFunc *f = (doubleUnaryFunc *)func; UNARY_LOOP { @@ -128,7 +128,7 @@ PyUFunc_e_e_As_d_d(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_f_f(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_f_f(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { floatUnaryFunc *f = (floatUnaryFunc *)func; UNARY_LOOP { @@ -139,7 +139,7 @@ PyUFunc_f_f(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_f_f_As_d_d(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_f_f_As_d_d(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { doubleUnaryFunc *f = (doubleUnaryFunc *)func; UNARY_LOOP { @@ -150,7 +150,7 @@ PyUFunc_f_f_As_d_d(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_ee_e(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_ee_e(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { halfBinaryFunc *f = (halfBinaryFunc *)func; BINARY_LOOP { @@ -162,7 +162,7 @@ PyUFunc_ee_e(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_ee_e_As_ff_f(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_ee_e_As_ff_f(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { floatBinaryFunc *f = (floatBinaryFunc *)func; BINARY_LOOP { @@ -174,7 +174,7 @@ PyUFunc_ee_e_As_ff_f(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_ee_e_As_dd_d(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_ee_e_As_dd_d(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { doubleBinaryFunc *f = (doubleBinaryFunc *)func; BINARY_LOOP { @@ -186,7 +186,7 @@ PyUFunc_ee_e_As_dd_d(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_ff_f(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_ff_f(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { floatBinaryFunc *f = (floatBinaryFunc *)func; BINARY_LOOP { @@ -198,7 +198,7 @@ PyUFunc_ff_f(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_ff_f_As_dd_d(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_ff_f_As_dd_d(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { doubleBinaryFunc *f = (doubleBinaryFunc *)func; BINARY_LOOP { @@ -210,7 +210,7 @@ PyUFunc_ff_f_As_dd_d(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_d_d(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_d_d(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { doubleUnaryFunc *f = (doubleUnaryFunc *)func; UNARY_LOOP { @@ -221,7 +221,7 @@ PyUFunc_d_d(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_dd_d(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_dd_d(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { doubleBinaryFunc *f = (doubleBinaryFunc *)func; BINARY_LOOP { @@ -233,7 +233,7 @@ PyUFunc_dd_d(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_g_g(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_g_g(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { longdoubleUnaryFunc *f = (longdoubleUnaryFunc *)func; UNARY_LOOP { @@ -244,7 +244,7 @@ PyUFunc_g_g(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_gg_g(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_gg_g(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { longdoubleBinaryFunc *f = (longdoubleBinaryFunc *)func; BINARY_LOOP { @@ -271,7 +271,7 @@ typedef void clongdoubleBinaryFunc(clongdouble *x, clongdouble *y, /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_F_F(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_F_F(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { cfloatUnaryFunc *f = (cfloatUnaryFunc *)func; UNARY_LOOP { @@ -283,7 +283,7 @@ PyUFunc_F_F(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_F_F_As_D_D(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_F_F_As_D_D(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { cdoubleUnaryFunc *f = (cdoubleUnaryFunc *)func; UNARY_LOOP { @@ -298,7 +298,7 @@ PyUFunc_F_F_As_D_D(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_FF_F(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_FF_F(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { cfloatBinaryFunc *f = (cfloatBinaryFunc *)func; BINARY_LOOP { @@ -311,7 +311,7 @@ PyUFunc_FF_F(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_FF_F_As_DD_D(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_FF_F_As_DD_D(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { cdoubleBinaryFunc *f = (cdoubleBinaryFunc *)func; BINARY_LOOP { @@ -328,7 +328,7 @@ PyUFunc_FF_F_As_DD_D(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_D_D(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_D_D(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { cdoubleUnaryFunc *f = (cdoubleUnaryFunc *)func; UNARY_LOOP { @@ -340,7 +340,7 @@ PyUFunc_D_D(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_DD_D(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_DD_D(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { cdoubleBinaryFunc *f = (cdoubleBinaryFunc *)func; BINARY_LOOP { @@ -353,7 +353,7 @@ PyUFunc_DD_D(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_G_G(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_G_G(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { clongdoubleUnaryFunc *f = (clongdoubleUnaryFunc *)func; UNARY_LOOP { @@ -365,7 +365,7 @@ PyUFunc_G_G(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_GG_G(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_GG_G(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { clongdoubleBinaryFunc *f = (clongdoubleBinaryFunc *)func; BINARY_LOOP { @@ -383,7 +383,7 @@ PyUFunc_GG_G(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_O_O(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_O_O(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { unaryfunc f = (unaryfunc)func; UNARY_LOOP { @@ -400,7 +400,7 @@ PyUFunc_O_O(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_O_O_method(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_O_O_method(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { char *meth = (char *)func; UNARY_LOOP { @@ -417,7 +417,7 @@ PyUFunc_O_O_method(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_OO_O(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_OO_O(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { binaryfunc f = (binaryfunc)func; BINARY_LOOP { @@ -435,7 +435,7 @@ PyUFunc_OO_O(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_OO_O_method(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_OO_O_method(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { char *meth = (char *)func; BINARY_LOOP { @@ -459,9 +459,9 @@ PyUFunc_OO_O_method(char **args, intp *dimensions, intp *steps, void *func) /*UFUNC_API*/ NPY_NO_EXPORT void -PyUFunc_On_Om(char **args, intp *dimensions, intp *steps, void *func) +PyUFunc_On_Om(char **args, npy_intp *dimensions, npy_intp *steps, void *func) { - intp n = dimensions[0]; + npy_intp n = dimensions[0]; PyUFunc_PyFuncData *data = (PyUFunc_PyFuncData *)func; int nin = data->nin; int nout = data->nout; @@ -469,7 +469,7 @@ PyUFunc_On_Om(char **args, intp *dimensions, intp *steps, void *func) char *ptrs[NPY_MAXARGS]; PyObject *arglist, *result; PyObject *in, **op; - intp i, j, ntot; + npy_intp i, j, ntot; ntot = nin+nout; @@ -530,7 +530,7 @@ PyUFunc_On_Om(char **args, intp *dimensions, intp *steps, void *func) **/ NPY_NO_EXPORT void -BOOL_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +BOOL_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { Bool in1 = *((Bool *)ip1) != 0; @@ -548,7 +548,7 @@ BOOL_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) **/ NPY_NO_EXPORT void -BOOL_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +BOOL_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { if(IS_BINARY_REDUCE) { BINARY_REDUCE_LOOP(Bool) { @@ -572,7 +572,7 @@ BOOL_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) NPY_NO_EXPORT void -BOOL_logical_xor(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +BOOL_logical_xor(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { Bool in1 = *((Bool *)ip1) != 0; @@ -586,7 +586,7 @@ BOOL_logical_xor(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(fu * #OP = >, <# **/ NPY_NO_EXPORT void -BOOL_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +BOOL_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { Bool in1 = *((Bool *)ip1) != 0; @@ -601,7 +601,7 @@ BOOL_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) * #OP = !=, ==# **/ NPY_NO_EXPORT void -BOOL_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +BOOL_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { Bool in1 = *(Bool *)ip1; @@ -611,7 +611,7 @@ BOOL_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) /**end repeat**/ NPY_NO_EXPORT void -BOOL_ones_like(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +BOOL_ones_like(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { OUTPUT_LOOP { *((Bool *)op1) = 1; @@ -642,7 +642,7 @@ BOOL_ones_like(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data #define @S@@TYPE@_fmin @S@@TYPE@_minimum NPY_NO_EXPORT void -@S@@TYPE@_ones_like(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +@S@@TYPE@_ones_like(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { OUTPUT_LOOP { *((@s@@type@ *)op1) = 1; @@ -650,7 +650,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@S@@TYPE@_square(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +@S@@TYPE@_square(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { UNARY_LOOP { const @s@@type@ in1 = *(@s@@type@ *)ip1; @@ -659,7 +659,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@S@@TYPE@_reciprocal(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +@S@@TYPE@_reciprocal(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { UNARY_LOOP { const @s@@type@ in1 = *(@s@@type@ *)ip1; @@ -668,7 +668,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@S@@TYPE@_conjugate(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@S@@TYPE@_conjugate(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @s@@type@ in1 = *(@s@@type@ *)ip1; @@ -677,7 +677,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@S@@TYPE@_negative(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@S@@TYPE@_negative(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @s@@type@ in1 = *(@s@@type@ *)ip1; @@ -686,7 +686,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@S@@TYPE@_logical_not(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@S@@TYPE@_logical_not(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @s@@type@ in1 = *(@s@@type@ *)ip1; @@ -695,7 +695,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@S@@TYPE@_invert(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@S@@TYPE@_invert(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @s@@type@ in1 = *(@s@@type@ *)ip1; @@ -710,7 +710,7 @@ NPY_NO_EXPORT void * #OP = +, -,*, &, |, ^, <<, >># */ NPY_NO_EXPORT void -@S@@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@S@@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { if(IS_BINARY_REDUCE) { BINARY_REDUCE_LOOP(@s@@type@) { @@ -734,7 +734,7 @@ NPY_NO_EXPORT void * #OP = ==, !=, >, >=, <, <=, &&, ||# */ NPY_NO_EXPORT void -@S@@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@S@@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @s@@type@ in1 = *(@s@@type@ *)ip1; @@ -745,7 +745,7 @@ NPY_NO_EXPORT void /**end repeat2**/ NPY_NO_EXPORT void -@S@@TYPE@_logical_xor(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@S@@TYPE@_logical_xor(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @s@@type@ in1 = *(@s@@type@ *)ip1; @@ -759,7 +759,7 @@ NPY_NO_EXPORT void * #OP = >, <# **/ NPY_NO_EXPORT void -@S@@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@S@@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { if (IS_BINARY_REDUCE) { BINARY_REDUCE_LOOP(@s@@type@) { @@ -779,7 +779,7 @@ NPY_NO_EXPORT void /**end repeat2**/ NPY_NO_EXPORT void -@S@@TYPE@_true_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@S@@TYPE@_true_divide(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const double in1 = (double)(*(@s@@type@ *)ip1); @@ -789,7 +789,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@S@@TYPE@_power(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@S@@TYPE@_power(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @ftype@ in1 = (@ftype@)*(@s@@type@ *)ip1; @@ -799,7 +799,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@S@@TYPE@_fmod(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@S@@TYPE@_fmod(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @s@@type@ in1 = *(@s@@type@ *)ip1; @@ -818,7 +818,7 @@ NPY_NO_EXPORT void /**end repeat1**/ NPY_NO_EXPORT void -U@TYPE@_absolute(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +U@TYPE@_absolute(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const u@type@ in1 = *(u@type@ *)ip1; @@ -827,7 +827,7 @@ U@TYPE@_absolute(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(fu } NPY_NO_EXPORT void -@TYPE@_absolute(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_absolute(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -836,7 +836,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -U@TYPE@_sign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +U@TYPE@_sign(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const u@type@ in1 = *(u@type@ *)ip1; @@ -845,7 +845,7 @@ U@TYPE@_sign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) } NPY_NO_EXPORT void -@TYPE@_sign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_sign(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -854,7 +854,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_divide(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -880,7 +880,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -U@TYPE@_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +U@TYPE@_divide(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const u@type@ in1 = *(u@type@ *)ip1; @@ -896,7 +896,7 @@ U@TYPE@_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func } NPY_NO_EXPORT void -@TYPE@_remainder(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_remainder(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -919,7 +919,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -U@TYPE@_remainder(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +U@TYPE@_remainder(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const u@type@ in1 = *(u@type@ *)ip1; @@ -943,7 +943,7 @@ U@TYPE@_remainder(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(f */ NPY_NO_EXPORT void -TIMEDELTA_negative(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +TIMEDELTA_negative(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const npy_timedelta in1 = *(npy_timedelta *)ip1; @@ -957,7 +957,7 @@ TIMEDELTA_negative(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED( } NPY_NO_EXPORT void -TIMEDELTA_absolute(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +TIMEDELTA_absolute(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const npy_timedelta in1 = *(npy_timedelta *)ip1; @@ -971,7 +971,7 @@ TIMEDELTA_absolute(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED( } NPY_NO_EXPORT void -TIMEDELTA_sign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +TIMEDELTA_sign(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const npy_timedelta in1 = *(npy_timedelta *)ip1; @@ -985,7 +985,7 @@ TIMEDELTA_sign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func */ NPY_NO_EXPORT void -@TYPE@_ones_like(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +@TYPE@_ones_like(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { OUTPUT_LOOP { *((@type@ *)op1) = 1; @@ -997,7 +997,7 @@ NPY_NO_EXPORT void * #OP = ==, !=, >, >=, <, <=# */ NPY_NO_EXPORT void -@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1012,7 +1012,7 @@ NPY_NO_EXPORT void * #OP = >, <# **/ NPY_NO_EXPORT void -@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { if (IS_BINARY_REDUCE) { BINARY_REDUCE_LOOP(@type@) { @@ -1042,7 +1042,7 @@ NPY_NO_EXPORT void /**end repeat**/ NPY_NO_EXPORT void -DATETIME_Mm_M_add(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +DATETIME_Mm_M_add(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { BINARY_LOOP { const datetime in1 = *(datetime *)ip1; @@ -1057,7 +1057,7 @@ DATETIME_Mm_M_add(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(d } NPY_NO_EXPORT void -DATETIME_mM_M_add(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +DATETIME_mM_M_add(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const timedelta in1 = *(timedelta *)ip1; @@ -1072,7 +1072,7 @@ DATETIME_mM_M_add(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(f } NPY_NO_EXPORT void -TIMEDELTA_mm_m_add(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +TIMEDELTA_mm_m_add(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const timedelta in1 = *(timedelta *)ip1; @@ -1087,7 +1087,7 @@ TIMEDELTA_mm_m_add(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED( } NPY_NO_EXPORT void -DATETIME_Mm_M_subtract(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +DATETIME_Mm_M_subtract(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const datetime in1 = *(datetime *)ip1; @@ -1102,7 +1102,7 @@ DATETIME_Mm_M_subtract(char **args, intp *dimensions, intp *steps, void *NPY_UNU } NPY_NO_EXPORT void -DATETIME_MM_m_subtract(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +DATETIME_MM_m_subtract(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const datetime in1 = *(datetime *)ip1; @@ -1117,7 +1117,7 @@ DATETIME_MM_m_subtract(char **args, intp *dimensions, intp *steps, void *NPY_UNU } NPY_NO_EXPORT void -TIMEDELTA_mm_m_subtract(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +TIMEDELTA_mm_m_subtract(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const timedelta in1 = *(timedelta *)ip1; @@ -1133,7 +1133,7 @@ TIMEDELTA_mm_m_subtract(char **args, intp *dimensions, intp *steps, void *NPY_UN /* Note: Assuming 'q' == NPY_LONGLONG */ NPY_NO_EXPORT void -TIMEDELTA_mq_m_multiply(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +TIMEDELTA_mq_m_multiply(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const npy_timedelta in1 = *(npy_timedelta *)ip1; @@ -1149,7 +1149,7 @@ TIMEDELTA_mq_m_multiply(char **args, intp *dimensions, intp *steps, void *NPY_UN /* Note: Assuming 'q' == NPY_LONGLONG */ NPY_NO_EXPORT void -TIMEDELTA_qm_m_multiply(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +TIMEDELTA_qm_m_multiply(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const npy_int64 in1 = *(npy_int64 *)ip1; @@ -1164,7 +1164,7 @@ TIMEDELTA_qm_m_multiply(char **args, intp *dimensions, intp *steps, void *NPY_UN } NPY_NO_EXPORT void -TIMEDELTA_md_m_multiply(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +TIMEDELTA_md_m_multiply(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const npy_timedelta in1 = *(npy_timedelta *)ip1; @@ -1179,7 +1179,7 @@ TIMEDELTA_md_m_multiply(char **args, intp *dimensions, intp *steps, void *NPY_UN } NPY_NO_EXPORT void -TIMEDELTA_dm_m_multiply(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +TIMEDELTA_dm_m_multiply(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const double in1 = *(double *)ip1; @@ -1195,7 +1195,7 @@ TIMEDELTA_dm_m_multiply(char **args, intp *dimensions, intp *steps, void *NPY_UN /* Note: Assuming 'q' == NPY_LONGLONG */ NPY_NO_EXPORT void -TIMEDELTA_mq_m_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +TIMEDELTA_mq_m_divide(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const npy_timedelta in1 = *(npy_timedelta *)ip1; @@ -1210,7 +1210,7 @@ TIMEDELTA_mq_m_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUS } NPY_NO_EXPORT void -TIMEDELTA_md_m_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +TIMEDELTA_md_m_divide(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const npy_timedelta in1 = *(npy_timedelta *)ip1; @@ -1231,7 +1231,7 @@ TIMEDELTA_md_m_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUS } NPY_NO_EXPORT void -TIMEDELTA_mm_d_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +TIMEDELTA_mm_d_divide(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const npy_timedelta in1 = *(npy_timedelta *)ip1; @@ -1267,7 +1267,7 @@ TIMEDELTA_mm_d_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUS * # OP = +, -, *, /# */ NPY_NO_EXPORT void -@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { if(IS_BINARY_REDUCE) { BINARY_REDUCE_LOOP(@type@) { @@ -1291,7 +1291,7 @@ NPY_NO_EXPORT void * #OP = ==, !=, <, <=, >, >=, &&, ||# */ NPY_NO_EXPORT void -@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1302,7 +1302,7 @@ NPY_NO_EXPORT void /**end repeat1**/ NPY_NO_EXPORT void -@TYPE@_logical_xor(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_logical_xor(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1312,7 +1312,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_logical_not(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_logical_not(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1325,7 +1325,7 @@ NPY_NO_EXPORT void * #func = npy_isnan, npy_isinf, npy_isfinite, npy_signbit# **/ NPY_NO_EXPORT void -@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1335,7 +1335,7 @@ NPY_NO_EXPORT void /**end repeat1**/ NPY_NO_EXPORT void -@TYPE@_spacing(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_spacing(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1344,7 +1344,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_copysign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_copysign(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1354,7 +1354,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_nextafter(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_nextafter(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1368,7 +1368,7 @@ NPY_NO_EXPORT void * #OP = >=, <=# **/ NPY_NO_EXPORT void -@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { /* */ if (IS_BINARY_REDUCE) { @@ -1393,7 +1393,7 @@ NPY_NO_EXPORT void * #OP = >=, <=# **/ NPY_NO_EXPORT void -@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { /* */ if (IS_BINARY_REDUCE) { @@ -1414,7 +1414,7 @@ NPY_NO_EXPORT void /**end repeat1**/ NPY_NO_EXPORT void -@TYPE@_floor_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_floor_divide(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1424,7 +1424,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_remainder(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_remainder(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1440,7 +1440,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_square(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +@TYPE@_square(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { UNARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1449,7 +1449,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_reciprocal(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +@TYPE@_reciprocal(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { UNARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1458,7 +1458,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_ones_like(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +@TYPE@_ones_like(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { OUTPUT_LOOP { *((@type@ *)op1) = 1; @@ -1466,7 +1466,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_conjugate(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_conjugate(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1475,7 +1475,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_absolute(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_absolute(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1486,7 +1486,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_negative(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_negative(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1495,7 +1495,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_sign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_sign(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { /* Sign of nan is nan */ UNARY_LOOP { @@ -1505,7 +1505,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_modf(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_modf(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP_TWO_OUT { const @type@ in1 = *(@type@ *)ip1; @@ -1515,7 +1515,7 @@ NPY_NO_EXPORT void #ifdef HAVE_FREXP@C@ NPY_NO_EXPORT void -@TYPE@_frexp(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_frexp(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP_TWO_OUT { const @type@ in1 = *(@type@ *)ip1; @@ -1526,7 +1526,7 @@ NPY_NO_EXPORT void #ifdef HAVE_LDEXP@C@ NPY_NO_EXPORT void -@TYPE@_ldexp(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_ldexp(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1 = *(@type@ *)ip1; @@ -1536,7 +1536,7 @@ NPY_NO_EXPORT void } NPY_NO_EXPORT void -@TYPE@_ldexp_long(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +@TYPE@_ldexp_long(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { /* * Additional loop to handle long integer inputs (cf. #866, #1633). @@ -1583,7 +1583,7 @@ NPY_NO_EXPORT void * # OP = +, -, *, /# */ NPY_NO_EXPORT void -HALF_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { if(IS_BINARY_REDUCE) { char *iop1 = args[0]; @@ -1611,7 +1611,7 @@ HALF_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) * #OP = npy_half_eq, npy_half_ne, npy_half_lt, npy_half_le, npy_half_gt, npy_half_ge, _HALF_LOGICAL_AND, _HALF_LOGICAL_OR# */ NPY_NO_EXPORT void -HALF_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const npy_half in1 = *(npy_half *)ip1; @@ -1624,7 +1624,7 @@ HALF_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) #undef _HALF_LOGICAL_OR NPY_NO_EXPORT void -HALF_logical_xor(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_logical_xor(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const int in1 = !npy_half_iszero(*(npy_half *)ip1); @@ -1634,7 +1634,7 @@ HALF_logical_xor(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(fu } NPY_NO_EXPORT void -HALF_logical_not(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_logical_not(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const npy_half in1 = *(npy_half *)ip1; @@ -1647,7 +1647,7 @@ HALF_logical_not(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(fu * #func = npy_half_isnan, npy_half_isinf, npy_half_isfinite, npy_half_signbit# **/ NPY_NO_EXPORT void -HALF_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const npy_half in1 = *(npy_half *)ip1; @@ -1657,7 +1657,7 @@ HALF_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) /**end repeat**/ NPY_NO_EXPORT void -HALF_spacing(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_spacing(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const npy_half in1 = *(npy_half *)ip1; @@ -1666,7 +1666,7 @@ HALF_spacing(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) } NPY_NO_EXPORT void -HALF_copysign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_copysign(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const npy_half in1 = *(npy_half *)ip1; @@ -1676,7 +1676,7 @@ HALF_copysign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func) } NPY_NO_EXPORT void -HALF_nextafter(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_nextafter(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const npy_half in1 = *(npy_half *)ip1; @@ -1690,7 +1690,7 @@ HALF_nextafter(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func * #OP = npy_half_ge, npy_half_le# **/ NPY_NO_EXPORT void -HALF_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { /* */ BINARY_LOOP { @@ -1706,7 +1706,7 @@ HALF_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) * #OP = npy_half_ge, npy_half_le# **/ NPY_NO_EXPORT void -HALF_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { /* */ BINARY_LOOP { @@ -1718,7 +1718,7 @@ HALF_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) /**end repeat**/ NPY_NO_EXPORT void -HALF_floor_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_floor_divide(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const float in1 = npy_half_to_float(*(npy_half *)ip1); @@ -1728,7 +1728,7 @@ HALF_floor_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(f } NPY_NO_EXPORT void -HALF_remainder(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_remainder(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const float in1 = npy_half_to_float(*(npy_half *)ip1); @@ -1744,7 +1744,7 @@ HALF_remainder(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func } NPY_NO_EXPORT void -HALF_square(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +HALF_square(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { UNARY_LOOP { const float in1 = npy_half_to_float(*(npy_half *)ip1); @@ -1753,7 +1753,7 @@ HALF_square(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) } NPY_NO_EXPORT void -HALF_reciprocal(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +HALF_reciprocal(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { UNARY_LOOP { const float in1 = npy_half_to_float(*(npy_half *)ip1); @@ -1762,7 +1762,7 @@ HALF_reciprocal(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(dat } NPY_NO_EXPORT void -HALF_ones_like(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +HALF_ones_like(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { OUTPUT_LOOP { *((npy_half *)op1) = NPY_HALF_ONE; @@ -1770,7 +1770,7 @@ HALF_ones_like(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data } NPY_NO_EXPORT void -HALF_conjugate(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_conjugate(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const npy_half in1 = *(npy_half *)ip1; @@ -1779,7 +1779,7 @@ HALF_conjugate(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func } NPY_NO_EXPORT void -HALF_absolute(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_absolute(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const npy_half in1 = *(npy_half *)ip1; @@ -1788,7 +1788,7 @@ HALF_absolute(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func) } NPY_NO_EXPORT void -HALF_negative(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_negative(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const npy_half in1 = *(npy_half *)ip1; @@ -1797,7 +1797,7 @@ HALF_negative(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func) } NPY_NO_EXPORT void -HALF_sign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_sign(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { /* Sign of nan is nan */ UNARY_LOOP { @@ -1809,7 +1809,7 @@ HALF_sign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) } NPY_NO_EXPORT void -HALF_modf(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_modf(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { float temp; @@ -1822,7 +1822,7 @@ HALF_modf(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) #ifdef HAVE_FREXPF NPY_NO_EXPORT void -HALF_frexp(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_frexp(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP_TWO_OUT { const float in1 = npy_half_to_float(*(npy_half *)ip1); @@ -1833,7 +1833,7 @@ HALF_frexp(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) #ifdef HAVE_LDEXPF NPY_NO_EXPORT void -HALF_ldexp(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_ldexp(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const float in1 = npy_half_to_float(*(npy_half *)ip1); @@ -1843,7 +1843,7 @@ HALF_ldexp(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) } NPY_NO_EXPORT void -HALF_ldexp_long(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +HALF_ldexp_long(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { /* * Additional loop to handle long integer inputs (cf. #866, #1633). @@ -1907,7 +1907,7 @@ HALF_ldexp_long(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(fun * #OP = +, -# */ NPY_NO_EXPORT void -C@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -1921,7 +1921,7 @@ C@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func /**end repeat1**/ NPY_NO_EXPORT void -C@TYPE@_multiply(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@_multiply(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -1934,7 +1934,7 @@ C@TYPE@_multiply(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(fu } NPY_NO_EXPORT void -C@TYPE@_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@_divide(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -1966,7 +1966,7 @@ C@TYPE@_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func } NPY_NO_EXPORT void -C@TYPE@_floor_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@_floor_divide(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -1991,7 +1991,7 @@ C@TYPE@_floor_divide(char **args, intp *dimensions, intp *steps, void *NPY_UNUSE * #OP = CGT, CGE, CLT, CLE, CEQ, CNE# */ NPY_NO_EXPORT void -C@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -2009,7 +2009,7 @@ C@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func #OP2 = &&, ||# */ NPY_NO_EXPORT void -C@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -2022,7 +2022,7 @@ C@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func /**end repeat1**/ NPY_NO_EXPORT void -C@TYPE@_logical_xor(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@_logical_xor(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -2036,7 +2036,7 @@ C@TYPE@_logical_xor(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED } NPY_NO_EXPORT void -C@TYPE@_logical_not(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@_logical_not(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -2051,7 +2051,7 @@ C@TYPE@_logical_not(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED * #OP = ||, ||, &&# **/ NPY_NO_EXPORT void -C@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -2062,7 +2062,7 @@ C@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func /**end repeat1**/ NPY_NO_EXPORT void -C@TYPE@_square(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +C@TYPE@_square(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { UNARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -2073,7 +2073,7 @@ C@TYPE@_square(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data } NPY_NO_EXPORT void -C@TYPE@_reciprocal(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +C@TYPE@_reciprocal(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { UNARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -2093,7 +2093,7 @@ C@TYPE@_reciprocal(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED( } NPY_NO_EXPORT void -C@TYPE@_ones_like(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(data)) +C@TYPE@_ones_like(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(data)) { OUTPUT_LOOP { ((@type@ *)op1)[0] = 1; @@ -2102,7 +2102,7 @@ C@TYPE@_ones_like(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(d } NPY_NO_EXPORT void -C@TYPE@_conjugate(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) { +C@TYPE@_conjugate(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; const @type@ in1i = ((@type@ *)ip1)[1]; @@ -2112,7 +2112,7 @@ C@TYPE@_conjugate(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(f } NPY_NO_EXPORT void -C@TYPE@_absolute(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@_absolute(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -2122,7 +2122,7 @@ C@TYPE@_absolute(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(fu } NPY_NO_EXPORT void -C@TYPE@__arg(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@__arg(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { UNARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -2132,7 +2132,7 @@ C@TYPE@__arg(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) } NPY_NO_EXPORT void -C@TYPE@_sign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@_sign(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { /* fixme: sign of nan is currently 0 */ UNARY_LOOP { @@ -2150,7 +2150,7 @@ C@TYPE@_sign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) * #OP = CGE, CLE# */ NPY_NO_EXPORT void -C@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -2174,7 +2174,7 @@ C@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func * #OP = CGE, CLE# */ NPY_NO_EXPORT void -C@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +C@TYPE@_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { const @type@ in1r = ((@type@ *)ip1)[0]; @@ -2215,7 +2215,7 @@ C@TYPE@_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func * #OP = EQ, NE, GT, GE, LT, LE# */ NPY_NO_EXPORT void -OBJECT_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) { +OBJECT_@kind@(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { BINARY_LOOP { PyObject *in1 = *(PyObject **)ip1; PyObject *in2 = *(PyObject **)ip2; @@ -2231,7 +2231,7 @@ OBJECT_@kind@(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func) /**end repeat**/ NPY_NO_EXPORT void -OBJECT_sign(char **args, intp *dimensions, intp *steps, void *NPY_UNUSED(func)) +OBJECT_sign(char **args, npy_intp *dimensions, npy_intp *steps, void *NPY_UNUSED(func)) { #if defined(NPY_PY3K) PyObject *zero = PyLong_FromLong(0); diff --git a/numpy/core/src/umath/ufunc_object.c b/numpy/core/src/umath/ufunc_object.c index 7d72b070b..a89c0f235 100644 --- a/numpy/core/src/umath/ufunc_object.c +++ b/numpy/core/src/umath/ufunc_object.c @@ -721,7 +721,8 @@ static int get_ufunc_arguments(PyUFuncObject *self, NPY_CASTING *out_casting, PyObject **out_extobj, PyObject **out_typetup, - int *out_subok) + int *out_subok, + PyArrayObject **out_wheremask) { npy_intp i, nargs, nin = self->nin; PyObject *obj, *context; @@ -732,6 +733,12 @@ static int get_ufunc_arguments(PyUFuncObject *self, ufunc_name = self->name ? self->name : "<unnamed ufunc>"; + *out_extobj = NULL; + *out_typetup = NULL; + if (out_wheremask != NULL) { + *out_wheremask = NULL; + } + /* Check number of arguments */ nargs = PyTuple_Size(args); if ((nargs < nin) || (nargs > self->nargs)) { @@ -843,6 +850,25 @@ static int get_ufunc_arguments(PyUFuncObject *self, bad_arg = 0; } break; + case 'd': + /* Another way to specify 'sig' */ + if (strncmp(str,"dtype",5) == 0) { + /* Allow this parameter to be None */ + PyArray_Descr *dtype; + if (!PyArray_DescrConverter2(value, &dtype)) { + goto fail; + } + if (dtype != NULL) { + if (*out_typetup != NULL) { + PyErr_SetString(PyExc_RuntimeError, + "cannot specify both 'sig' and 'dtype'"); + goto fail; + } + *out_typetup = Py_BuildValue("(N)", dtype); + } + bad_arg = 0; + } + break; case 'e': /* * Overrides the global parameters buffer size, @@ -910,24 +936,27 @@ static int get_ufunc_arguments(PyUFuncObject *self, bad_arg = 0; } break; - case 'd': - /* Another way to specify 'sig' */ - if (strncmp(str,"dtype",5) == 0) { - /* Allow this parameter to be None */ + case 'w': + /* + * Provides a boolean array 'where=' mask if + * out_wheremask is supplied. + */ + if (out_wheremask != NULL && + strncmp(str,"where",5) == 0) { PyArray_Descr *dtype; - if (!PyArray_DescrConverter2(value, &dtype)) { + dtype = PyArray_DescrFromType(NPY_BOOL); + if (dtype == NULL) { goto fail; } - if (dtype != NULL) { - if (*out_typetup != NULL) { - PyErr_SetString(PyExc_RuntimeError, - "cannot specify both 'sig' and 'dtype'"); - goto fail; - } - *out_typetup = Py_BuildValue("(N)", dtype); + *out_wheremask = (PyArrayObject *)PyArray_FromAny( + value, dtype, + 0, 0, 0, NULL); + if (*out_wheremask == NULL) { + goto fail; } bad_arg = 0; } + break; } if (bad_arg) { @@ -943,144 +972,15 @@ static int get_ufunc_arguments(PyUFuncObject *self, fail: Py_XDECREF(str_key_obj); - return -1; -} - -static const char * -_casting_to_string(NPY_CASTING casting) -{ - switch (casting) { - case NPY_NO_CASTING: - return "no"; - case NPY_EQUIV_CASTING: - return "equiv"; - case NPY_SAFE_CASTING: - return "safe"; - case NPY_SAME_KIND_CASTING: - return "same_kind"; - case NPY_UNSAFE_CASTING: - return "unsafe"; - default: - return "<unknown>"; + Py_XDECREF(*out_extobj); + *out_extobj = NULL; + Py_XDECREF(*out_typetup); + *out_typetup = NULL; + if (out_wheremask != NULL) { + Py_XDECREF(*out_wheremask); + *out_wheremask = NULL; } -} - - -static int -ufunc_loop_matches(PyUFuncObject *self, - PyArrayObject **op, - NPY_CASTING input_casting, - NPY_CASTING output_casting, - int any_object, - int use_min_scalar, - int *types, - int *out_no_castable_output, - char *out_err_src_typecode, - char *out_err_dst_typecode) -{ - npy_intp i, nin = self->nin, nop = nin + self->nout; - - /* - * First check if all the inputs can be safely cast - * to the types for this function - */ - for (i = 0; i < nin; ++i) { - PyArray_Descr *tmp; - - /* - * If no inputs are objects and there are more than one - * loop, don't allow conversion to object. The rationale - * behind this is mostly performance. Except for custom - * ufuncs built with just one object-parametered inner loop, - * only the types that are supported are implemented. Trying - * the object version of logical_or on float arguments doesn't - * seem right. - */ - if (types[i] == NPY_OBJECT && !any_object && self->ntypes > 1) { - return 0; - } - - tmp = PyArray_DescrFromType(types[i]); - if (tmp == NULL) { - return -1; - } - -#if NPY_UF_DBG_TRACING - printf("Checking type for op %d, type %d: ", (int)i, (int)types[i]); - PyObject_Print((PyObject *)tmp, stdout, 0); - printf(", operand type: "); - PyObject_Print((PyObject *)PyArray_DESCR(op[i]), stdout, 0); - printf("\n"); -#endif - /* - * If all the inputs are scalars, use the regular - * promotion rules, not the special value-checking ones. - */ - if (!use_min_scalar) { - if (!PyArray_CanCastTypeTo(PyArray_DESCR(op[i]), tmp, - input_casting)) { - Py_DECREF(tmp); - return 0; - } - } - else { - if (!PyArray_CanCastArrayTo(op[i], tmp, input_casting)) { - Py_DECREF(tmp); - return 0; - } - } - Py_DECREF(tmp); - } - NPY_UF_DBG_PRINT("The inputs all worked\n"); - - /* - * If all the inputs were ok, then check casting back to the - * outputs. - */ - for (i = nin; i < nop; ++i) { - if (op[i] != NULL) { - PyArray_Descr *tmp = PyArray_DescrFromType(types[i]); - if (tmp == NULL) { - return -1; - } - if (!PyArray_CanCastTypeTo(tmp, PyArray_DESCR(op[i]), - output_casting)) { - if (!(*out_no_castable_output)) { - *out_no_castable_output = 1; - *out_err_src_typecode = tmp->type; - *out_err_dst_typecode = PyArray_DESCR(op[i])->type; - } - Py_DECREF(tmp); - return 0; - } - Py_DECREF(tmp); - } - } - NPY_UF_DBG_PRINT("The outputs all worked\n"); - - return 1; -} - -static int -set_ufunc_loop_data_types(PyUFuncObject *self, PyArrayObject **op, - PyArray_Descr **out_dtype, - int *types) -{ - int i, nin = self->nin, nop = nin + self->nout; - - /* Fill the dtypes array */ - for (i = 0; i < nop; ++i) { - out_dtype[i] = PyArray_DescrFromType(types[i]); - if (out_dtype[i] == NULL) { - while (--i >= 0) { - Py_DECREF(out_dtype[i]); - out_dtype[i] = NULL; - } - return -1; - } - } - - return 0; + return -1; } /* @@ -1135,1910 +1035,6 @@ check_for_trivial_loop(PyUFuncObject *self, return 1; } -/* - * Does a search through the arguments and the loops - */ -static int -find_ufunc_matching_userloop(PyUFuncObject *self, - PyArrayObject **op, - NPY_CASTING input_casting, - NPY_CASTING output_casting, - int any_object, - int use_min_scalar, - PyArray_Descr **out_dtype, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata, - int *out_no_castable_output, - char *out_err_src_typecode, - char *out_err_dst_typecode) -{ - npy_intp i, nin = self->nin; - PyUFunc_Loop1d *funcdata; - - /* Use this to try to avoid repeating the same userdef loop search */ - int last_userdef = -1; - - for (i = 0; i < nin; ++i) { - int type_num = PyArray_DESCR(op[i])->type_num; - if (type_num != last_userdef && PyTypeNum_ISUSERDEF(type_num)) { - PyObject *key, *obj; - - last_userdef = type_num; - - key = PyInt_FromLong(type_num); - if (key == NULL) { - return -1; - } - obj = PyDict_GetItem(self->userloops, key); - Py_DECREF(key); - if (obj == NULL) { - continue; - } - funcdata = (PyUFunc_Loop1d *)NpyCapsule_AsVoidPtr(obj); - while (funcdata != NULL) { - int *types = funcdata->arg_types; - switch (ufunc_loop_matches(self, op, - input_casting, output_casting, - any_object, use_min_scalar, - types, - out_no_castable_output, out_err_src_typecode, - out_err_dst_typecode)) { - /* Error */ - case -1: - return -1; - /* Found a match */ - case 1: - set_ufunc_loop_data_types(self, op, out_dtype, types); - - /* Save the inner loop and its data */ - *out_innerloop = funcdata->func; - *out_innerloopdata = funcdata->data; - - NPY_UF_DBG_PRINT("Returning userdef inner " - "loop successfully\n"); - - return 0; - } - - funcdata = funcdata->next; - } - } - } - - /* Didn't find a match */ - return 0; -} - -/* - * Does a search through the arguments and the loops - */ -static int -find_ufunc_specified_userloop(PyUFuncObject *self, - int n_specified, - int *specified_types, - PyArrayObject **op, - NPY_CASTING casting, - int any_object, - int use_min_scalar, - PyArray_Descr **out_dtype, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata) -{ - int i, j, nin = self->nin, nop = nin + self->nout; - PyUFunc_Loop1d *funcdata; - - /* Use this to try to avoid repeating the same userdef loop search */ - int last_userdef = -1; - - int no_castable_output = 0; - char err_src_typecode = '-', err_dst_typecode = '-'; - - for (i = 0; i < nin; ++i) { - int type_num = PyArray_DESCR(op[i])->type_num; - if (type_num != last_userdef && PyTypeNum_ISUSERDEF(type_num)) { - PyObject *key, *obj; - - last_userdef = type_num; - - key = PyInt_FromLong(type_num); - if (key == NULL) { - return -1; - } - obj = PyDict_GetItem(self->userloops, key); - Py_DECREF(key); - if (obj == NULL) { - continue; - } - funcdata = (PyUFunc_Loop1d *)NpyCapsule_AsVoidPtr(obj); - while (funcdata != NULL) { - int *types = funcdata->arg_types; - int matched = 1; - - if (n_specified == nop) { - for (j = 0; j < nop; ++j) { - if (types[j] != specified_types[j]) { - matched = 0; - break; - } - } - } else { - if (types[nin] != specified_types[0]) { - matched = 0; - } - } - if (!matched) { - continue; - } - - switch (ufunc_loop_matches(self, op, - casting, casting, - any_object, use_min_scalar, - types, - &no_castable_output, &err_src_typecode, - &err_dst_typecode)) { - /* It works */ - case 1: - set_ufunc_loop_data_types(self, op, out_dtype, types); - - /* Save the inner loop and its data */ - *out_innerloop = funcdata->func; - *out_innerloopdata = funcdata->data; - - NPY_UF_DBG_PRINT("Returning userdef inner " - "loop successfully\n"); - - return 0; - /* Didn't match */ - case 0: - PyErr_Format(PyExc_TypeError, - "found a user loop for ufunc '%s' " - "matching the type-tuple, " - "but the inputs and/or outputs could not be " - "cast according to the casting rule", - self->name ? self->name : "(unknown)"); - return -1; - /* Error */ - case -1: - return -1; - } - - funcdata = funcdata->next; - } - } - } - - /* Didn't find a match */ - return 0; -} - -/* - * Provides an ordering for the dtype 'kind' character codes, to help - * determine when to use the min_scalar_type function. This groups - * 'kind' into boolean, integer, floating point, and everything else. - */ - -static int -dtype_kind_to_simplified_ordering(char kind) -{ - switch (kind) { - /* Boolean kind */ - case 'b': - return 0; - /* Unsigned int kind */ - case 'u': - /* Signed int kind */ - case 'i': - return 1; - /* Float kind */ - case 'f': - /* Complex kind */ - case 'c': - return 2; - /* Anything else */ - default: - return 3; - } -} - -static int -should_use_min_scalar(PyArrayObject **op, int nop) -{ - int i, use_min_scalar, kind; - int all_scalars = 1, max_scalar_kind = -1, max_array_kind = -1; - - /* - * Determine if there are any scalars, and if so, whether - * the maximum "kind" of the scalars surpasses the maximum - * "kind" of the arrays - */ - use_min_scalar = 0; - if (nop > 1) { - for(i = 0; i < nop; ++i) { - kind = dtype_kind_to_simplified_ordering( - PyArray_DESCR(op[i])->kind); - if (PyArray_NDIM(op[i]) == 0) { - if (kind > max_scalar_kind) { - max_scalar_kind = kind; - } - } - else { - all_scalars = 0; - if (kind > max_array_kind) { - max_array_kind = kind; - } - - } - } - - /* Indicate whether to use the min_scalar_type function */ - if (!all_scalars && max_array_kind >= max_scalar_kind) { - use_min_scalar = 1; - } - } - - return use_min_scalar; -} - -/* - * Does a linear search for the best inner loop of the ufunc. - * - * Note that if an error is returned, the caller must free the non-zero - * references in out_dtype. This function does not do its own clean-up. - */ -static int -find_best_ufunc_inner_loop(PyUFuncObject *self, - PyArrayObject **op, - NPY_CASTING input_casting, - NPY_CASTING output_casting, - int any_object, - PyArray_Descr **out_dtype, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata) -{ - npy_intp i, j, nin = self->nin, nop = nin + self->nout; - int types[NPY_MAXARGS]; - char *ufunc_name; - int no_castable_output, use_min_scalar; - - /* For making a better error message on coercion error */ - char err_dst_typecode = '-', err_src_typecode = '-'; - - ufunc_name = self->name ? self->name : "(unknown)"; - - use_min_scalar = should_use_min_scalar(op, nin); - - /* If the ufunc has userloops, search for them. */ - if (self->userloops) { - switch (find_ufunc_matching_userloop(self, op, - input_casting, output_casting, - any_object, use_min_scalar, - out_dtype, out_innerloop, out_innerloopdata, - &no_castable_output, &err_src_typecode, - &err_dst_typecode)) { - /* Error */ - case -1: - return -1; - /* A loop was found */ - case 1: - return 0; - } - } - - /* - * Determine the UFunc loop. This could in general be *much* faster, - * and a better way to implement it might be for the ufunc to - * provide a function which gives back the result type and inner - * loop function. - * - * A default fast mechanism could be provided for functions which - * follow the most typical pattern, when all functions have signatures - * "xx...x -> x" for some built-in data type x, as follows. - * - Use PyArray_ResultType to get the output type - * - Look up the inner loop in a table based on the output type_num - * - * The method for finding the loop in the previous code did not - * appear consistent (as noted by some asymmetry in the generated - * coercion tables for np.add). - */ - no_castable_output = 0; - for (i = 0; i < self->ntypes; ++i) { - char *orig_types = self->types + i*self->nargs; - - /* Copy the types into an int array for matching */ - for (j = 0; j < nop; ++j) { - types[j] = orig_types[j]; - } - - NPY_UF_DBG_PRINT1("Trying function loop %d\n", (int)i); - switch (ufunc_loop_matches(self, op, - input_casting, output_casting, - any_object, use_min_scalar, - types, - &no_castable_output, &err_src_typecode, - &err_dst_typecode)) { - /* Error */ - case -1: - return -1; - /* Found a match */ - case 1: - set_ufunc_loop_data_types(self, op, out_dtype, types); - - /* Save the inner loop and its data */ - *out_innerloop = self->functions[i]; - *out_innerloopdata = self->data[i]; - - NPY_UF_DBG_PRINT("Returning inner loop successfully\n"); - - return 0; - } - - } - - /* If no function was found, throw an error */ - NPY_UF_DBG_PRINT("No loop was found\n"); - if (no_castable_output) { - PyErr_Format(PyExc_TypeError, - "ufunc '%s' output (typecode '%c') could not be coerced to " - "provided output parameter (typecode '%c') according " - "to the casting rule '%s'", - ufunc_name, err_src_typecode, err_dst_typecode, - _casting_to_string(output_casting)); - } - else { - /* - * TODO: We should try again if the casting rule is same_kind - * or unsafe, and look for a function more liberally. - */ - PyErr_Format(PyExc_TypeError, - "ufunc '%s' not supported for the input types, and the " - "inputs could not be safely coerced to any supported " - "types according to the casting rule '%s'", - ufunc_name, - _casting_to_string(input_casting)); - } - - return -1; -} - -/* - * Does a linear search for the inner loop of the ufunc specified by type_tup. - * - * Note that if an error is returned, the caller must free the non-zero - * references in out_dtype. This function does not do its own clean-up. - */ -static int -find_specified_ufunc_inner_loop(PyUFuncObject *self, - PyObject *type_tup, - PyArrayObject **op, - NPY_CASTING casting, - int any_object, - PyArray_Descr **out_dtype, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata) -{ - npy_intp i, j, n, nin = self->nin, nop = nin + self->nout; - int n_specified = 0; - int specified_types[NPY_MAXARGS], types[NPY_MAXARGS]; - char *ufunc_name; - int no_castable_output, use_min_scalar; - - /* For making a better error message on coercion error */ - char err_dst_typecode = '-', err_src_typecode = '-'; - - ufunc_name = self->name ? self->name : "(unknown)"; - - use_min_scalar = should_use_min_scalar(op, nin); - - /* Fill in specified_types from the tuple or string */ - if (PyTuple_Check(type_tup)) { - n = PyTuple_GET_SIZE(type_tup); - if (n != 1 && n != nop) { - PyErr_Format(PyExc_ValueError, - "a type-tuple must be specified " \ - "of length 1 or %d for ufunc '%s'", (int)nop, - self->name ? self->name : "(unknown)"); - return -1; - } - - for (i = 0; i < n; ++i) { - PyArray_Descr *dtype = NULL; - if (!PyArray_DescrConverter(PyTuple_GET_ITEM(type_tup, i), - &dtype)) { - return -1; - } - specified_types[i] = dtype->type_num; - Py_DECREF(dtype); - } - - n_specified = n; - } - else if (PyBytes_Check(type_tup) || PyUnicode_Check(type_tup)) { - Py_ssize_t length; - char *str; - PyObject *str_obj = NULL; - - if (PyUnicode_Check(type_tup)) { - str_obj = PyUnicode_AsASCIIString(type_tup); - if (str_obj == NULL) { - return -1; - } - type_tup = str_obj; - } - - if (!PyBytes_AsStringAndSize(type_tup, &str, &length) < 0) { - Py_XDECREF(str_obj); - return -1; - } - if (length != 1 && (length != nop + 2 || - str[nin] != '-' || str[nin+1] != '>')) { - PyErr_Format(PyExc_ValueError, - "a type-string for %s, " \ - "requires 1 typecode, or " - "%d typecode(s) before " \ - "and %d after the -> sign", - self->name ? self->name : "(unknown)", - self->nin, self->nout); - Py_XDECREF(str_obj); - return -1; - } - if (length == 1) { - PyArray_Descr *dtype; - n_specified = 1; - dtype = PyArray_DescrFromType(str[0]); - if (dtype == NULL) { - Py_XDECREF(str_obj); - return -1; - } - NPY_UF_DBG_PRINT2("signature character '%c', type num %d\n", - str[0], dtype->type_num); - specified_types[0] = dtype->type_num; - Py_DECREF(dtype); - } - else { - PyArray_Descr *dtype; - n_specified = (int)nop; - - for (i = 0; i < nop; ++i) { - npy_intp istr = i < nin ? i : i+2; - - dtype = PyArray_DescrFromType(str[istr]); - if (dtype == NULL) { - Py_XDECREF(str_obj); - return -1; - } - NPY_UF_DBG_PRINT2("signature character '%c', type num %d\n", - str[istr], dtype->type_num); - specified_types[i] = dtype->type_num; - Py_DECREF(dtype); - } - } - Py_XDECREF(str_obj); - } - - /* If the ufunc has userloops, search for them. */ - if (self->userloops) { - NPY_UF_DBG_PRINT("Searching user loops for specified sig\n"); - switch (find_ufunc_specified_userloop(self, - n_specified, specified_types, - op, casting, - any_object, use_min_scalar, - out_dtype, out_innerloop, out_innerloopdata)) { - /* Error */ - case -1: - return -1; - /* Found matching loop */ - case 1: - return 0; - } - } - - NPY_UF_DBG_PRINT("Searching loops for specified sig\n"); - for (i = 0; i < self->ntypes; ++i) { - char *orig_types = self->types + i*self->nargs; - int matched = 1; - - NPY_UF_DBG_PRINT1("Trying function loop %d\n", (int)i); - - /* Copy the types into an int array for matching */ - for (j = 0; j < nop; ++j) { - types[j] = orig_types[j]; - } - - if (n_specified == nop) { - for (j = 0; j < nop; ++j) { - if (types[j] != specified_types[j]) { - matched = 0; - break; - } - } - } else { - NPY_UF_DBG_PRINT2("Specified type: %d, first output type: %d\n", - specified_types[0], types[nin]); - if (types[nin] != specified_types[0]) { - matched = 0; - } - } - if (!matched) { - continue; - } - - NPY_UF_DBG_PRINT("It matches, confirming type casting\n"); - switch (ufunc_loop_matches(self, op, - casting, casting, - any_object, use_min_scalar, - types, - &no_castable_output, &err_src_typecode, - &err_dst_typecode)) { - /* Error */ - case -1: - return -1; - /* It worked */ - case 1: - set_ufunc_loop_data_types(self, op, out_dtype, types); - - /* Save the inner loop and its data */ - *out_innerloop = self->functions[i]; - *out_innerloopdata = self->data[i]; - - NPY_UF_DBG_PRINT("Returning specified inner loop successfully\n"); - - return 0; - /* Didn't work */ - case 0: - PyErr_Format(PyExc_TypeError, - "found a loop for ufunc '%s' " - "matching the type-tuple, " - "but the inputs and/or outputs could not be " - "cast according to the casting rule", - ufunc_name); - return -1; - } - - } - - /* If no function was found, throw an error */ - NPY_UF_DBG_PRINT("No specified loop was found\n"); - - PyErr_Format(PyExc_TypeError, - "No loop matching the specified signature was found " - "for ufunc %s", ufunc_name); - - return -1; -} - -/* - * Returns a new reference to type if it is already NBO, otherwise - * returns a copy converted to NBO. - */ -static PyArray_Descr * -ensure_dtype_nbo(PyArray_Descr *type) -{ - if (PyArray_ISNBO(type->byteorder)) { - Py_INCREF(type); - return type; - } - else { - return PyArray_DescrNewByteorder(type, NPY_NATIVE); - } -} - -/*UFUNC_API - * - * This function applies the default type resolution rules - * for the provided ufunc, filling out_dtypes, out_innerloop, - * and out_innerloopdata. - * - * Returns 0 on success, -1 on error. - */ -NPY_NO_EXPORT int -PyUFunc_DefaultTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata) -{ - int i, nop = ufunc->nin + ufunc->nout; - int retval = 0, any_object = 0; - NPY_CASTING input_casting; - - for (i = 0; i < nop; ++i) { - if (operands[i] != NULL && - PyTypeNum_ISOBJECT(PyArray_DESCR(operands[i])->type_num)) { - any_object = 1; - break; - } - } - - /* - * Decide the casting rules for inputs and outputs. We want - * NPY_SAFE_CASTING or stricter, so that the loop selection code - * doesn't choose an integer loop for float inputs, for example. - */ - input_casting = (casting > NPY_SAFE_CASTING) ? NPY_SAFE_CASTING : casting; - - if (type_tup == NULL) { - /* Find the best ufunc inner loop, and fill in the dtypes */ - retval = find_best_ufunc_inner_loop(ufunc, operands, - input_casting, casting, any_object, - out_dtypes, out_innerloop, out_innerloopdata); - } else { - /* Find the specified ufunc inner loop, and fill in the dtypes */ - retval = find_specified_ufunc_inner_loop(ufunc, type_tup, - operands, casting, any_object, out_dtypes, - out_innerloop, out_innerloopdata); - } - - return retval; -} - -/* - * This function applies special type resolution rules for the case - * where all the functions have the pattern XX->bool, using - * PyArray_ResultType instead of a linear search to get the best - * loop. - * - * Note that a simpler linear search through the functions loop - * is still done, but switching to a simple array lookup for - * built-in types would be better at some point. - * - * Returns 0 on success, -1 on error. - */ -NPY_NO_EXPORT int -PyUFunc_SimpleBinaryComparisonTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata) -{ - int i, type_num, type_num1, type_num2; - char *ufunc_name; - - ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; - - if (ufunc->nin != 2 || ufunc->nout != 1) { - PyErr_Format(PyExc_RuntimeError, "ufunc %s is configured " - "to use binary comparison type resolution but has " - "the wrong number of inputs or outputs", - ufunc_name); - return -1; - } - - /* - * Use the default type resolution if there's a custom data type - * or object arrays. - */ - type_num1 = PyArray_DESCR(operands[0])->type_num; - type_num2 = PyArray_DESCR(operands[1])->type_num; - if (type_num1 >= NPY_NTYPES || type_num2 >= NPY_NTYPES || - type_num1 == NPY_OBJECT || type_num2 == NPY_OBJECT) { - return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, - type_tup, out_dtypes, out_innerloop, out_innerloopdata); - } - - if (type_tup == NULL) { - /* Input types are the result type */ - out_dtypes[0] = PyArray_ResultType(2, operands, 0, NULL); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - } - else { - /* - * If the type tuple isn't a single-element tuple, let the - * default type resolution handle this one. - */ - if (!PyTuple_Check(type_tup) || PyTuple_GET_SIZE(type_tup) != 1) { - return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, - type_tup, out_dtypes, out_innerloop, out_innerloopdata); - } - - if (!PyArray_DescrCheck(PyTuple_GET_ITEM(type_tup, 0))) { - PyErr_SetString(PyExc_ValueError, - "require data type in the type tuple"); - return -1; - } - - out_dtypes[0] = ensure_dtype_nbo( - (PyArray_Descr *)PyTuple_GET_ITEM(type_tup, 0)); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - } - - /* Output type is always boolean */ - out_dtypes[2] = PyArray_DescrFromType(NPY_BOOL); - if (out_dtypes[2] == NULL) { - for (i = 0; i < 2; ++i) { - Py_DECREF(out_dtypes[i]); - out_dtypes[i] = NULL; - } - return -1; - } - - /* Check against the casting rules */ - if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { - for (i = 0; i < 3; ++i) { - Py_DECREF(out_dtypes[i]); - out_dtypes[i] = NULL; - } - return -1; - } - - type_num = out_dtypes[0]->type_num; - - /* If we have a built-in type, search in the functions list */ - if (type_num < NPY_NTYPES) { - char *types = ufunc->types; - int n = ufunc->ntypes; - - for (i = 0; i < n; ++i) { - if (types[3*i] == type_num) { - *out_innerloop = ufunc->functions[i]; - *out_innerloopdata = ufunc->data[i]; - return 0; - } - } - - PyErr_Format(PyExc_TypeError, - "ufunc '%s' not supported for the input types", - ufunc_name); - return -1; - } - else { - PyErr_SetString(PyExc_RuntimeError, - "user type shouldn't have resulted from type promotion"); - return -1; - } -} - -/* - * This function applies special type resolution rules for the case - * where all the functions have the pattern X->X, copying - * the input descr directly so that metadata is maintained. - * - * Note that a simpler linear search through the functions loop - * is still done, but switching to a simple array lookup for - * built-in types would be better at some point. - * - * Returns 0 on success, -1 on error. - */ -NPY_NO_EXPORT int -PyUFunc_SimpleUnaryOperationTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata) -{ - int i, type_num, type_num1; - char *ufunc_name; - - ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; - - if (ufunc->nin != 1 || ufunc->nout != 1) { - PyErr_Format(PyExc_RuntimeError, "ufunc %s is configured " - "to use unary operation type resolution but has " - "the wrong number of inputs or outputs", - ufunc_name); - return -1; - } - - /* - * Use the default type resolution if there's a custom data type - * or object arrays. - */ - type_num1 = PyArray_DESCR(operands[0])->type_num; - if (type_num1 >= NPY_NTYPES || type_num1 == NPY_OBJECT) { - return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, - type_tup, out_dtypes, out_innerloop, out_innerloopdata); - } - - if (type_tup == NULL) { - /* Input types are the result type */ - out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - } - else { - /* - * If the type tuple isn't a single-element tuple, let the - * default type resolution handle this one. - */ - if (!PyTuple_Check(type_tup) || PyTuple_GET_SIZE(type_tup) != 1) { - return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, - type_tup, out_dtypes, out_innerloop, out_innerloopdata); - } - - if (!PyArray_DescrCheck(PyTuple_GET_ITEM(type_tup, 0))) { - PyErr_SetString(PyExc_ValueError, - "require data type in the type tuple"); - return -1; - } - - out_dtypes[0] = ensure_dtype_nbo( - (PyArray_Descr *)PyTuple_GET_ITEM(type_tup, 0)); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - } - - /* Check against the casting rules */ - if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { - for (i = 0; i < 2; ++i) { - Py_DECREF(out_dtypes[i]); - out_dtypes[i] = NULL; - } - return -1; - } - - type_num = out_dtypes[0]->type_num; - - /* If we have a built-in type, search in the functions list */ - if (type_num < NPY_NTYPES) { - char *types = ufunc->types; - int n = ufunc->ntypes; - - for (i = 0; i < n; ++i) { - if (types[2*i] == type_num) { - *out_innerloop = ufunc->functions[i]; - *out_innerloopdata = ufunc->data[i]; - return 0; - } - } - - PyErr_Format(PyExc_TypeError, - "ufunc '%s' not supported for the input types", - ufunc_name); - return -1; - } - else { - PyErr_SetString(PyExc_RuntimeError, - "user type shouldn't have resulted from type promotion"); - return -1; - } -} - -/* - * The ones_like function shouldn't really be a ufunc, but while it - * still is, this provides type resolution that always forces UNSAFE - * casting. - */ -NPY_NO_EXPORT int -PyUFunc_OnesLikeTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING NPY_UNUSED(casting), - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata) -{ - return PyUFunc_SimpleUnaryOperationTypeResolution(ufunc, - NPY_UNSAFE_CASTING, - operands, type_tup, out_dtypes, - out_innerloop, out_innerloopdata); -} - - -/* - * This function applies special type resolution rules for the case - * where all the functions have the pattern XX->X, using - * PyArray_ResultType instead of a linear search to get the best - * loop. - * - * Note that a simpler linear search through the functions loop - * is still done, but switching to a simple array lookup for - * built-in types would be better at some point. - * - * Returns 0 on success, -1 on error. - */ -NPY_NO_EXPORT int -PyUFunc_SimpleBinaryOperationTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata) -{ - int i, type_num, type_num1, type_num2; - char *ufunc_name; - - ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; - - if (ufunc->nin != 2 || ufunc->nout != 1) { - PyErr_Format(PyExc_RuntimeError, "ufunc %s is configured " - "to use binary operation type resolution but has " - "the wrong number of inputs or outputs", - ufunc_name); - return -1; - } - - /* - * Use the default type resolution if there's a custom data type - * or object arrays. - */ - type_num1 = PyArray_DESCR(operands[0])->type_num; - type_num2 = PyArray_DESCR(operands[1])->type_num; - if (type_num1 >= NPY_NTYPES || type_num2 >= NPY_NTYPES || - type_num1 == NPY_OBJECT || type_num2 == NPY_OBJECT) { - return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, - type_tup, out_dtypes, out_innerloop, out_innerloopdata); - } - - if (type_tup == NULL) { - /* Input types are the result type */ - out_dtypes[0] = PyArray_ResultType(2, operands, 0, NULL); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - } - else { - /* - * If the type tuple isn't a single-element tuple, let the - * default type resolution handle this one. - */ - if (!PyTuple_Check(type_tup) || PyTuple_GET_SIZE(type_tup) != 1) { - return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, - type_tup, out_dtypes, out_innerloop, out_innerloopdata); - } - - if (!PyArray_DescrCheck(PyTuple_GET_ITEM(type_tup, 0))) { - PyErr_SetString(PyExc_ValueError, - "require data type in the type tuple"); - return -1; - } - - out_dtypes[0] = ensure_dtype_nbo( - (PyArray_Descr *)PyTuple_GET_ITEM(type_tup, 0)); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - } - - /* Check against the casting rules */ - if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { - for (i = 0; i < 3; ++i) { - Py_DECREF(out_dtypes[i]); - out_dtypes[i] = NULL; - } - return -1; - } - - type_num = out_dtypes[0]->type_num; - - /* If we have a built-in type, search in the functions list */ - if (type_num < NPY_NTYPES) { - char *types = ufunc->types; - int n = ufunc->ntypes; - - for (i = 0; i < n; ++i) { - if (types[3*i] == type_num) { - *out_innerloop = ufunc->functions[i]; - *out_innerloopdata = ufunc->data[i]; - return 0; - } - } - - PyErr_Format(PyExc_TypeError, - "ufunc '%s' not supported for the input types", - ufunc_name); - return -1; - } - else { - PyErr_SetString(PyExc_RuntimeError, - "user type shouldn't have resulted from type promotion"); - return -1; - } -} - -/* - * This function applies special type resolution rules for the absolute - * ufunc. This ufunc converts complex -> float, so isn't covered - * by the simple unary type resolution. - * - * Returns 0 on success, -1 on error. - */ -NPY_NO_EXPORT int -PyUFunc_AbsoluteTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata) -{ - /* Use the default for complex types, to find the loop producing float */ - if (PyTypeNum_ISCOMPLEX(PyArray_DESCR(operands[0])->type_num)) { - return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, - type_tup, out_dtypes, out_innerloop, out_innerloopdata); - } - else { - return PyUFunc_SimpleUnaryOperationTypeResolution(ufunc, casting, - operands, type_tup, out_dtypes, out_innerloop, - out_innerloopdata); - } -} - - -/* - * This function returns the a new reference to the - * capsule with the datetime metadata. - * - * NOTE: This function is copied from datetime.c in multiarray, - * because umath and multiarray are not linked together. - */ -static PyObject * -get_datetime_metacobj_from_dtype(PyArray_Descr *dtype) -{ - PyObject *metacobj; - - /* Check that the dtype has metadata */ - if (dtype->metadata == NULL) { - PyErr_SetString(PyExc_TypeError, - "Datetime type object is invalid, lacks metadata"); - return NULL; - } - - /* Check that the dtype has unit metadata */ - metacobj = PyDict_GetItemString(dtype->metadata, NPY_METADATA_DTSTR); - if (metacobj == NULL) { - PyErr_SetString(PyExc_TypeError, - "Datetime type object is invalid, lacks unit metadata"); - return NULL; - } - - Py_INCREF(metacobj); - return metacobj; -} - -/* - * This function returns a pointer to the DateTimeMetaData - * contained within the provided datetime dtype. - * - * NOTE: This function is copied from datetime.c in multiarray, - * because umath and multiarray are not linked together. - */ -static PyArray_DatetimeMetaData * -get_datetime_metadata_from_dtype(PyArray_Descr *dtype) -{ - PyObject *metacobj; - PyArray_DatetimeMetaData *meta = NULL; - - metacobj = get_datetime_metacobj_from_dtype(dtype); - if (metacobj == NULL) { - return NULL; - } - - /* Check that the dtype has an NpyCapsule for the metadata */ - meta = (PyArray_DatetimeMetaData *)NpyCapsule_AsVoidPtr(metacobj); - if (meta == NULL) { - PyErr_SetString(PyExc_TypeError, - "Datetime type object is invalid, unit metadata is corrupt"); - return NULL; - } - - return meta; -} - -/* - * Creates a new NPY_TIMEDELTA dtype, copying the datetime metadata - * from the given dtype. - * - * NOTE: This function is copied from datetime.c in multiarray, - * because umath and multiarray are not linked together. - */ -static PyArray_Descr * -timedelta_dtype_with_copied_meta(PyArray_Descr *dtype) -{ - PyArray_Descr *ret; - PyObject *metacobj; - - ret = PyArray_DescrNewFromType(NPY_TIMEDELTA); - if (ret == NULL) { - return NULL; - } - Py_XDECREF(ret->metadata); - ret->metadata = PyDict_New(); - if (ret->metadata == NULL) { - Py_DECREF(ret); - return NULL; - } - - metacobj = get_datetime_metacobj_from_dtype(dtype); - if (metacobj == NULL) { - Py_DECREF(ret); - return NULL; - } - - if (PyDict_SetItemString(ret->metadata, NPY_METADATA_DTSTR, - metacobj) < 0) { - Py_DECREF(metacobj); - Py_DECREF(ret); - return NULL; - } - - return ret; -} - -/* - * This function applies the type resolution rules for addition. - * In particular, there are a number of special cases with datetime: - * m8[<A>] + m8[<B>] => m8[gcd(<A>,<B>)] + m8[gcd(<A>,<B>)] - * m8[<A>] + int => m8[<A>] + m8[<A>] - * int + m8[<A>] => m8[<A>] + m8[<A>] - * M8[<A>] + int => M8[<A>] + m8[<A>] - * int + M8[<A>] => m8[<A>] + M8[<A>] - * M8[<A>] + m8[<B>] => M8[gcd(<A>,<B>)] + m8[gcd(<A>,<B>)] - * m8[<A>] + M8[<B>] => m8[gcd(<A>,<B>)] + M8[gcd(<A>,<B>)] - * TODO: Non-linear time unit cases require highly special-cased loops - * M8[<A>] + m8[Y|M|B] - * m8[Y|M|B] + M8[<A>] - */ -NPY_NO_EXPORT int -PyUFunc_AdditionTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata) -{ - int type_num1, type_num2; - char *types; - int i, n; - char *ufunc_name; - - ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; - - type_num1 = PyArray_DESCR(operands[0])->type_num; - type_num2 = PyArray_DESCR(operands[1])->type_num; - - /* Use the default when datetime and timedelta are not involved */ - if (!PyTypeNum_ISDATETIME(type_num1) && !PyTypeNum_ISDATETIME(type_num2)) { - return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, - type_tup, out_dtypes, out_innerloop, out_innerloopdata); - } - - if (type_num1 == NPY_TIMEDELTA) { - /* m8[<A>] + m8[<B>] => m8[gcd(<A>,<B>)] + m8[gcd(<A>,<B>)] */ - if (type_num2 == NPY_TIMEDELTA) { - out_dtypes[0] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), - PyArray_DESCR(operands[1])); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - } - /* m8[<A>] + M8[<B>] => m8[gcd(<A>,<B>)] + M8[gcd(<A>,<B>)] */ - else if (type_num2 == NPY_DATETIME) { - out_dtypes[1] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), - PyArray_DESCR(operands[1])); - if (out_dtypes[1] == NULL) { - return -1; - } - /* Make a new NPY_TIMEDELTA, and copy the datetime's metadata */ - out_dtypes[0] = timedelta_dtype_with_copied_meta(out_dtypes[1]); - if (out_dtypes[0] == NULL) { - Py_DECREF(out_dtypes[1]); - out_dtypes[1] = NULL; - return -1; - } - out_dtypes[2] = out_dtypes[1]; - Py_INCREF(out_dtypes[2]); - } - /* m8[<A>] + int => m8[<A>] + m8[<A>] */ - else if (PyTypeNum_ISINTEGER(type_num2) || - PyTypeNum_ISBOOL(type_num2)) { - out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - - type_num2 = NPY_TIMEDELTA; - } - else { - goto type_reso_error; - } - } - else if (type_num1 == NPY_DATETIME) { - /* M8[<A>] + m8[<B>] => M8[gcd(<A>,<B>)] + m8[gcd(<A>,<B>)] */ - if (type_num2 == NPY_TIMEDELTA) { - out_dtypes[0] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), - PyArray_DESCR(operands[1])); - if (out_dtypes[0] == NULL) { - return -1; - } - /* Make a new NPY_TIMEDELTA, and copy the datetime's metadata */ - out_dtypes[1] = timedelta_dtype_with_copied_meta(out_dtypes[0]); - if (out_dtypes[1] == NULL) { - Py_DECREF(out_dtypes[0]); - out_dtypes[0] = NULL; - return -1; - } - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - } - /* M8[<A>] + int => M8[<A>] + m8[<A>] */ - else if (PyTypeNum_ISINTEGER(type_num2) || - PyTypeNum_ISBOOL(type_num2)) { - out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); - if (out_dtypes[0] == NULL) { - return -1; - } - /* Make a new NPY_TIMEDELTA, and copy type1's metadata */ - out_dtypes[1] = timedelta_dtype_with_copied_meta( - PyArray_DESCR(operands[0])); - if (out_dtypes[1] == NULL) { - Py_DECREF(out_dtypes[0]); - out_dtypes[0] = NULL; - return -1; - } - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - - type_num2 = NPY_TIMEDELTA; - } - else { - goto type_reso_error; - } - } - else if (PyTypeNum_ISINTEGER(type_num1) || PyTypeNum_ISBOOL(type_num1)) { - /* int + m8[<A>] => m8[<A>] + m8[<A>] */ - if (type_num2 == NPY_TIMEDELTA) { - out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[1])); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - - type_num1 = NPY_TIMEDELTA; - } - else if (type_num2 == NPY_DATETIME) { - /* Make a new NPY_TIMEDELTA, and copy type2's metadata */ - out_dtypes[0] = timedelta_dtype_with_copied_meta( - PyArray_DESCR(operands[1])); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = ensure_dtype_nbo(PyArray_DESCR(operands[1])); - if (out_dtypes[1] == NULL) { - Py_DECREF(out_dtypes[0]); - out_dtypes[0] = NULL; - return -1; - } - out_dtypes[2] = out_dtypes[1]; - Py_INCREF(out_dtypes[2]); - - type_num1 = NPY_TIMEDELTA; - } - else { - goto type_reso_error; - } - } - else { - goto type_reso_error; - } - - /* Check against the casting rules */ - if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { - for (i = 0; i < 3; ++i) { - Py_DECREF(out_dtypes[i]); - out_dtypes[i] = NULL; - } - return -1; - } - - /* Search in the functions list */ - types = ufunc->types; - n = ufunc->ntypes; - - for (i = 0; i < n; ++i) { - if (types[3*i] == type_num1 && types[3*i+1] == type_num2) { - *out_innerloop = ufunc->functions[i]; - *out_innerloopdata = ufunc->data[i]; - return 0; - } - } - - PyErr_Format(PyExc_TypeError, - "internal error: could not find appropriate datetime " - "inner loop in %s ufunc", ufunc_name); - return -1; - -type_reso_error: { - PyObject *errmsg; - errmsg = PyUString_FromFormat("ufunc %s cannot use operands " - "with types ", ufunc_name); - PyUString_ConcatAndDel(&errmsg, - PyObject_Repr((PyObject *)PyArray_DESCR(operands[0]))); - PyUString_ConcatAndDel(&errmsg, - PyUString_FromString(" and ")); - PyUString_ConcatAndDel(&errmsg, - PyObject_Repr((PyObject *)PyArray_DESCR(operands[1]))); - PyErr_SetObject(PyExc_TypeError, errmsg); - return -1; - } -} - -/* - * This function applies the type resolution rules for subtraction. - * In particular, there are a number of special cases with datetime: - * m8[<A>] - m8[<B>] => m8[gcd(<A>,<B>)] - m8[gcd(<A>,<B>)] - * m8[<A>] - int => m8[<A>] - m8[<A>] - * int - m8[<A>] => m8[<A>] - m8[<A>] - * M8[<A>] - int => M8[<A>] - m8[<A>] - * M8[<A>] - m8[<B>] => M8[gcd(<A>,<B>)] - m8[gcd(<A>,<B>)] - * TODO: Non-linear time unit cases require highly special-cased loops - * M8[<A>] - m8[Y|M|B] - */ -NPY_NO_EXPORT int -PyUFunc_SubtractionTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata) -{ - int type_num1, type_num2; - char *types; - int i, n; - char *ufunc_name; - - ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; - - type_num1 = PyArray_DESCR(operands[0])->type_num; - type_num2 = PyArray_DESCR(operands[1])->type_num; - - /* Use the default when datetime and timedelta are not involved */ - if (!PyTypeNum_ISDATETIME(type_num1) && !PyTypeNum_ISDATETIME(type_num2)) { - return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, - type_tup, out_dtypes, out_innerloop, out_innerloopdata); - } - - if (type_num1 == NPY_TIMEDELTA) { - /* m8[<A>] - m8[<B>] => m8[gcd(<A>,<B>)] - m8[gcd(<A>,<B>)] */ - if (type_num2 == NPY_TIMEDELTA) { - out_dtypes[0] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), - PyArray_DESCR(operands[1])); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - } - /* m8[<A>] - int => m8[<A>] - m8[<A>] */ - else if (PyTypeNum_ISINTEGER(type_num2) || - PyTypeNum_ISBOOL(type_num2)) { - out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - - type_num2 = NPY_TIMEDELTA; - } - else { - goto type_reso_error; - } - } - else if (type_num1 == NPY_DATETIME) { - /* M8[<A>] - m8[<B>] => M8[gcd(<A>,<B>)] - m8[gcd(<A>,<B>)] */ - if (type_num2 == NPY_TIMEDELTA) { - out_dtypes[0] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), - PyArray_DESCR(operands[1])); - if (out_dtypes[0] == NULL) { - return -1; - } - /* Make a new NPY_TIMEDELTA, and copy the datetime's metadata */ - out_dtypes[1] = timedelta_dtype_with_copied_meta(out_dtypes[0]); - if (out_dtypes[1] == NULL) { - Py_DECREF(out_dtypes[0]); - out_dtypes[0] = NULL; - return -1; - } - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - } - /* M8[<A>] - int => M8[<A>] - m8[<A>] */ - else if (PyTypeNum_ISINTEGER(type_num2) || - PyTypeNum_ISBOOL(type_num2)) { - out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); - if (out_dtypes[0] == NULL) { - return -1; - } - /* Make a new NPY_TIMEDELTA, and copy type1's metadata */ - out_dtypes[1] = timedelta_dtype_with_copied_meta( - PyArray_DESCR(operands[0])); - if (out_dtypes[1] == NULL) { - Py_DECREF(out_dtypes[0]); - out_dtypes[0] = NULL; - return -1; - } - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - - type_num2 = NPY_TIMEDELTA; - } - /* M8[<A>] - M8[<B>] => M8[gcd(<A>,<B>)] - M8[gcd(<A>,<B>)] */ - else if (type_num2 == NPY_DATETIME) { - out_dtypes[0] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), - PyArray_DESCR(operands[1])); - if (out_dtypes[0] == NULL) { - return -1; - } - /* Make a new NPY_TIMEDELTA, and copy type1's metadata */ - out_dtypes[2] = timedelta_dtype_with_copied_meta(out_dtypes[0]); - if (out_dtypes[2] == NULL) { - Py_DECREF(out_dtypes[0]); - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - } - else { - goto type_reso_error; - } - } - else if (PyTypeNum_ISINTEGER(type_num1) || PyTypeNum_ISBOOL(type_num1)) { - /* int - m8[<A>] => m8[<A>] - m8[<A>] */ - if (type_num2 == NPY_TIMEDELTA) { - out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[1])); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - - type_num1 = NPY_TIMEDELTA; - } - else { - goto type_reso_error; - } - } - else { - goto type_reso_error; - } - - /* Check against the casting rules */ - if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { - for (i = 0; i < 3; ++i) { - Py_DECREF(out_dtypes[i]); - out_dtypes[i] = NULL; - } - return -1; - } - - /* Search in the functions list */ - types = ufunc->types; - n = ufunc->ntypes; - - for (i = 0; i < n; ++i) { - if (types[3*i] == type_num1 && types[3*i+1] == type_num2) { - *out_innerloop = ufunc->functions[i]; - *out_innerloopdata = ufunc->data[i]; - return 0; - } - } - - PyErr_Format(PyExc_TypeError, - "internal error: could not find appropriate datetime " - "inner loop in %s ufunc", ufunc_name); - return -1; - -type_reso_error: { - PyObject *errmsg; - errmsg = PyUString_FromFormat("ufunc %s cannot use operands " - "with types ", ufunc_name); - PyUString_ConcatAndDel(&errmsg, - PyObject_Repr((PyObject *)PyArray_DESCR(operands[0]))); - PyUString_ConcatAndDel(&errmsg, - PyUString_FromString(" and ")); - PyUString_ConcatAndDel(&errmsg, - PyObject_Repr((PyObject *)PyArray_DESCR(operands[1]))); - PyErr_SetObject(PyExc_TypeError, errmsg); - return -1; - } -} - -/* - * This function applies the type resolution rules for multiplication. - * In particular, there are a number of special cases with datetime: - * int## * m8[<A>] => int64 * m8[<A>] - * m8[<A>] * int## => m8[<A>] * int64 - * float## * m8[<A>] => float64 * m8[<A>] - * m8[<A>] * float## => m8[<A>] * float64 - */ -NPY_NO_EXPORT int -PyUFunc_MultiplicationTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata) -{ - int type_num1, type_num2; - char *types; - int i, n; - char *ufunc_name; - - ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; - - type_num1 = PyArray_DESCR(operands[0])->type_num; - type_num2 = PyArray_DESCR(operands[1])->type_num; - - /* Use the default when datetime and timedelta are not involved */ - if (!PyTypeNum_ISDATETIME(type_num1) && !PyTypeNum_ISDATETIME(type_num2)) { - return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, - type_tup, out_dtypes, out_innerloop, out_innerloopdata); - } - - if (type_num1 == NPY_TIMEDELTA) { - /* m8[<A>] * int## => m8[<A>] * int64 */ - if (PyTypeNum_ISINTEGER(type_num2) || PyTypeNum_ISBOOL(type_num2)) { - out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = PyArray_DescrNewFromType(NPY_LONGLONG); - if (out_dtypes[1] == NULL) { - Py_DECREF(out_dtypes[0]); - out_dtypes[0] = NULL; - return -1; - } - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - - type_num2 = NPY_LONGLONG; - } - /* m8[<A>] * float## => m8[<A>] * float64 */ - else if (PyTypeNum_ISFLOAT(type_num2)) { - out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = PyArray_DescrNewFromType(NPY_DOUBLE); - if (out_dtypes[1] == NULL) { - Py_DECREF(out_dtypes[0]); - out_dtypes[0] = NULL; - return -1; - } - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - - type_num2 = NPY_DOUBLE; - } - else { - goto type_reso_error; - } - } - else if (PyTypeNum_ISINTEGER(type_num1) || PyTypeNum_ISBOOL(type_num1)) { - /* int## * m8[<A>] => int64 * m8[<A>] */ - if (type_num2 == NPY_TIMEDELTA) { - out_dtypes[0] = PyArray_DescrNewFromType(NPY_LONGLONG); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = ensure_dtype_nbo(PyArray_DESCR(operands[1])); - if (out_dtypes[1] == NULL) { - Py_DECREF(out_dtypes[0]); - out_dtypes[0] = NULL; - return -1; - } - out_dtypes[2] = out_dtypes[1]; - Py_INCREF(out_dtypes[2]); - - type_num1 = NPY_LONGLONG; - } - else { - goto type_reso_error; - } - } - else if (PyTypeNum_ISFLOAT(type_num1)) { - /* float## * m8[<A>] => float64 * m8[<A>] */ - if (type_num2 == NPY_TIMEDELTA) { - out_dtypes[0] = PyArray_DescrNewFromType(NPY_DOUBLE); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = ensure_dtype_nbo(PyArray_DESCR(operands[1])); - if (out_dtypes[1] == NULL) { - Py_DECREF(out_dtypes[0]); - out_dtypes[0] = NULL; - return -1; - } - out_dtypes[2] = out_dtypes[1]; - Py_INCREF(out_dtypes[2]); - - type_num1 = NPY_DOUBLE; - } - else { - goto type_reso_error; - } - } - else { - goto type_reso_error; - } - - /* Check against the casting rules */ - if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { - for (i = 0; i < 3; ++i) { - Py_DECREF(out_dtypes[i]); - out_dtypes[i] = NULL; - } - return -1; - } - - /* Search in the functions list */ - types = ufunc->types; - n = ufunc->ntypes; - - for (i = 0; i < n; ++i) { - if (types[3*i] == type_num1 && types[3*i+1] == type_num2) { - *out_innerloop = ufunc->functions[i]; - *out_innerloopdata = ufunc->data[i]; - return 0; - } - } - - PyErr_Format(PyExc_TypeError, - "internal error: could not find appropriate datetime " - "inner loop in %s ufunc", ufunc_name); - return -1; - -type_reso_error: { - PyObject *errmsg; - errmsg = PyUString_FromFormat("ufunc %s cannot use operands " - "with types ", ufunc_name); - PyUString_ConcatAndDel(&errmsg, - PyObject_Repr((PyObject *)PyArray_DESCR(operands[0]))); - PyUString_ConcatAndDel(&errmsg, - PyUString_FromString(" and ")); - PyUString_ConcatAndDel(&errmsg, - PyObject_Repr((PyObject *)PyArray_DESCR(operands[1]))); - PyErr_SetObject(PyExc_TypeError, errmsg); - return -1; - } -} - -/* - * This function applies the type resolution rules for division. - * In particular, there are a number of special cases with datetime: - * m8[<A>] / m8[<B>] to m8[gcd(<A>,<B>)] / m8[gcd(<A>,<B>)] -> float64 - * m8[<A>] / int## to m8[<A>] / int64 -> m8[<A>] - * m8[<A>] / float## to m8[<A>] / float64 -> m8[<A>] - */ -NPY_NO_EXPORT int -PyUFunc_DivisionTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata) -{ - int type_num1, type_num2; - char *types; - int i, n; - char *ufunc_name; - - ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; - - type_num1 = PyArray_DESCR(operands[0])->type_num; - type_num2 = PyArray_DESCR(operands[1])->type_num; - - /* Use the default when datetime and timedelta are not involved */ - if (!PyTypeNum_ISDATETIME(type_num1) && !PyTypeNum_ISDATETIME(type_num2)) { - return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, - type_tup, out_dtypes, out_innerloop, out_innerloopdata); - } - - if (type_num1 == NPY_TIMEDELTA) { - /* - * m8[<A>] / m8[<B>] to - * m8[gcd(<A>,<B>)] / m8[gcd(<A>,<B>)] -> float64 - */ - if (type_num2 == NPY_TIMEDELTA) { - out_dtypes[0] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), - PyArray_DESCR(operands[1])); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = out_dtypes[0]; - Py_INCREF(out_dtypes[1]); - out_dtypes[2] = PyArray_DescrFromType(NPY_DOUBLE); - if (out_dtypes[2] == NULL) { - Py_DECREF(out_dtypes[0]); - out_dtypes[0] = NULL; - Py_DECREF(out_dtypes[1]); - out_dtypes[1] = NULL; - return -1; - } - } - /* m8[<A>] / int## => m8[<A>] / int64 */ - else if (PyTypeNum_ISINTEGER(type_num2)) { - out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = PyArray_DescrFromType(NPY_LONGLONG); - if (out_dtypes[1] == NULL) { - Py_DECREF(out_dtypes[0]); - out_dtypes[0] = NULL; - return -1; - } - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - - type_num2 = NPY_LONGLONG; - } - /* m8[<A>] / float## => m8[<A>] / float64 */ - else if (PyTypeNum_ISFLOAT(type_num2)) { - out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); - if (out_dtypes[0] == NULL) { - return -1; - } - out_dtypes[1] = PyArray_DescrNewFromType(NPY_DOUBLE); - if (out_dtypes[1] == NULL) { - Py_DECREF(out_dtypes[0]); - out_dtypes[0] = NULL; - return -1; - } - out_dtypes[2] = out_dtypes[0]; - Py_INCREF(out_dtypes[2]); - - type_num2 = NPY_DOUBLE; - } - else { - goto type_reso_error; - } - } - else { - goto type_reso_error; - } - - /* Check against the casting rules */ - if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { - for (i = 0; i < 3; ++i) { - Py_DECREF(out_dtypes[i]); - out_dtypes[i] = NULL; - } - return -1; - } - - /* Search in the functions list */ - types = ufunc->types; - n = ufunc->ntypes; - - for (i = 0; i < n; ++i) { - if (types[3*i] == type_num1 && types[3*i+1] == type_num2) { - *out_innerloop = ufunc->functions[i]; - *out_innerloopdata = ufunc->data[i]; - return 0; - } - } - - PyErr_Format(PyExc_TypeError, - "internal error: could not find appropriate datetime " - "inner loop in %s ufunc", ufunc_name); - return -1; - -type_reso_error: { - PyObject *errmsg; - errmsg = PyUString_FromFormat("ufunc %s cannot use operands " - "with types ", ufunc_name); - PyUString_ConcatAndDel(&errmsg, - PyObject_Repr((PyObject *)PyArray_DESCR(operands[0]))); - PyUString_ConcatAndDel(&errmsg, - PyUString_FromString(" and ")); - PyUString_ConcatAndDel(&errmsg, - PyObject_Repr((PyObject *)PyArray_DESCR(operands[1]))); - PyErr_SetObject(PyExc_TypeError, errmsg); - return -1; - } -} - -/*UFUNC_API - * - * Validates that the input operands can be cast to - * the input types, and the output types can be cast to - * the output operands where provided. - * - * Returns 0 on success, -1 (with exception raised) on validation failure. - */ -NPY_NO_EXPORT int -PyUFunc_ValidateCasting(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyArray_Descr **dtypes) -{ - int i, nin = ufunc->nin, nop = nin + ufunc->nout; - char *ufunc_name; - - ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; - - for (i = 0; i < nop; ++i) { - if (i < nin) { - if (!PyArray_CanCastArrayTo(operands[i], dtypes[i], casting)) { - PyObject *errmsg; - errmsg = PyUString_FromFormat("Cannot cast ufunc %s " - "input from ", ufunc_name); - PyUString_ConcatAndDel(&errmsg, - PyObject_Repr((PyObject *)PyArray_DESCR(operands[i]))); - PyUString_ConcatAndDel(&errmsg, - PyUString_FromString(" to ")); - PyUString_ConcatAndDel(&errmsg, - PyObject_Repr((PyObject *)dtypes[i])); - PyUString_ConcatAndDel(&errmsg, - PyUString_FromFormat(" with casting rule %s", - _casting_to_string(casting))); - PyErr_SetObject(PyExc_TypeError, errmsg); - return -1; - } - } else if (operands[i] != NULL) { - if (!PyArray_CanCastTypeTo(dtypes[i], - PyArray_DESCR(operands[i]), casting)) { - PyObject *errmsg; - errmsg = PyUString_FromFormat("Cannot cast ufunc %s " - "output from ", ufunc_name); - PyUString_ConcatAndDel(&errmsg, - PyObject_Repr((PyObject *)dtypes[i])); - PyUString_ConcatAndDel(&errmsg, - PyUString_FromString(" to ")); - PyUString_ConcatAndDel(&errmsg, - PyObject_Repr((PyObject *)PyArray_DESCR(operands[i]))); - PyUString_ConcatAndDel(&errmsg, - PyUString_FromFormat(" with casting rule %s", - _casting_to_string(casting))); - PyErr_SetObject(PyExc_TypeError, errmsg); - return -1; - } - } - } - - return 0; -} - static void trivial_two_operand_loop(PyArrayObject **op, PyUFuncGenericFunction innerloop, @@ -3425,6 +1421,162 @@ execute_ufunc_loop(PyUFuncObject *self, return 0; } +/* + * nin - number of inputs + * nout - number of outputs + * wheremask - if not NULL, the 'where=' parameter to the ufunc. + * op - the operands (nin + nout of them) + * order - the loop execution order/output memory order + * buffersize - how big of a buffer to use + * arr_prep - the __array_prepare__ functions for the outputs + * innerloop - the inner loop function + * innerloopdata - data to pass to the inner loop + */ +static int +execute_ufunc_masked_loop(PyUFuncObject *self, + PyArrayObject *wheremask, + PyArrayObject **op, + PyArray_Descr **dtype, + NPY_ORDER order, + npy_intp buffersize, + PyObject **arr_prep, + PyObject *arr_prep_args, + PyUFuncGenericMaskedFunction innerloop, + NpyAuxData *innerloopdata) +{ + npy_intp i, nin = self->nin, nout = self->nout; + npy_intp nop = nin + nout; + npy_uint32 op_flags[NPY_MAXARGS]; + NpyIter *iter; + char *baseptrs[NPY_MAXARGS]; + int needs_api; + + NpyIter_IterNextFunc *iternext; + char **dataptr; + npy_intp *stride; + npy_intp *count_ptr; + + PyArrayObject **op_it; + + NPY_BEGIN_THREADS_DEF; + + if (wheremask != NULL) { + if (nop + 1 > NPY_MAXARGS) { + PyErr_SetString(PyExc_ValueError, + "Too many operands when including where= parameter"); + return -1; + } + op[nop] = wheremask; + dtype[nop] = NULL; + } + + /* Set up the flags */ + for (i = 0; i < nin; ++i) { + op_flags[i] = NPY_ITER_READONLY| + NPY_ITER_ALIGNED; + } + for (i = nin; i < nop; ++i) { + op_flags[i] = NPY_ITER_WRITEONLY| + NPY_ITER_ALIGNED| + NPY_ITER_ALLOCATE| + NPY_ITER_NO_BROADCAST| + NPY_ITER_NO_SUBTYPE; + } + op_flags[nop] = NPY_ITER_READONLY; + + NPY_UF_DBG_PRINT("Making iterator\n"); + + /* + * Allocate the iterator. Because the types of the inputs + * were already checked, we use the casting rule 'unsafe' which + * is faster to calculate. + */ + iter = NpyIter_AdvancedNew(nop + ((wheremask != NULL) ? 1 : 0), op, + NPY_ITER_EXTERNAL_LOOP| + NPY_ITER_REFS_OK| + NPY_ITER_ZEROSIZE_OK| + NPY_ITER_BUFFERED| + NPY_ITER_GROWINNER| + NPY_ITER_DELAY_BUFALLOC, + order, NPY_UNSAFE_CASTING, + op_flags, dtype, + 0, NULL, NULL, buffersize); + if (iter == NULL) { + return -1; + } + + NPY_UF_DBG_PRINT("Made iterator\n"); + + needs_api = NpyIter_IterationNeedsAPI(iter); + + /* Copy any allocated outputs */ + op_it = NpyIter_GetOperandArray(iter); + for (i = nin; i < nop; ++i) { + if (op[i] == NULL) { + op[i] = op_it[i]; + Py_INCREF(op[i]); + } + } + + /* Call the __array_prepare__ functions where necessary */ + for (i = 0; i < nout; ++i) { + if (prepare_ufunc_output(self, &op[nin+i], + arr_prep[i], arr_prep_args, i) < 0) { + NpyIter_Deallocate(iter); + return -1; + } + } + + /* Only do the loop if the iteration size is non-zero */ + if (NpyIter_GetIterSize(iter) != 0) { + + /* Reset the iterator with the base pointers from the wrapped outputs */ + for (i = 0; i < nin; ++i) { + baseptrs[i] = PyArray_BYTES(op_it[i]); + } + for (i = nin; i < nop; ++i) { + baseptrs[i] = PyArray_BYTES(op[i]); + } + if (wheremask != NULL) { + baseptrs[nop] = PyArray_BYTES(op[nop]); + } + NPY_UF_DBG_PRINT("reset base pointers call:\n"); + if (NpyIter_ResetBasePointers(iter, baseptrs, NULL) != NPY_SUCCEED) { + NpyIter_Deallocate(iter); + return -1; + } + NPY_UF_DBG_PRINT("finished reset base pointers call\n"); + + /* Get the variables needed for the loop */ + iternext = NpyIter_GetIterNext(iter, NULL); + if (iternext == NULL) { + NpyIter_Deallocate(iter); + return -1; + } + dataptr = NpyIter_GetDataPtrArray(iter); + stride = NpyIter_GetInnerStrideArray(iter); + count_ptr = NpyIter_GetInnerLoopSizePtr(iter); + + if (!needs_api) { + NPY_BEGIN_THREADS; + } + + NPY_UF_DBG_PRINT("Actual inner loop:\n"); + /* Execute the loop */ + do { + NPY_UF_DBG_PRINT1("iterator loop count %d\n", (int)*count_ptr); + innerloop(dataptr, count_ptr, stride, innerloopdata); + } while (iternext(iter)); + + if (!needs_api) { + NPY_END_THREADS; + } + } + + NpyIter_Deallocate(iter); + return 0; +} + static PyObject * make_arr_prep_args(npy_intp nin, PyObject *args, PyObject *kwds) { @@ -3540,7 +1692,8 @@ PyUFunc_GeneralizedFunction(PyUFuncObject *self, /* Get all the arguments */ retval = get_ufunc_arguments(self, args, kwds, - op, &order, &casting, &extobj, &type_tup, &subok); + op, &order, &casting, &extobj, + &type_tup, &subok, NULL); if (retval < 0) { goto fail; } @@ -3858,6 +2011,8 @@ fail: * * This generic function is called with the ufunc object, the arguments to it, * and an array of (pointers to) PyArrayObjects which are NULL. + * + * 'op' is an array of at least NPY_MAXARGS PyArrayObject *. */ NPY_NO_EXPORT int PyUFunc_GenericFunction(PyUFuncObject *self, @@ -3868,6 +2023,7 @@ PyUFunc_GenericFunction(PyUFuncObject *self, int i, nop; char *ufunc_name; int retval = -1, subok = 1; + int usemaskedloop = 0; PyArray_Descr *dtype[NPY_MAXARGS]; @@ -3880,6 +2036,16 @@ PyUFunc_GenericFunction(PyUFuncObject *self, PyUFuncGenericFunction innerloop = NULL; void *innerloopdata = NULL; + /* + * The selected masked inner loop, when the 'where=' + * parameter or arrays with missing values are in op. + */ + PyUFuncGenericMaskedFunction masked_innerloop = NULL; + NpyAuxData *masked_innerloopdata = NULL; + + /* The mask provided in the 'where=' parameter */ + PyArrayObject *wheremask = NULL; + /* The __array_prepare__ function to call for each output */ PyObject *arr_prep[NPY_MAXARGS]; /* @@ -3926,11 +2092,20 @@ PyUFunc_GenericFunction(PyUFuncObject *self, /* Get all the arguments */ retval = get_ufunc_arguments(self, args, kwds, - op, &order, &casting, &extobj, &type_tup, &subok); + op, &order, &casting, &extobj, + &type_tup, &subok, &wheremask); if (retval < 0) { goto fail; } + /* + * For now just the where mask triggers this, but later arrays + * with missing data will trigger it as well. + */ + if (wheremask != NULL) { + usemaskedloop = 1; + } + /* Get the buffersize, errormask, and error object globals */ if (extobj == NULL) { if (PyUFunc_GetPyValues(ufunc_name, @@ -3949,20 +2124,33 @@ PyUFunc_GenericFunction(PyUFuncObject *self, NPY_UF_DBG_PRINT("Finding inner loop\n"); - retval = self->type_resolution_function(self, casting, - op, type_tup, dtype, &innerloop, &innerloopdata); - if (retval < 0) { - goto fail; + if (usemaskedloop) { + retval = self->type_resolution_masked_function(self, casting, + op, type_tup, dtype, + &masked_innerloop, &masked_innerloopdata); + if (retval < 0) { + goto fail; + } } + else { + retval = self->type_resolution_function(self, casting, + op, type_tup, dtype, + &innerloop, &innerloopdata); + if (retval < 0) { + goto fail; + } - /* - * This checks whether a trivial loop is ok, - * making copies of scalar and one dimensional operands if that will - * help. - */ - trivial_loop_ok = check_for_trivial_loop(self, op, dtype, buffersize); - if (trivial_loop_ok < 0) { - goto fail; + /* + * This checks whether a trivial loop is ok, + * making copies of scalar and one dimensional operands if that will + * help. + * + * Only do the trivial loop check for the unmasked version. + */ + trivial_loop_ok = check_for_trivial_loop(self, op, dtype, buffersize); + if (trivial_loop_ok < 0) { + goto fail; + } } /* @@ -4014,20 +2202,37 @@ PyUFunc_GenericFunction(PyUFuncObject *self, } } - /* If the loop wants the arrays, provide them */ - if (_does_loop_use_arrays(innerloopdata)) { + /* + * If the loop wants the arrays, provide them. + * + * TODO: Remove this, since this is already basically broken + * with the addition of the masked inner loops and + * not worth fixing. + */ + if (!usemaskedloop && _does_loop_use_arrays(innerloopdata)) { innerloopdata = (void*)op; } /* Start with the floating-point exception flags cleared */ PyUFunc_clearfperr(); - NPY_UF_DBG_PRINT("Executing inner loop\n"); - /* Do the ufunc loop */ - retval = execute_ufunc_loop(self, trivial_loop_ok, op, dtype, order, - buffersize, arr_prep, arr_prep_args, - innerloop, innerloopdata); + if (usemaskedloop) { + NPY_UF_DBG_PRINT("Executing masked inner loop\n"); + + retval = execute_ufunc_masked_loop(self, wheremask, + op, dtype, order, + buffersize, arr_prep, arr_prep_args, + masked_innerloop, masked_innerloopdata); + } + else { + NPY_UF_DBG_PRINT("Executing unmasked inner loop\n"); + + retval = execute_ufunc_loop(self, trivial_loop_ok, + op, dtype, order, + buffersize, arr_prep, arr_prep_args, + innerloop, innerloopdata); + } if (retval < 0) { goto fail; } @@ -4047,6 +2252,7 @@ PyUFunc_GenericFunction(PyUFuncObject *self, Py_XDECREF(errobj); Py_XDECREF(type_tup); Py_XDECREF(arr_prep_args); + Py_XDECREF(wheremask); NPY_UF_DBG_PRINT("Returning Success\n"); @@ -4063,6 +2269,7 @@ fail: Py_XDECREF(errobj); Py_XDECREF(type_tup); Py_XDECREF(arr_prep_args); + Py_XDECREF(wheremask); return retval; } @@ -5781,6 +3988,8 @@ PyUFunc_FromFuncAndDataAndSignature(PyUFuncGenericFunction *func, void **data, self->userloops=NULL; self->type_resolution_function = &PyUFunc_DefaultTypeResolution; + self->type_resolution_masked_function = + &PyUFunc_DefaultTypeResolutionMasked; if (name == NULL) { self->name = "?"; diff --git a/numpy/core/src/umath/ufunc_object.h b/numpy/core/src/umath/ufunc_object.h index 59754380c..a8886be05 100644 --- a/numpy/core/src/umath/ufunc_object.h +++ b/numpy/core/src/umath/ufunc_object.h @@ -7,84 +7,4 @@ ufunc_geterr(PyObject *NPY_UNUSED(dummy), PyObject *args); NPY_NO_EXPORT PyObject * ufunc_seterr(PyObject *NPY_UNUSED(dummy), PyObject *args); -NPY_NO_EXPORT int -PyUFunc_SimpleBinaryComparisonTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata); - -NPY_NO_EXPORT int -PyUFunc_SimpleUnaryOperationTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata); - -NPY_NO_EXPORT int -PyUFunc_OnesLikeTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata); - -NPY_NO_EXPORT int -PyUFunc_SimpleBinaryOperationTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata); - -NPY_NO_EXPORT int -PyUFunc_AbsoluteTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata); - -NPY_NO_EXPORT int -PyUFunc_AdditionTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata); - -NPY_NO_EXPORT int -PyUFunc_SubtractionTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata); - -NPY_NO_EXPORT int -PyUFunc_MultiplicationTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata); -NPY_NO_EXPORT int -PyUFunc_DivisionTypeResolution(PyUFuncObject *ufunc, - NPY_CASTING casting, - PyArrayObject **operands, - PyObject *type_tup, - PyArray_Descr **out_dtypes, - PyUFuncGenericFunction *out_innerloop, - void **out_innerloopdata); - #endif diff --git a/numpy/core/src/umath/ufunc_type_resolution.c b/numpy/core/src/umath/ufunc_type_resolution.c new file mode 100644 index 000000000..8eb1f8ddf --- /dev/null +++ b/numpy/core/src/umath/ufunc_type_resolution.c @@ -0,0 +1,2143 @@ +/* + * This file implements type resolution for NumPy element-wise ufuncs. + * This mechanism is still backwards-compatible with the pre-existing + * legacy mechanism, so performs much slower than is necessary. + * + * Written by Mark Wiebe (mwwiebe@gmail.com) + * Copyright (c) 2011 by Enthought, Inc. + * + * See LICENSE.txt for the license. + */ +#define _UMATHMODULE +#define NPY_NO_DEPRECATED_API + +#include "Python.h" + +#include "npy_config.h" +#ifdef ENABLE_SEPARATE_COMPILATION +#define PY_ARRAY_UNIQUE_SYMBOL _npy_umathmodule_ARRAY_API +#define NO_IMPORT_ARRAY +#endif + +#include "numpy/npy_3kcompat.h" + +#include "numpy/ufuncobject.h" +#include "ufunc_type_resolution.h" + +static const char * +npy_casting_to_string(NPY_CASTING casting) +{ + switch (casting) { + case NPY_NO_CASTING: + return "'no'"; + case NPY_EQUIV_CASTING: + return "'equiv'"; + case NPY_SAFE_CASTING: + return "'safe'"; + case NPY_SAME_KIND_CASTING: + return "'same_kind'"; + case NPY_UNSAFE_CASTING: + return "'unsafe'"; + default: + return "<unknown>"; + } +} +/*UFUNC_API + * + * Validates that the input operands can be cast to + * the input types, and the output types can be cast to + * the output operands where provided. + * + * Returns 0 on success, -1 (with exception raised) on validation failure. + */ +NPY_NO_EXPORT int +PyUFunc_ValidateCasting(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyArray_Descr **dtypes) +{ + int i, nin = ufunc->nin, nop = nin + ufunc->nout; + char *ufunc_name; + + ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; + + for (i = 0; i < nop; ++i) { + if (i < nin) { + if (!PyArray_CanCastArrayTo(operands[i], dtypes[i], casting)) { + PyObject *errmsg; + errmsg = PyUString_FromFormat("Cannot cast ufunc %s " + "input from ", ufunc_name); + PyUString_ConcatAndDel(&errmsg, + PyObject_Repr((PyObject *)PyArray_DESCR(operands[i]))); + PyUString_ConcatAndDel(&errmsg, + PyUString_FromString(" to ")); + PyUString_ConcatAndDel(&errmsg, + PyObject_Repr((PyObject *)dtypes[i])); + PyUString_ConcatAndDel(&errmsg, + PyUString_FromFormat(" with casting rule %s", + npy_casting_to_string(casting))); + PyErr_SetObject(PyExc_TypeError, errmsg); + return -1; + } + } else if (operands[i] != NULL) { + if (!PyArray_CanCastTypeTo(dtypes[i], + PyArray_DESCR(operands[i]), casting)) { + PyObject *errmsg; + errmsg = PyUString_FromFormat("Cannot cast ufunc %s " + "output from ", ufunc_name); + PyUString_ConcatAndDel(&errmsg, + PyObject_Repr((PyObject *)dtypes[i])); + PyUString_ConcatAndDel(&errmsg, + PyUString_FromString(" to ")); + PyUString_ConcatAndDel(&errmsg, + PyObject_Repr((PyObject *)PyArray_DESCR(operands[i]))); + PyUString_ConcatAndDel(&errmsg, + PyUString_FromFormat(" with casting rule %s", + npy_casting_to_string(casting))); + PyErr_SetObject(PyExc_TypeError, errmsg); + return -1; + } + } + } + + return 0; +} + +/* + * Returns a new reference to type if it is already NBO, otherwise + * returns a copy converted to NBO. + */ +static PyArray_Descr * +ensure_dtype_nbo(PyArray_Descr *type) +{ + if (PyArray_ISNBO(type->byteorder)) { + Py_INCREF(type); + return type; + } + else { + return PyArray_DescrNewByteorder(type, NPY_NATIVE); + } +} + +/*UFUNC_API + * + * This function applies the default type resolution rules + * for the provided ufunc, filling out_dtypes, out_innerloop, + * and out_innerloopdata. + * + * Returns 0 on success, -1 on error. + */ +NPY_NO_EXPORT int +PyUFunc_DefaultTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata) +{ + int i, nop = ufunc->nin + ufunc->nout; + int retval = 0, any_object = 0; + NPY_CASTING input_casting; + + for (i = 0; i < nop; ++i) { + if (operands[i] != NULL && + PyTypeNum_ISOBJECT(PyArray_DESCR(operands[i])->type_num)) { + any_object = 1; + break; + } + } + + /* + * Decide the casting rules for inputs and outputs. We want + * NPY_SAFE_CASTING or stricter, so that the loop selection code + * doesn't choose an integer loop for float inputs, for example. + */ + input_casting = (casting > NPY_SAFE_CASTING) ? NPY_SAFE_CASTING : casting; + + if (type_tup == NULL) { + /* Find the best ufunc inner loop, and fill in the dtypes */ + retval = find_best_ufunc_inner_loop(ufunc, operands, + input_casting, casting, any_object, + out_dtypes, out_innerloop, out_innerloopdata); + } else { + /* Find the specified ufunc inner loop, and fill in the dtypes */ + retval = find_specified_ufunc_inner_loop(ufunc, type_tup, + operands, casting, any_object, out_dtypes, + out_innerloop, out_innerloopdata); + } + + return retval; +} + +/* + * This function applies special type resolution rules for the case + * where all the functions have the pattern XX->bool, using + * PyArray_ResultType instead of a linear search to get the best + * loop. + * + * Note that a simpler linear search through the functions loop + * is still done, but switching to a simple array lookup for + * built-in types would be better at some point. + * + * Returns 0 on success, -1 on error. + */ +NPY_NO_EXPORT int +PyUFunc_SimpleBinaryComparisonTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata) +{ + int i, type_num, type_num1, type_num2; + char *ufunc_name; + + ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; + + if (ufunc->nin != 2 || ufunc->nout != 1) { + PyErr_Format(PyExc_RuntimeError, "ufunc %s is configured " + "to use binary comparison type resolution but has " + "the wrong number of inputs or outputs", + ufunc_name); + return -1; + } + + /* + * Use the default type resolution if there's a custom data type + * or object arrays. + */ + type_num1 = PyArray_DESCR(operands[0])->type_num; + type_num2 = PyArray_DESCR(operands[1])->type_num; + if (type_num1 >= NPY_NTYPES || type_num2 >= NPY_NTYPES || + type_num1 == NPY_OBJECT || type_num2 == NPY_OBJECT) { + return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, + type_tup, out_dtypes, out_innerloop, out_innerloopdata); + } + + if (type_tup == NULL) { + /* Input types are the result type */ + out_dtypes[0] = PyArray_ResultType(2, operands, 0, NULL); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + } + else { + /* + * If the type tuple isn't a single-element tuple, let the + * default type resolution handle this one. + */ + if (!PyTuple_Check(type_tup) || PyTuple_GET_SIZE(type_tup) != 1) { + return PyUFunc_DefaultTypeResolution(ufunc, casting, + operands, type_tup, out_dtypes, + out_innerloop, out_innerloopdata); + } + + if (!PyArray_DescrCheck(PyTuple_GET_ITEM(type_tup, 0))) { + PyErr_SetString(PyExc_ValueError, + "require data type in the type tuple"); + return -1; + } + + out_dtypes[0] = ensure_dtype_nbo( + (PyArray_Descr *)PyTuple_GET_ITEM(type_tup, 0)); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + } + + /* Output type is always boolean */ + out_dtypes[2] = PyArray_DescrFromType(NPY_BOOL); + if (out_dtypes[2] == NULL) { + for (i = 0; i < 2; ++i) { + Py_DECREF(out_dtypes[i]); + out_dtypes[i] = NULL; + } + return -1; + } + + /* Check against the casting rules */ + if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { + for (i = 0; i < 3; ++i) { + Py_DECREF(out_dtypes[i]); + out_dtypes[i] = NULL; + } + return -1; + } + + type_num = out_dtypes[0]->type_num; + + /* If we have a built-in type, search in the functions list */ + if (type_num < NPY_NTYPES) { + char *types = ufunc->types; + int n = ufunc->ntypes; + + for (i = 0; i < n; ++i) { + if (types[3*i] == type_num) { + *out_innerloop = ufunc->functions[i]; + *out_innerloopdata = ufunc->data[i]; + return 0; + } + } + + PyErr_Format(PyExc_TypeError, + "ufunc '%s' not supported for the input types", + ufunc_name); + return -1; + } + else { + PyErr_SetString(PyExc_RuntimeError, + "user type shouldn't have resulted from type promotion"); + return -1; + } +} + +/* + * This function applies special type resolution rules for the case + * where all the functions have the pattern X->X, copying + * the input descr directly so that metadata is maintained. + * + * Note that a simpler linear search through the functions loop + * is still done, but switching to a simple array lookup for + * built-in types would be better at some point. + * + * Returns 0 on success, -1 on error. + */ +NPY_NO_EXPORT int +PyUFunc_SimpleUnaryOperationTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata) +{ + int i, type_num, type_num1; + char *ufunc_name; + + ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; + + if (ufunc->nin != 1 || ufunc->nout != 1) { + PyErr_Format(PyExc_RuntimeError, "ufunc %s is configured " + "to use unary operation type resolution but has " + "the wrong number of inputs or outputs", + ufunc_name); + return -1; + } + + /* + * Use the default type resolution if there's a custom data type + * or object arrays. + */ + type_num1 = PyArray_DESCR(operands[0])->type_num; + if (type_num1 >= NPY_NTYPES || type_num1 == NPY_OBJECT) { + return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, + type_tup, out_dtypes, out_innerloop, out_innerloopdata); + } + + if (type_tup == NULL) { + /* Input types are the result type */ + out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + } + else { + /* + * If the type tuple isn't a single-element tuple, let the + * default type resolution handle this one. + */ + if (!PyTuple_Check(type_tup) || PyTuple_GET_SIZE(type_tup) != 1) { + return PyUFunc_DefaultTypeResolution(ufunc, casting, + operands, type_tup, out_dtypes, + out_innerloop, out_innerloopdata); + } + + if (!PyArray_DescrCheck(PyTuple_GET_ITEM(type_tup, 0))) { + PyErr_SetString(PyExc_ValueError, + "require data type in the type tuple"); + return -1; + } + + out_dtypes[0] = ensure_dtype_nbo( + (PyArray_Descr *)PyTuple_GET_ITEM(type_tup, 0)); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + } + + /* Check against the casting rules */ + if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { + for (i = 0; i < 2; ++i) { + Py_DECREF(out_dtypes[i]); + out_dtypes[i] = NULL; + } + return -1; + } + + type_num = out_dtypes[0]->type_num; + + /* If we have a built-in type, search in the functions list */ + if (type_num < NPY_NTYPES) { + char *types = ufunc->types; + int n = ufunc->ntypes; + + for (i = 0; i < n; ++i) { + if (types[2*i] == type_num) { + *out_innerloop = ufunc->functions[i]; + *out_innerloopdata = ufunc->data[i]; + return 0; + } + } + + PyErr_Format(PyExc_TypeError, + "ufunc '%s' not supported for the input types", + ufunc_name); + return -1; + } + else { + PyErr_SetString(PyExc_RuntimeError, + "user type shouldn't have resulted from type promotion"); + return -1; + } +} + +/* + * The ones_like function shouldn't really be a ufunc, but while it + * still is, this provides type resolution that always forces UNSAFE + * casting. + */ +NPY_NO_EXPORT int +PyUFunc_OnesLikeTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING NPY_UNUSED(casting), + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata) +{ + return PyUFunc_SimpleUnaryOperationTypeResolution(ufunc, + NPY_UNSAFE_CASTING, + operands, type_tup, out_dtypes, + out_innerloop, out_innerloopdata); +} + + +/* + * This function applies special type resolution rules for the case + * where all the functions have the pattern XX->X, using + * PyArray_ResultType instead of a linear search to get the best + * loop. + * + * Note that a simpler linear search through the functions loop + * is still done, but switching to a simple array lookup for + * built-in types would be better at some point. + * + * Returns 0 on success, -1 on error. + */ +NPY_NO_EXPORT int +PyUFunc_SimpleBinaryOperationTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata) +{ + int i, type_num, type_num1, type_num2; + char *ufunc_name; + + ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; + + if (ufunc->nin != 2 || ufunc->nout != 1) { + PyErr_Format(PyExc_RuntimeError, "ufunc %s is configured " + "to use binary operation type resolution but has " + "the wrong number of inputs or outputs", + ufunc_name); + return -1; + } + + /* + * Use the default type resolution if there's a custom data type + * or object arrays. + */ + type_num1 = PyArray_DESCR(operands[0])->type_num; + type_num2 = PyArray_DESCR(operands[1])->type_num; + if (type_num1 >= NPY_NTYPES || type_num2 >= NPY_NTYPES || + type_num1 == NPY_OBJECT || type_num2 == NPY_OBJECT) { + return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, + type_tup, out_dtypes, out_innerloop, out_innerloopdata); + } + + if (type_tup == NULL) { + /* Input types are the result type */ + out_dtypes[0] = PyArray_ResultType(2, operands, 0, NULL); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + } + else { + /* + * If the type tuple isn't a single-element tuple, let the + * default type resolution handle this one. + */ + if (!PyTuple_Check(type_tup) || PyTuple_GET_SIZE(type_tup) != 1) { + return PyUFunc_DefaultTypeResolution(ufunc, casting, + operands, type_tup, out_dtypes, + out_innerloop, out_innerloopdata); + } + + if (!PyArray_DescrCheck(PyTuple_GET_ITEM(type_tup, 0))) { + PyErr_SetString(PyExc_ValueError, + "require data type in the type tuple"); + return -1; + } + + out_dtypes[0] = ensure_dtype_nbo( + (PyArray_Descr *)PyTuple_GET_ITEM(type_tup, 0)); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + } + + /* Check against the casting rules */ + if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { + for (i = 0; i < 3; ++i) { + Py_DECREF(out_dtypes[i]); + out_dtypes[i] = NULL; + } + return -1; + } + + type_num = out_dtypes[0]->type_num; + + /* If we have a built-in type, search in the functions list */ + if (type_num < NPY_NTYPES) { + char *types = ufunc->types; + int n = ufunc->ntypes; + + for (i = 0; i < n; ++i) { + if (types[3*i] == type_num) { + *out_innerloop = ufunc->functions[i]; + *out_innerloopdata = ufunc->data[i]; + return 0; + } + } + + PyErr_Format(PyExc_TypeError, + "ufunc '%s' not supported for the input types", + ufunc_name); + return -1; + } + else { + PyErr_SetString(PyExc_RuntimeError, + "user type shouldn't have resulted from type promotion"); + return -1; + } +} + +/* + * This function applies special type resolution rules for the absolute + * ufunc. This ufunc converts complex -> float, so isn't covered + * by the simple unary type resolution. + * + * Returns 0 on success, -1 on error. + */ +NPY_NO_EXPORT int +PyUFunc_AbsoluteTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata) +{ + /* Use the default for complex types, to find the loop producing float */ + if (PyTypeNum_ISCOMPLEX(PyArray_DESCR(operands[0])->type_num)) { + return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, + type_tup, out_dtypes, out_innerloop, out_innerloopdata); + } + else { + return PyUFunc_SimpleUnaryOperationTypeResolution(ufunc, casting, + operands, type_tup, out_dtypes, + out_innerloop, out_innerloopdata); + } +} + + +/* + * This function returns the a new reference to the + * capsule with the datetime metadata. + * + * NOTE: This function is copied from datetime.c in multiarray, + * because umath and multiarray are not linked together. + */ +static PyObject * +get_datetime_metacobj_from_dtype(PyArray_Descr *dtype) +{ + PyObject *metacobj; + + /* Check that the dtype has metadata */ + if (dtype->metadata == NULL) { + PyErr_SetString(PyExc_TypeError, + "Datetime type object is invalid, lacks metadata"); + return NULL; + } + + /* Check that the dtype has unit metadata */ + metacobj = PyDict_GetItemString(dtype->metadata, NPY_METADATA_DTSTR); + if (metacobj == NULL) { + PyErr_SetString(PyExc_TypeError, + "Datetime type object is invalid, lacks unit metadata"); + return NULL; + } + + Py_INCREF(metacobj); + return metacobj; +} + +/* + * Creates a new NPY_TIMEDELTA dtype, copying the datetime metadata + * from the given dtype. + * + * NOTE: This function is copied from datetime.c in multiarray, + * because umath and multiarray are not linked together. + */ +static PyArray_Descr * +timedelta_dtype_with_copied_meta(PyArray_Descr *dtype) +{ + PyArray_Descr *ret; + PyObject *metacobj; + + ret = PyArray_DescrNewFromType(NPY_TIMEDELTA); + if (ret == NULL) { + return NULL; + } + Py_XDECREF(ret->metadata); + ret->metadata = PyDict_New(); + if (ret->metadata == NULL) { + Py_DECREF(ret); + return NULL; + } + + metacobj = get_datetime_metacobj_from_dtype(dtype); + if (metacobj == NULL) { + Py_DECREF(ret); + return NULL; + } + + if (PyDict_SetItemString(ret->metadata, NPY_METADATA_DTSTR, + metacobj) < 0) { + Py_DECREF(metacobj); + Py_DECREF(ret); + return NULL; + } + + return ret; +} + +/* + * This function applies the type resolution rules for addition. + * In particular, there are a number of special cases with datetime: + * m8[<A>] + m8[<B>] => m8[gcd(<A>,<B>)] + m8[gcd(<A>,<B>)] + * m8[<A>] + int => m8[<A>] + m8[<A>] + * int + m8[<A>] => m8[<A>] + m8[<A>] + * M8[<A>] + int => M8[<A>] + m8[<A>] + * int + M8[<A>] => m8[<A>] + M8[<A>] + * M8[<A>] + m8[<B>] => M8[gcd(<A>,<B>)] + m8[gcd(<A>,<B>)] + * m8[<A>] + M8[<B>] => m8[gcd(<A>,<B>)] + M8[gcd(<A>,<B>)] + * TODO: Non-linear time unit cases require highly special-cased loops + * M8[<A>] + m8[Y|M|B] + * m8[Y|M|B] + M8[<A>] + */ +NPY_NO_EXPORT int +PyUFunc_AdditionTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata) +{ + int type_num1, type_num2; + char *types; + int i, n; + char *ufunc_name; + + ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; + + type_num1 = PyArray_DESCR(operands[0])->type_num; + type_num2 = PyArray_DESCR(operands[1])->type_num; + + /* Use the default when datetime and timedelta are not involved */ + if (!PyTypeNum_ISDATETIME(type_num1) && !PyTypeNum_ISDATETIME(type_num2)) { + return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, + type_tup, out_dtypes, out_innerloop, out_innerloopdata); + } + + if (type_num1 == NPY_TIMEDELTA) { + /* m8[<A>] + m8[<B>] => m8[gcd(<A>,<B>)] + m8[gcd(<A>,<B>)] */ + if (type_num2 == NPY_TIMEDELTA) { + out_dtypes[0] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), + PyArray_DESCR(operands[1])); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + } + /* m8[<A>] + M8[<B>] => m8[gcd(<A>,<B>)] + M8[gcd(<A>,<B>)] */ + else if (type_num2 == NPY_DATETIME) { + out_dtypes[1] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), + PyArray_DESCR(operands[1])); + if (out_dtypes[1] == NULL) { + return -1; + } + /* Make a new NPY_TIMEDELTA, and copy the datetime's metadata */ + out_dtypes[0] = timedelta_dtype_with_copied_meta(out_dtypes[1]); + if (out_dtypes[0] == NULL) { + Py_DECREF(out_dtypes[1]); + out_dtypes[1] = NULL; + return -1; + } + out_dtypes[2] = out_dtypes[1]; + Py_INCREF(out_dtypes[2]); + } + /* m8[<A>] + int => m8[<A>] + m8[<A>] */ + else if (PyTypeNum_ISINTEGER(type_num2) || + PyTypeNum_ISBOOL(type_num2)) { + out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + + type_num2 = NPY_TIMEDELTA; + } + else { + goto type_reso_error; + } + } + else if (type_num1 == NPY_DATETIME) { + /* M8[<A>] + m8[<B>] => M8[gcd(<A>,<B>)] + m8[gcd(<A>,<B>)] */ + if (type_num2 == NPY_TIMEDELTA) { + out_dtypes[0] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), + PyArray_DESCR(operands[1])); + if (out_dtypes[0] == NULL) { + return -1; + } + /* Make a new NPY_TIMEDELTA, and copy the datetime's metadata */ + out_dtypes[1] = timedelta_dtype_with_copied_meta(out_dtypes[0]); + if (out_dtypes[1] == NULL) { + Py_DECREF(out_dtypes[0]); + out_dtypes[0] = NULL; + return -1; + } + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + } + /* M8[<A>] + int => M8[<A>] + m8[<A>] */ + else if (PyTypeNum_ISINTEGER(type_num2) || + PyTypeNum_ISBOOL(type_num2)) { + out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); + if (out_dtypes[0] == NULL) { + return -1; + } + /* Make a new NPY_TIMEDELTA, and copy type1's metadata */ + out_dtypes[1] = timedelta_dtype_with_copied_meta( + PyArray_DESCR(operands[0])); + if (out_dtypes[1] == NULL) { + Py_DECREF(out_dtypes[0]); + out_dtypes[0] = NULL; + return -1; + } + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + + type_num2 = NPY_TIMEDELTA; + } + else { + goto type_reso_error; + } + } + else if (PyTypeNum_ISINTEGER(type_num1) || PyTypeNum_ISBOOL(type_num1)) { + /* int + m8[<A>] => m8[<A>] + m8[<A>] */ + if (type_num2 == NPY_TIMEDELTA) { + out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[1])); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + + type_num1 = NPY_TIMEDELTA; + } + else if (type_num2 == NPY_DATETIME) { + /* Make a new NPY_TIMEDELTA, and copy type2's metadata */ + out_dtypes[0] = timedelta_dtype_with_copied_meta( + PyArray_DESCR(operands[1])); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = ensure_dtype_nbo(PyArray_DESCR(operands[1])); + if (out_dtypes[1] == NULL) { + Py_DECREF(out_dtypes[0]); + out_dtypes[0] = NULL; + return -1; + } + out_dtypes[2] = out_dtypes[1]; + Py_INCREF(out_dtypes[2]); + + type_num1 = NPY_TIMEDELTA; + } + else { + goto type_reso_error; + } + } + else { + goto type_reso_error; + } + + /* Check against the casting rules */ + if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { + for (i = 0; i < 3; ++i) { + Py_DECREF(out_dtypes[i]); + out_dtypes[i] = NULL; + } + return -1; + } + + /* Search in the functions list */ + types = ufunc->types; + n = ufunc->ntypes; + + for (i = 0; i < n; ++i) { + if (types[3*i] == type_num1 && types[3*i+1] == type_num2) { + *out_innerloop = ufunc->functions[i]; + *out_innerloopdata = ufunc->data[i]; + return 0; + } + } + + PyErr_Format(PyExc_TypeError, + "internal error: could not find appropriate datetime " + "inner loop in %s ufunc", ufunc_name); + return -1; + +type_reso_error: { + PyObject *errmsg; + errmsg = PyUString_FromFormat("ufunc %s cannot use operands " + "with types ", ufunc_name); + PyUString_ConcatAndDel(&errmsg, + PyObject_Repr((PyObject *)PyArray_DESCR(operands[0]))); + PyUString_ConcatAndDel(&errmsg, + PyUString_FromString(" and ")); + PyUString_ConcatAndDel(&errmsg, + PyObject_Repr((PyObject *)PyArray_DESCR(operands[1]))); + PyErr_SetObject(PyExc_TypeError, errmsg); + return -1; + } +} + +/* + * This function applies the type resolution rules for subtraction. + * In particular, there are a number of special cases with datetime: + * m8[<A>] - m8[<B>] => m8[gcd(<A>,<B>)] - m8[gcd(<A>,<B>)] + * m8[<A>] - int => m8[<A>] - m8[<A>] + * int - m8[<A>] => m8[<A>] - m8[<A>] + * M8[<A>] - int => M8[<A>] - m8[<A>] + * M8[<A>] - m8[<B>] => M8[gcd(<A>,<B>)] - m8[gcd(<A>,<B>)] + * TODO: Non-linear time unit cases require highly special-cased loops + * M8[<A>] - m8[Y|M|B] + */ +NPY_NO_EXPORT int +PyUFunc_SubtractionTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata) +{ + int type_num1, type_num2; + char *types; + int i, n; + char *ufunc_name; + + ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; + + type_num1 = PyArray_DESCR(operands[0])->type_num; + type_num2 = PyArray_DESCR(operands[1])->type_num; + + /* Use the default when datetime and timedelta are not involved */ + if (!PyTypeNum_ISDATETIME(type_num1) && !PyTypeNum_ISDATETIME(type_num2)) { + return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, + type_tup, out_dtypes, out_innerloop, out_innerloopdata); + } + + if (type_num1 == NPY_TIMEDELTA) { + /* m8[<A>] - m8[<B>] => m8[gcd(<A>,<B>)] - m8[gcd(<A>,<B>)] */ + if (type_num2 == NPY_TIMEDELTA) { + out_dtypes[0] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), + PyArray_DESCR(operands[1])); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + } + /* m8[<A>] - int => m8[<A>] - m8[<A>] */ + else if (PyTypeNum_ISINTEGER(type_num2) || + PyTypeNum_ISBOOL(type_num2)) { + out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + + type_num2 = NPY_TIMEDELTA; + } + else { + goto type_reso_error; + } + } + else if (type_num1 == NPY_DATETIME) { + /* M8[<A>] - m8[<B>] => M8[gcd(<A>,<B>)] - m8[gcd(<A>,<B>)] */ + if (type_num2 == NPY_TIMEDELTA) { + out_dtypes[0] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), + PyArray_DESCR(operands[1])); + if (out_dtypes[0] == NULL) { + return -1; + } + /* Make a new NPY_TIMEDELTA, and copy the datetime's metadata */ + out_dtypes[1] = timedelta_dtype_with_copied_meta(out_dtypes[0]); + if (out_dtypes[1] == NULL) { + Py_DECREF(out_dtypes[0]); + out_dtypes[0] = NULL; + return -1; + } + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + } + /* M8[<A>] - int => M8[<A>] - m8[<A>] */ + else if (PyTypeNum_ISINTEGER(type_num2) || + PyTypeNum_ISBOOL(type_num2)) { + out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); + if (out_dtypes[0] == NULL) { + return -1; + } + /* Make a new NPY_TIMEDELTA, and copy type1's metadata */ + out_dtypes[1] = timedelta_dtype_with_copied_meta( + PyArray_DESCR(operands[0])); + if (out_dtypes[1] == NULL) { + Py_DECREF(out_dtypes[0]); + out_dtypes[0] = NULL; + return -1; + } + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + + type_num2 = NPY_TIMEDELTA; + } + /* M8[<A>] - M8[<B>] => M8[gcd(<A>,<B>)] - M8[gcd(<A>,<B>)] */ + else if (type_num2 == NPY_DATETIME) { + out_dtypes[0] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), + PyArray_DESCR(operands[1])); + if (out_dtypes[0] == NULL) { + return -1; + } + /* Make a new NPY_TIMEDELTA, and copy type1's metadata */ + out_dtypes[2] = timedelta_dtype_with_copied_meta(out_dtypes[0]); + if (out_dtypes[2] == NULL) { + Py_DECREF(out_dtypes[0]); + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + } + else { + goto type_reso_error; + } + } + else if (PyTypeNum_ISINTEGER(type_num1) || PyTypeNum_ISBOOL(type_num1)) { + /* int - m8[<A>] => m8[<A>] - m8[<A>] */ + if (type_num2 == NPY_TIMEDELTA) { + out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[1])); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + + type_num1 = NPY_TIMEDELTA; + } + else { + goto type_reso_error; + } + } + else { + goto type_reso_error; + } + + /* Check against the casting rules */ + if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { + for (i = 0; i < 3; ++i) { + Py_DECREF(out_dtypes[i]); + out_dtypes[i] = NULL; + } + return -1; + } + + /* Search in the functions list */ + types = ufunc->types; + n = ufunc->ntypes; + + for (i = 0; i < n; ++i) { + if (types[3*i] == type_num1 && types[3*i+1] == type_num2) { + *out_innerloop = ufunc->functions[i]; + *out_innerloopdata = ufunc->data[i]; + return 0; + } + } + + PyErr_Format(PyExc_TypeError, + "internal error: could not find appropriate datetime " + "inner loop in %s ufunc", ufunc_name); + return -1; + +type_reso_error: { + PyObject *errmsg; + errmsg = PyUString_FromFormat("ufunc %s cannot use operands " + "with types ", ufunc_name); + PyUString_ConcatAndDel(&errmsg, + PyObject_Repr((PyObject *)PyArray_DESCR(operands[0]))); + PyUString_ConcatAndDel(&errmsg, + PyUString_FromString(" and ")); + PyUString_ConcatAndDel(&errmsg, + PyObject_Repr((PyObject *)PyArray_DESCR(operands[1]))); + PyErr_SetObject(PyExc_TypeError, errmsg); + return -1; + } +} + +/* + * This function applies the type resolution rules for multiplication. + * In particular, there are a number of special cases with datetime: + * int## * m8[<A>] => int64 * m8[<A>] + * m8[<A>] * int## => m8[<A>] * int64 + * float## * m8[<A>] => float64 * m8[<A>] + * m8[<A>] * float## => m8[<A>] * float64 + */ +NPY_NO_EXPORT int +PyUFunc_MultiplicationTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata) +{ + int type_num1, type_num2; + char *types; + int i, n; + char *ufunc_name; + + ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; + + type_num1 = PyArray_DESCR(operands[0])->type_num; + type_num2 = PyArray_DESCR(operands[1])->type_num; + + /* Use the default when datetime and timedelta are not involved */ + if (!PyTypeNum_ISDATETIME(type_num1) && !PyTypeNum_ISDATETIME(type_num2)) { + return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, + type_tup, out_dtypes, out_innerloop, out_innerloopdata); + } + + if (type_num1 == NPY_TIMEDELTA) { + /* m8[<A>] * int## => m8[<A>] * int64 */ + if (PyTypeNum_ISINTEGER(type_num2) || PyTypeNum_ISBOOL(type_num2)) { + out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = PyArray_DescrNewFromType(NPY_LONGLONG); + if (out_dtypes[1] == NULL) { + Py_DECREF(out_dtypes[0]); + out_dtypes[0] = NULL; + return -1; + } + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + + type_num2 = NPY_LONGLONG; + } + /* m8[<A>] * float## => m8[<A>] * float64 */ + else if (PyTypeNum_ISFLOAT(type_num2)) { + out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = PyArray_DescrNewFromType(NPY_DOUBLE); + if (out_dtypes[1] == NULL) { + Py_DECREF(out_dtypes[0]); + out_dtypes[0] = NULL; + return -1; + } + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + + type_num2 = NPY_DOUBLE; + } + else { + goto type_reso_error; + } + } + else if (PyTypeNum_ISINTEGER(type_num1) || PyTypeNum_ISBOOL(type_num1)) { + /* int## * m8[<A>] => int64 * m8[<A>] */ + if (type_num2 == NPY_TIMEDELTA) { + out_dtypes[0] = PyArray_DescrNewFromType(NPY_LONGLONG); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = ensure_dtype_nbo(PyArray_DESCR(operands[1])); + if (out_dtypes[1] == NULL) { + Py_DECREF(out_dtypes[0]); + out_dtypes[0] = NULL; + return -1; + } + out_dtypes[2] = out_dtypes[1]; + Py_INCREF(out_dtypes[2]); + + type_num1 = NPY_LONGLONG; + } + else { + goto type_reso_error; + } + } + else if (PyTypeNum_ISFLOAT(type_num1)) { + /* float## * m8[<A>] => float64 * m8[<A>] */ + if (type_num2 == NPY_TIMEDELTA) { + out_dtypes[0] = PyArray_DescrNewFromType(NPY_DOUBLE); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = ensure_dtype_nbo(PyArray_DESCR(operands[1])); + if (out_dtypes[1] == NULL) { + Py_DECREF(out_dtypes[0]); + out_dtypes[0] = NULL; + return -1; + } + out_dtypes[2] = out_dtypes[1]; + Py_INCREF(out_dtypes[2]); + + type_num1 = NPY_DOUBLE; + } + else { + goto type_reso_error; + } + } + else { + goto type_reso_error; + } + + /* Check against the casting rules */ + if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { + for (i = 0; i < 3; ++i) { + Py_DECREF(out_dtypes[i]); + out_dtypes[i] = NULL; + } + return -1; + } + + /* Search in the functions list */ + types = ufunc->types; + n = ufunc->ntypes; + + for (i = 0; i < n; ++i) { + if (types[3*i] == type_num1 && types[3*i+1] == type_num2) { + *out_innerloop = ufunc->functions[i]; + *out_innerloopdata = ufunc->data[i]; + return 0; + } + } + + PyErr_Format(PyExc_TypeError, + "internal error: could not find appropriate datetime " + "inner loop in %s ufunc", ufunc_name); + return -1; + +type_reso_error: { + PyObject *errmsg; + errmsg = PyUString_FromFormat("ufunc %s cannot use operands " + "with types ", ufunc_name); + PyUString_ConcatAndDel(&errmsg, + PyObject_Repr((PyObject *)PyArray_DESCR(operands[0]))); + PyUString_ConcatAndDel(&errmsg, + PyUString_FromString(" and ")); + PyUString_ConcatAndDel(&errmsg, + PyObject_Repr((PyObject *)PyArray_DESCR(operands[1]))); + PyErr_SetObject(PyExc_TypeError, errmsg); + return -1; + } +} + +/* + * This function applies the type resolution rules for division. + * In particular, there are a number of special cases with datetime: + * m8[<A>] / m8[<B>] to m8[gcd(<A>,<B>)] / m8[gcd(<A>,<B>)] -> float64 + * m8[<A>] / int## to m8[<A>] / int64 -> m8[<A>] + * m8[<A>] / float## to m8[<A>] / float64 -> m8[<A>] + */ +NPY_NO_EXPORT int +PyUFunc_DivisionTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata) +{ + int type_num1, type_num2; + char *types; + int i, n; + char *ufunc_name; + + ufunc_name = ufunc->name ? ufunc->name : "<unnamed ufunc>"; + + type_num1 = PyArray_DESCR(operands[0])->type_num; + type_num2 = PyArray_DESCR(operands[1])->type_num; + + /* Use the default when datetime and timedelta are not involved */ + if (!PyTypeNum_ISDATETIME(type_num1) && !PyTypeNum_ISDATETIME(type_num2)) { + return PyUFunc_DefaultTypeResolution(ufunc, casting, operands, + type_tup, out_dtypes, out_innerloop, out_innerloopdata); + } + + if (type_num1 == NPY_TIMEDELTA) { + /* + * m8[<A>] / m8[<B>] to + * m8[gcd(<A>,<B>)] / m8[gcd(<A>,<B>)] -> float64 + */ + if (type_num2 == NPY_TIMEDELTA) { + out_dtypes[0] = PyArray_PromoteTypes(PyArray_DESCR(operands[0]), + PyArray_DESCR(operands[1])); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = out_dtypes[0]; + Py_INCREF(out_dtypes[1]); + out_dtypes[2] = PyArray_DescrFromType(NPY_DOUBLE); + if (out_dtypes[2] == NULL) { + Py_DECREF(out_dtypes[0]); + out_dtypes[0] = NULL; + Py_DECREF(out_dtypes[1]); + out_dtypes[1] = NULL; + return -1; + } + } + /* m8[<A>] / int## => m8[<A>] / int64 */ + else if (PyTypeNum_ISINTEGER(type_num2)) { + out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = PyArray_DescrFromType(NPY_LONGLONG); + if (out_dtypes[1] == NULL) { + Py_DECREF(out_dtypes[0]); + out_dtypes[0] = NULL; + return -1; + } + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + + type_num2 = NPY_LONGLONG; + } + /* m8[<A>] / float## => m8[<A>] / float64 */ + else if (PyTypeNum_ISFLOAT(type_num2)) { + out_dtypes[0] = ensure_dtype_nbo(PyArray_DESCR(operands[0])); + if (out_dtypes[0] == NULL) { + return -1; + } + out_dtypes[1] = PyArray_DescrNewFromType(NPY_DOUBLE); + if (out_dtypes[1] == NULL) { + Py_DECREF(out_dtypes[0]); + out_dtypes[0] = NULL; + return -1; + } + out_dtypes[2] = out_dtypes[0]; + Py_INCREF(out_dtypes[2]); + + type_num2 = NPY_DOUBLE; + } + else { + goto type_reso_error; + } + } + else { + goto type_reso_error; + } + + /* Check against the casting rules */ + if (PyUFunc_ValidateCasting(ufunc, casting, operands, out_dtypes) < 0) { + for (i = 0; i < 3; ++i) { + Py_DECREF(out_dtypes[i]); + out_dtypes[i] = NULL; + } + return -1; + } + + /* Search in the functions list */ + types = ufunc->types; + n = ufunc->ntypes; + + for (i = 0; i < n; ++i) { + if (types[3*i] == type_num1 && types[3*i+1] == type_num2) { + *out_innerloop = ufunc->functions[i]; + *out_innerloopdata = ufunc->data[i]; + return 0; + } + } + + PyErr_Format(PyExc_TypeError, + "internal error: could not find appropriate datetime " + "inner loop in %s ufunc", ufunc_name); + return -1; + +type_reso_error: { + PyObject *errmsg; + errmsg = PyUString_FromFormat("ufunc %s cannot use operands " + "with types ", ufunc_name); + PyUString_ConcatAndDel(&errmsg, + PyObject_Repr((PyObject *)PyArray_DESCR(operands[0]))); + PyUString_ConcatAndDel(&errmsg, + PyUString_FromString(" and ")); + PyUString_ConcatAndDel(&errmsg, + PyObject_Repr((PyObject *)PyArray_DESCR(operands[1]))); + PyErr_SetObject(PyExc_TypeError, errmsg); + return -1; + } +} + +typedef struct { + NpyAuxData base; + PyUFuncGenericFunction unmasked_innerloop; + void *unmasked_innerloopdata; + int nargs; +} _ufunc_masker_data; + +static NpyAuxData * +ufunc_masker_data_clone(NpyAuxData *data) +{ + _ufunc_masker_data *n; + + /* Allocate a new one */ + n = (_ufunc_masker_data *)PyArray_malloc(sizeof(_ufunc_masker_data)); + if (n == NULL) { + return NULL; + } + + /* Copy the data (unmasked data doesn't have object semantics) */ + memcpy(n, data, sizeof(_ufunc_masker_data)); + + return (NpyAuxData *)n; +} + +/* + * This function wraps a regular unmasked ufunc inner loop as a + * masked ufunc inner loop, only calling the function for + * elements where the mask is True. + */ +static void +unmasked_ufunc_loop_as_masked( + char **args, + npy_intp *dimensions, + npy_intp *steps, + NpyAuxData *innerloopdata) +{ + _ufunc_masker_data *data; + int iargs, nargs; + PyUFuncGenericFunction unmasked_innerloop; + void *unmasked_innerloopdata; + npy_intp loopsize, subloopsize; + char *mask; + npy_intp maskstep; + + /* Put the aux data into local variables */ + data = (_ufunc_masker_data *)innerloopdata; + unmasked_innerloop = data->unmasked_innerloop; + unmasked_innerloopdata = data->unmasked_innerloopdata; + nargs = data->nargs; + loopsize = *dimensions; + mask = args[nargs]; + maskstep = steps[nargs]; + + + /* Process the data as runs of unmasked values */ + do { + /* Skip masked values */ + subloopsize = 0; + while (subloopsize < loopsize && *(npy_bool *)mask == 0) { + ++subloopsize; + mask += maskstep; + } + for (iargs = 0; iargs < nargs; ++iargs) { + args[iargs] += subloopsize * steps[iargs]; + } + loopsize -= subloopsize; + /* + * Process unmasked values (assumes unmasked loop doesn't + * mess with the 'args' pointer values) + */ + subloopsize = 0; + while (subloopsize < loopsize && *(npy_bool *)mask != 0) { + ++subloopsize; + mask += maskstep; + } + unmasked_innerloop(args, &subloopsize, steps, unmasked_innerloopdata); + for (iargs = 0; iargs < nargs; ++iargs) { + args[iargs] += subloopsize * steps[iargs]; + } + loopsize -= subloopsize; + } while (loopsize > 0); +} + + +/*UFUNC_API + * + * This function calls the unmasked type resolution function of the + * ufunc, then wraps it with a function which only calls the inner + * loop where the mask is True. + * + * Returns 0 on success, -1 on error. + */ +NPY_NO_EXPORT int +PyUFunc_DefaultTypeResolutionMasked(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericMaskedFunction *out_innerloop, + NpyAuxData **out_innerloopdata) +{ + int retcode; + _ufunc_masker_data *data; + + /* Create a new NpyAuxData object for the masker data */ + data = (_ufunc_masker_data *)PyArray_malloc(sizeof(_ufunc_masker_data)); + if (data == NULL) { + PyErr_NoMemory(); + return -1; + } + memset(data, 0, sizeof(_ufunc_masker_data)); + data->base.free = (NpyAuxData_FreeFunc *)&PyArray_free; + data->base.clone = &ufunc_masker_data_clone; + data->nargs = ufunc->nin + ufunc->nout; + + /* Get the unmasked ufunc inner loop */ + retcode = ufunc->type_resolution_function(ufunc, casting, + operands, type_tup, out_dtypes, + &data->unmasked_innerloop, &data->unmasked_innerloopdata); + if (retcode < 0) { + PyArray_free(data); + return retcode; + } + + /* Return the loop function + aux data */ + *out_innerloop = &unmasked_ufunc_loop_as_masked; + *out_innerloopdata = (NpyAuxData *)data; + return 0; +} + +static int +ufunc_loop_matches(PyUFuncObject *self, + PyArrayObject **op, + NPY_CASTING input_casting, + NPY_CASTING output_casting, + int any_object, + int use_min_scalar, + int *types, + int *out_no_castable_output, + char *out_err_src_typecode, + char *out_err_dst_typecode) +{ + npy_intp i, nin = self->nin, nop = nin + self->nout; + + /* + * First check if all the inputs can be safely cast + * to the types for this function + */ + for (i = 0; i < nin; ++i) { + PyArray_Descr *tmp; + + /* + * If no inputs are objects and there are more than one + * loop, don't allow conversion to object. The rationale + * behind this is mostly performance. Except for custom + * ufuncs built with just one object-parametered inner loop, + * only the types that are supported are implemented. Trying + * the object version of logical_or on float arguments doesn't + * seem right. + */ + if (types[i] == NPY_OBJECT && !any_object && self->ntypes > 1) { + return 0; + } + + tmp = PyArray_DescrFromType(types[i]); + if (tmp == NULL) { + return -1; + } + +#if NPY_UF_DBG_TRACING + printf("Checking type for op %d, type %d: ", (int)i, (int)types[i]); + PyObject_Print((PyObject *)tmp, stdout, 0); + printf(", operand type: "); + PyObject_Print((PyObject *)PyArray_DESCR(op[i]), stdout, 0); + printf("\n"); +#endif + /* + * If all the inputs are scalars, use the regular + * promotion rules, not the special value-checking ones. + */ + if (!use_min_scalar) { + if (!PyArray_CanCastTypeTo(PyArray_DESCR(op[i]), tmp, + input_casting)) { + Py_DECREF(tmp); + return 0; + } + } + else { + if (!PyArray_CanCastArrayTo(op[i], tmp, input_casting)) { + Py_DECREF(tmp); + return 0; + } + } + Py_DECREF(tmp); + } + + /* + * If all the inputs were ok, then check casting back to the + * outputs. + */ + for (i = nin; i < nop; ++i) { + if (op[i] != NULL) { + PyArray_Descr *tmp = PyArray_DescrFromType(types[i]); + if (tmp == NULL) { + return -1; + } + if (!PyArray_CanCastTypeTo(tmp, PyArray_DESCR(op[i]), + output_casting)) { + if (!(*out_no_castable_output)) { + *out_no_castable_output = 1; + *out_err_src_typecode = tmp->type; + *out_err_dst_typecode = PyArray_DESCR(op[i])->type; + } + Py_DECREF(tmp); + return 0; + } + Py_DECREF(tmp); + } + } + + return 1; +} + +static int +set_ufunc_loop_data_types(PyUFuncObject *self, PyArrayObject **op, + PyArray_Descr **out_dtype, + int *types) +{ + int i, nin = self->nin, nop = nin + self->nout; + + /* Fill the dtypes array */ + for (i = 0; i < nop; ++i) { + out_dtype[i] = PyArray_DescrFromType(types[i]); + if (out_dtype[i] == NULL) { + while (--i >= 0) { + Py_DECREF(out_dtype[i]); + out_dtype[i] = NULL; + } + return -1; + } + } + + return 0; +} + +/* + * Does a search through the arguments and the loops + */ +static int +find_ufunc_matching_userloop(PyUFuncObject *self, + PyArrayObject **op, + NPY_CASTING input_casting, + NPY_CASTING output_casting, + int any_object, + int use_min_scalar, + PyArray_Descr **out_dtype, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata, + int *out_no_castable_output, + char *out_err_src_typecode, + char *out_err_dst_typecode) +{ + npy_intp i, nin = self->nin; + PyUFunc_Loop1d *funcdata; + + /* Use this to try to avoid repeating the same userdef loop search */ + int last_userdef = -1; + + for (i = 0; i < nin; ++i) { + int type_num = PyArray_DESCR(op[i])->type_num; + if (type_num != last_userdef && PyTypeNum_ISUSERDEF(type_num)) { + PyObject *key, *obj; + + last_userdef = type_num; + + key = PyInt_FromLong(type_num); + if (key == NULL) { + return -1; + } + obj = PyDict_GetItem(self->userloops, key); + Py_DECREF(key); + if (obj == NULL) { + continue; + } + funcdata = (PyUFunc_Loop1d *)NpyCapsule_AsVoidPtr(obj); + while (funcdata != NULL) { + int *types = funcdata->arg_types; + switch (ufunc_loop_matches(self, op, + input_casting, output_casting, + any_object, use_min_scalar, + types, + out_no_castable_output, out_err_src_typecode, + out_err_dst_typecode)) { + /* Error */ + case -1: + return -1; + /* Found a match */ + case 1: + set_ufunc_loop_data_types(self, op, out_dtype, types); + + /* Save the inner loop and its data */ + *out_innerloop = funcdata->func; + *out_innerloopdata = funcdata->data; + + return 0; + } + + funcdata = funcdata->next; + } + } + } + + /* Didn't find a match */ + return 0; +} + +/* + * Does a search through the arguments and the loops + */ +static int +find_ufunc_specified_userloop(PyUFuncObject *self, + int n_specified, + int *specified_types, + PyArrayObject **op, + NPY_CASTING casting, + int any_object, + int use_min_scalar, + PyArray_Descr **out_dtype, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata) +{ + int i, j, nin = self->nin, nop = nin + self->nout; + PyUFunc_Loop1d *funcdata; + + /* Use this to try to avoid repeating the same userdef loop search */ + int last_userdef = -1; + + int no_castable_output = 0; + char err_src_typecode = '-', err_dst_typecode = '-'; + + for (i = 0; i < nin; ++i) { + int type_num = PyArray_DESCR(op[i])->type_num; + if (type_num != last_userdef && PyTypeNum_ISUSERDEF(type_num)) { + PyObject *key, *obj; + + last_userdef = type_num; + + key = PyInt_FromLong(type_num); + if (key == NULL) { + return -1; + } + obj = PyDict_GetItem(self->userloops, key); + Py_DECREF(key); + if (obj == NULL) { + continue; + } + funcdata = (PyUFunc_Loop1d *)NpyCapsule_AsVoidPtr(obj); + while (funcdata != NULL) { + int *types = funcdata->arg_types; + int matched = 1; + + if (n_specified == nop) { + for (j = 0; j < nop; ++j) { + if (types[j] != specified_types[j]) { + matched = 0; + break; + } + } + } else { + if (types[nin] != specified_types[0]) { + matched = 0; + } + } + if (!matched) { + continue; + } + + switch (ufunc_loop_matches(self, op, + casting, casting, + any_object, use_min_scalar, + types, + &no_castable_output, &err_src_typecode, + &err_dst_typecode)) { + /* It works */ + case 1: + set_ufunc_loop_data_types(self, op, out_dtype, types); + + /* Save the inner loop and its data */ + *out_innerloop = funcdata->func; + *out_innerloopdata = funcdata->data; + + return 0; + /* Didn't match */ + case 0: + PyErr_Format(PyExc_TypeError, + "found a user loop for ufunc '%s' " + "matching the type-tuple, " + "but the inputs and/or outputs could not be " + "cast according to the casting rule", + self->name ? self->name : "(unknown)"); + return -1; + /* Error */ + case -1: + return -1; + } + + funcdata = funcdata->next; + } + } + } + + /* Didn't find a match */ + return 0; +} + +/* + * Provides an ordering for the dtype 'kind' character codes, to help + * determine when to use the min_scalar_type function. This groups + * 'kind' into boolean, integer, floating point, and everything else. + */ + +static int +dtype_kind_to_simplified_ordering(char kind) +{ + switch (kind) { + /* Boolean kind */ + case 'b': + return 0; + /* Unsigned int kind */ + case 'u': + /* Signed int kind */ + case 'i': + return 1; + /* Float kind */ + case 'f': + /* Complex kind */ + case 'c': + return 2; + /* Anything else */ + default: + return 3; + } +} + +static int +should_use_min_scalar(PyArrayObject **op, int nop) +{ + int i, use_min_scalar, kind; + int all_scalars = 1, max_scalar_kind = -1, max_array_kind = -1; + + /* + * Determine if there are any scalars, and if so, whether + * the maximum "kind" of the scalars surpasses the maximum + * "kind" of the arrays + */ + use_min_scalar = 0; + if (nop > 1) { + for(i = 0; i < nop; ++i) { + kind = dtype_kind_to_simplified_ordering( + PyArray_DESCR(op[i])->kind); + if (PyArray_NDIM(op[i]) == 0) { + if (kind > max_scalar_kind) { + max_scalar_kind = kind; + } + } + else { + all_scalars = 0; + if (kind > max_array_kind) { + max_array_kind = kind; + } + + } + } + + /* Indicate whether to use the min_scalar_type function */ + if (!all_scalars && max_array_kind >= max_scalar_kind) { + use_min_scalar = 1; + } + } + + return use_min_scalar; +} + +/* + * Does a linear search for the best inner loop of the ufunc. + * + * Note that if an error is returned, the caller must free the non-zero + * references in out_dtype. This function does not do its own clean-up. + */ +NPY_NO_EXPORT int +find_best_ufunc_inner_loop(PyUFuncObject *self, + PyArrayObject **op, + NPY_CASTING input_casting, + NPY_CASTING output_casting, + int any_object, + PyArray_Descr **out_dtype, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata) +{ + npy_intp i, j, nin = self->nin, nop = nin + self->nout; + int types[NPY_MAXARGS]; + char *ufunc_name; + int no_castable_output, use_min_scalar; + + /* For making a better error message on coercion error */ + char err_dst_typecode = '-', err_src_typecode = '-'; + + ufunc_name = self->name ? self->name : "(unknown)"; + + use_min_scalar = should_use_min_scalar(op, nin); + + /* If the ufunc has userloops, search for them. */ + if (self->userloops) { + switch (find_ufunc_matching_userloop(self, op, + input_casting, output_casting, + any_object, use_min_scalar, + out_dtype, out_innerloop, out_innerloopdata, + &no_castable_output, &err_src_typecode, + &err_dst_typecode)) { + /* Error */ + case -1: + return -1; + /* A loop was found */ + case 1: + return 0; + } + } + + /* + * Determine the UFunc loop. This could in general be *much* faster, + * and a better way to implement it might be for the ufunc to + * provide a function which gives back the result type and inner + * loop function. + * + * A default fast mechanism could be provided for functions which + * follow the most typical pattern, when all functions have signatures + * "xx...x -> x" for some built-in data type x, as follows. + * - Use PyArray_ResultType to get the output type + * - Look up the inner loop in a table based on the output type_num + * + * The method for finding the loop in the previous code did not + * appear consistent (as noted by some asymmetry in the generated + * coercion tables for np.add). + */ + no_castable_output = 0; + for (i = 0; i < self->ntypes; ++i) { + char *orig_types = self->types + i*self->nargs; + + /* Copy the types into an int array for matching */ + for (j = 0; j < nop; ++j) { + types[j] = orig_types[j]; + } + + switch (ufunc_loop_matches(self, op, + input_casting, output_casting, + any_object, use_min_scalar, + types, + &no_castable_output, &err_src_typecode, + &err_dst_typecode)) { + /* Error */ + case -1: + return -1; + /* Found a match */ + case 1: + set_ufunc_loop_data_types(self, op, out_dtype, types); + + /* Save the inner loop and its data */ + *out_innerloop = self->functions[i]; + *out_innerloopdata = self->data[i]; + + return 0; + } + + } + + /* If no function was found, throw an error */ + if (no_castable_output) { + PyErr_Format(PyExc_TypeError, + "ufunc '%s' output (typecode '%c') could not be coerced to " + "provided output parameter (typecode '%c') according " + "to the casting rule '%s'", + ufunc_name, err_src_typecode, err_dst_typecode, + npy_casting_to_string(output_casting)); + } + else { + /* + * TODO: We should try again if the casting rule is same_kind + * or unsafe, and look for a function more liberally. + */ + PyErr_Format(PyExc_TypeError, + "ufunc '%s' not supported for the input types, and the " + "inputs could not be safely coerced to any supported " + "types according to the casting rule '%s'", + ufunc_name, + npy_casting_to_string(input_casting)); + } + + return -1; +} + +/* + * Does a linear search for the inner loop of the ufunc specified by type_tup. + * + * Note that if an error is returned, the caller must free the non-zero + * references in out_dtype. This function does not do its own clean-up. + */ +NPY_NO_EXPORT int +find_specified_ufunc_inner_loop(PyUFuncObject *self, + PyObject *type_tup, + PyArrayObject **op, + NPY_CASTING casting, + int any_object, + PyArray_Descr **out_dtype, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata) +{ + npy_intp i, j, n, nin = self->nin, nop = nin + self->nout; + int n_specified = 0; + int specified_types[NPY_MAXARGS], types[NPY_MAXARGS]; + char *ufunc_name; + int no_castable_output, use_min_scalar; + + /* For making a better error message on coercion error */ + char err_dst_typecode = '-', err_src_typecode = '-'; + + ufunc_name = self->name ? self->name : "(unknown)"; + + use_min_scalar = should_use_min_scalar(op, nin); + + /* Fill in specified_types from the tuple or string */ + if (PyTuple_Check(type_tup)) { + n = PyTuple_GET_SIZE(type_tup); + if (n != 1 && n != nop) { + PyErr_Format(PyExc_ValueError, + "a type-tuple must be specified " \ + "of length 1 or %d for ufunc '%s'", (int)nop, + self->name ? self->name : "(unknown)"); + return -1; + } + + for (i = 0; i < n; ++i) { + PyArray_Descr *dtype = NULL; + if (!PyArray_DescrConverter(PyTuple_GET_ITEM(type_tup, i), + &dtype)) { + return -1; + } + specified_types[i] = dtype->type_num; + Py_DECREF(dtype); + } + + n_specified = n; + } + else if (PyBytes_Check(type_tup) || PyUnicode_Check(type_tup)) { + Py_ssize_t length; + char *str; + PyObject *str_obj = NULL; + + if (PyUnicode_Check(type_tup)) { + str_obj = PyUnicode_AsASCIIString(type_tup); + if (str_obj == NULL) { + return -1; + } + type_tup = str_obj; + } + + if (!PyBytes_AsStringAndSize(type_tup, &str, &length) < 0) { + Py_XDECREF(str_obj); + return -1; + } + if (length != 1 && (length != nop + 2 || + str[nin] != '-' || str[nin+1] != '>')) { + PyErr_Format(PyExc_ValueError, + "a type-string for %s, " \ + "requires 1 typecode, or " + "%d typecode(s) before " \ + "and %d after the -> sign", + self->name ? self->name : "(unknown)", + self->nin, self->nout); + Py_XDECREF(str_obj); + return -1; + } + if (length == 1) { + PyArray_Descr *dtype; + n_specified = 1; + dtype = PyArray_DescrFromType(str[0]); + if (dtype == NULL) { + Py_XDECREF(str_obj); + return -1; + } + specified_types[0] = dtype->type_num; + Py_DECREF(dtype); + } + else { + PyArray_Descr *dtype; + n_specified = (int)nop; + + for (i = 0; i < nop; ++i) { + npy_intp istr = i < nin ? i : i+2; + + dtype = PyArray_DescrFromType(str[istr]); + if (dtype == NULL) { + Py_XDECREF(str_obj); + return -1; + } + specified_types[i] = dtype->type_num; + Py_DECREF(dtype); + } + } + Py_XDECREF(str_obj); + } + + /* If the ufunc has userloops, search for them. */ + if (self->userloops) { + switch (find_ufunc_specified_userloop(self, + n_specified, specified_types, + op, casting, + any_object, use_min_scalar, + out_dtype, out_innerloop, out_innerloopdata)) { + /* Error */ + case -1: + return -1; + /* Found matching loop */ + case 1: + return 0; + } + } + + for (i = 0; i < self->ntypes; ++i) { + char *orig_types = self->types + i*self->nargs; + int matched = 1; + + /* Copy the types into an int array for matching */ + for (j = 0; j < nop; ++j) { + types[j] = orig_types[j]; + } + + if (n_specified == nop) { + for (j = 0; j < nop; ++j) { + if (types[j] != specified_types[j]) { + matched = 0; + break; + } + } + } else { + if (types[nin] != specified_types[0]) { + matched = 0; + } + } + if (!matched) { + continue; + } + + switch (ufunc_loop_matches(self, op, + casting, casting, + any_object, use_min_scalar, + types, + &no_castable_output, &err_src_typecode, + &err_dst_typecode)) { + /* Error */ + case -1: + return -1; + /* It worked */ + case 1: + set_ufunc_loop_data_types(self, op, out_dtype, types); + + /* Save the inner loop and its data */ + *out_innerloop = self->functions[i]; + *out_innerloopdata = self->data[i]; + + return 0; + /* Didn't work */ + case 0: + PyErr_Format(PyExc_TypeError, + "found a loop for ufunc '%s' " + "matching the type-tuple, " + "but the inputs and/or outputs could not be " + "cast according to the casting rule", + ufunc_name); + return -1; + } + + } + + /* If no function was found, throw an error */ + PyErr_Format(PyExc_TypeError, + "No loop matching the specified signature was found " + "for ufunc %s", ufunc_name); + + return -1; +} + + diff --git a/numpy/core/src/umath/ufunc_type_resolution.h b/numpy/core/src/umath/ufunc_type_resolution.h new file mode 100644 index 000000000..f1ded2e9b --- /dev/null +++ b/numpy/core/src/umath/ufunc_type_resolution.h @@ -0,0 +1,116 @@ +#ifndef _NPY_PRIVATE__UFUNC_TYPE_RESOLUTION_H_ +#define _NPY_PRIVATE__UFUNC_TYPE_RESOLUTION_H_ + +NPY_NO_EXPORT int +PyUFunc_SimpleBinaryComparisonTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata); + +NPY_NO_EXPORT int +PyUFunc_SimpleUnaryOperationTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata); + +NPY_NO_EXPORT int +PyUFunc_OnesLikeTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata); + +NPY_NO_EXPORT int +PyUFunc_SimpleBinaryOperationTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata); + +NPY_NO_EXPORT int +PyUFunc_AbsoluteTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata); + +NPY_NO_EXPORT int +PyUFunc_AdditionTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata); + +NPY_NO_EXPORT int +PyUFunc_SubtractionTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata); + +NPY_NO_EXPORT int +PyUFunc_MultiplicationTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata); +NPY_NO_EXPORT int +PyUFunc_DivisionTypeResolution(PyUFuncObject *ufunc, + NPY_CASTING casting, + PyArrayObject **operands, + PyObject *type_tup, + PyArray_Descr **out_dtypes, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata); + +/* + * Does a linear search for the best inner loop of the ufunc. + * + * Note that if an error is returned, the caller must free the non-zero + * references in out_dtype. This function does not do its own clean-up. + */ +NPY_NO_EXPORT int +find_best_ufunc_inner_loop(PyUFuncObject *self, + PyArrayObject **op, + NPY_CASTING input_casting, + NPY_CASTING output_casting, + int any_object, + PyArray_Descr **out_dtype, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata); + +/* + * Does a linear search for the inner loop of the ufunc specified by type_tup. + * + * Note that if an error is returned, the caller must free the non-zero + * references in out_dtype. This function does not do its own clean-up. + */ +NPY_NO_EXPORT int +find_specified_ufunc_inner_loop(PyUFuncObject *self, + PyObject *type_tup, + PyArrayObject **op, + NPY_CASTING casting, + int any_object, + PyArray_Descr **out_dtype, + PyUFuncGenericFunction *out_innerloop, + void **out_innerloopdata); + +#endif diff --git a/numpy/core/src/umath/umathmodule.c.src b/numpy/core/src/umath/umathmodule.c.src index 82afd7fd8..52dcd4c1b 100644 --- a/numpy/core/src/umath/umathmodule.c.src +++ b/numpy/core/src/umath/umathmodule.c.src @@ -39,6 +39,7 @@ #include "funcs.inc" #include "loops.h" #include "ufunc_object.h" +#include "ufunc_type_resolution.h" #include "__umath_generated.c" #include "__ufunc_api.c" diff --git a/numpy/core/src/umath/umathmodule_onefile.c b/numpy/core/src/umath/umathmodule_onefile.c index 722f74eec..2255daf76 100644 --- a/numpy/core/src/umath/umathmodule_onefile.c +++ b/numpy/core/src/umath/umathmodule_onefile.c @@ -1,4 +1,5 @@ #include "loops.c" #include "ufunc_object.c" +#include "ufunc_type_resolution.c" #include "umathmodule.c" diff --git a/numpy/core/tests/test_ufunc.py b/numpy/core/tests/test_ufunc.py index 3f6c8e0d2..a7af19486 100644 --- a/numpy/core/tests/test_ufunc.py +++ b/numpy/core/tests/test_ufunc.py @@ -484,6 +484,32 @@ class TestUfunc(TestCase): np.subtract(a, 0, out=b) assert_equal(b, 0) + def test_where_param(self): + # Test that the where= ufunc parameter works with regular arrays + a = np.arange(7) + b = np.ones(7) + c = np.zeros(7) + np.add(a, b, out=c, where=(a % 2 == 1)) + assert_equal(c, [0,2,0,4,0,6,0]) + + a = np.arange(4).reshape(2,2) + 2 + np.power(a, [2,3], out=a, where=[[0,1],[1,0]]) + assert_equal(a, [[2, 27], [16, 5]]) + # Broadcasting the where= parameter + np.subtract(a, 2, out=a, where=[True,False]) + assert_equal(a, [[0, 27], [14, 5]]) + + @dec.knownfailureif(True) + def test_where_param_buffer_output(self): + # This test is temporarily skipped because it requires + # adding masking features to the nditer to work properly + + # With casting on output + a = np.ones(10, np.int64) + b = np.ones(10, np.int64) + c = 1.5 * np.ones(10, np.float64) + np.add(a, b, out=c, where=[1,0,0,1,0,0,1,1,1,0]) + assert_equal(c, [2,1.5,1.5,2,1.5,1.5,2,2,2,1.5]) if __name__ == "__main__": run_module_suite() diff --git a/numpy/ma/tests/test_core.py b/numpy/ma/tests/test_core.py index b4ec3e540..d82f3bd81 100644 --- a/numpy/ma/tests/test_core.py +++ b/numpy/ma/tests/test_core.py @@ -1700,9 +1700,9 @@ class TestMaskedArrayInPlaceArithmetics(TestCase): x = arange(10) * 2 xm = arange(10) * 2 xm[2] = masked - x /= 2 + x //= 2 assert_equal(x, y) - xm /= 2 + xm //= 2 assert_equal(xm, y) def test_inplace_division_scalar_float(self): diff --git a/numpy/ma/tests/test_old_ma.py b/numpy/ma/tests/test_old_ma.py index 0f58741fa..656f0b318 100644 --- a/numpy/ma/tests/test_old_ma.py +++ b/numpy/ma/tests/test_old_ma.py @@ -486,9 +486,9 @@ class TestMa(TestCase): x = arange(10) * 2 xm = arange(10) xm[2] = masked - x /= 2 + x //= 2 assert_(eq(x, y)) - xm /= 2 + xm //= 2 assert_(eq(x, y)) x = arange(10) * 1.0 |