BLD: introduce use of BLAS_LIBS and LAPACK_LIBS in distutils/system_info (#18737)

This enables the behaviour of the build honouring the environment
variables NPY_BLAS_LIBS, NPY_CBLAS_LIBS, and NPY_LAPACK_LIBS
to select any standard-conforming implementation of the f77
interfaces.

This is needed to sanely have automated builds provide BLAS libs
in a way common to multiple packages (using some environment variable,
BLAS_LIBS being canonical, but variants exist). Hacking a site.cfg
is fine for a single user, but does not scale well. As example,
pkgsrc, the NetBSD packages collection, offers differing builds
of Netlib or OpenBLAS, the latter in variables openblas,
openblas_openmp, and openblas_pthread. With this patch, differing
builds can just use different variable values like

NPY_BLAS_LIBS=-lblas NPY_CBLAS_LIBS=-lcblas \
  NPY_LAPACK_LIBS='-lapack -lblas' \
  python setup.py build

NPY_BLAS_LIBS=-lopenblas_pthread NPY_CBLAS_LIBS= \
  NPY_LAPACK_LIBS='-lopenblas_pthread' \
  python setup.py build

(NPY_LAPACK_LIBS contains reference to BLAS, too, for packages that only
use LAPACK and would miss underlying BLAS otherwise for static linking.)

3 files changed, 74 insertions, 2 deletions

diff --git a/doc/release/upcoming_changes/18737.new_feature.rst b/doc/release/upcoming_changes/18737.new_feature.rst
new file mode 100644
index 000000000..e451ac90a
--- /dev/null
+++ b/doc/release/upcoming_changes/18737.new_feature.rst
@@ -0,0 +1,12 @@
+BLAS and LAPACK configuration via environment variables
+-------------------------------------------------------
+Autodetection of installed BLAS and LAPACK libraries can be bypassed by using
+the ``NPY_BLAS_LIBS`` and ``NPY_LAPACK_LIBS`` environment variables. Instead,
+the link flags in these environment variables will be used directly, and the
+language is assumed to be F77.  This is especially useful in automated builds
+where the BLAS and LAPACK that are installed are known exactly.  A use case is
+replacing the actual implementation at runtime via stub library links.
+
+If ``NPY_CBLAS_LIBS`` is set (optional in addition to ``NPY_BLAS_LIBS``), this
+will be used as well, by defining ``HAVE_CBLAS`` and appending the environment
+variable content to the link flags.
diff --git a/doc/source/user/building.rst b/doc/source/user/building.rst
index 47399139e..52d7330bf 100644
--- a/doc/source/user/building.rst
+++ b/doc/source/user/building.rst
@@ -6,7 +6,7 @@ Building from source
 A general overview of building NumPy from source is given here, with detailed
 instructions for specific platforms given separately.
 
-.. 
+..
   This page is referenced from numpy/numpy/__init__.py. Please keep its
   location in sync with the link there.
 
@@ -123,6 +123,9 @@ in the ``site.cfg.example`` file.
 BLAS
 ~~~~
 
+Note that both BLAS and CBLAS interfaces are needed for a properly
+optimized build of NumPy.
+
 The default order for the libraries are:
 
 1. MKL
@@ -131,6 +134,12 @@ The default order for the libraries are:
 4. ATLAS
 5. BLAS (NetLIB)
 
+The detection of BLAS libraries may be bypassed by defining the environment
+variable ``NPY_BLAS_LIBS`` , which should contain the exact linker flags you
+want to use (interface is assumed to be Fortran 77).  Also define
+``NPY_CBLAS_LIBS`` (even empty if CBLAS is contained in your BLAS library) to
+trigger use of CBLAS and avoid slow fallback code for matrix calculations.
+
 If you wish to build against OpenBLAS but you also have BLIS available one
 may predefine the order of searching via the environment variable
 ``NPY_BLAS_ORDER`` which is a comma-separated list of the above names which
@@ -163,6 +172,9 @@ The default order for the libraries are:
 4. ATLAS
 5. LAPACK (NetLIB)
 
+The detection of LAPACK libraries may be bypassed by defining the environment
+variable ``NPY_LAPACK_LIBS``, which should contain the exact linker flags you
+want to use (language is assumed to be Fortran 77).
 
 If you wish to build against OpenBLAS but you also have MKL available one
 may predefine the order of searching via the environment variable
@@ -185,7 +197,6 @@ list is retained.
 One cannot mix negation and positives, nor have multiple negations, such cases will
 raise an error.
 
-
 .. deprecated:: 1.20
   The native libraries on macOS, provided by Accelerate, are not fit for use
   in NumPy since they have bugs that cause wrong output under easily reproducible
diff --git a/numpy/distutils/system_info.py b/numpy/distutils/system_info.py
index ae6f83e12..9e192329f 100644
--- a/numpy/distutils/system_info.py
+++ b/numpy/distutils/system_info.py
@@ -114,6 +114,19 @@ Currently, the following classes are available, along with their section names:
     x11_info:x11
     xft_info:xft
 
+Note that blas_opt_info and lapack_opt_info honor the NPY_BLAS_ORDER
+and NPY_LAPACK_ORDER environment variables to determine the order in which
+specific BLAS and LAPACK libraries are searched for.
+
+This search (or autodetection) can be bypassed by defining the environment
+variables NPY_BLAS_LIBS and NPY_LAPACK_LIBS, which should then contain the
+exact linker flags to use (language will be set to F77). Building against
+Netlib BLAS/LAPACK or stub files, in order to be able to switch BLAS and LAPACK
+implementations at runtime. If using this to build NumPy itself, it is
+recommended to also define NPY_CBLAS_LIBS (assuming your BLAS library has a
+CBLAS interface) to enable CBLAS usage for matrix multiplication (unoptimized
+otherwise).
+
 Example:
 ----------
 [DEFAULT]
@@ -1847,6 +1860,16 @@ class lapack_opt_info(system_info):
             return True
         return False
 
+    def _calc_info_from_envvar(self):
+        info = {}
+        info['language'] = 'f77'
+        info['libraries'] = []
+        info['include_dirs'] = []
+        info['define_macros'] = []
+        info['extra_link_args'] = os.environ['NPY_LAPACK_LIBS'].split()
+        self.set_info(**info)
+        return True
+
     def _calc_info(self, name):
         return getattr(self, '_calc_info_{}'.format(name))()
 
@@ -1857,6 +1880,12 @@ class lapack_opt_info(system_info):
                              "LAPACK order has unacceptable "
                              "values: {}".format(unknown_order))
 
+        if 'NPY_LAPACK_LIBS' in os.environ:
+            # Bypass autodetection, set language to F77 and use env var linker
+            # flags directly
+            self._calc_info_from_envvar()
+            return
+
         for lapack in lapack_order:
             if self._calc_info(lapack):
                 return
@@ -1979,6 +2008,20 @@ class blas_opt_info(system_info):
         self.set_info(**info)
         return True
 
+    def _calc_info_from_envvar(self):
+        info = {}
+        info['language'] = 'f77'
+        info['libraries'] = []
+        info['include_dirs'] = []
+        info['define_macros'] = []
+        info['extra_link_args'] = os.environ['NPY_BLAS_LIBS'].split()
+        if 'NPY_CBLAS_LIBS' in os.environ:
+            info['define_macros'].append(('HAVE_CBLAS', None))
+            info['extra_link_args'].extend(
+                                        os.environ['NPY_CBLAS_LIBS'].split())
+        self.set_info(**info)
+        return True
+
     def _calc_info(self, name):
         return getattr(self, '_calc_info_{}'.format(name))()
 
@@ -1987,6 +2030,12 @@ class blas_opt_info(system_info):
         if len(unknown_order) > 0:
             raise ValueError("blas_opt_info user defined BLAS order has unacceptable values: {}".format(unknown_order))
 
+        if 'NPY_BLAS_LIBS' in os.environ:
+            # Bypass autodetection, set language to F77 and use env var linker
+            # flags directly
+            self._calc_info_from_envvar()
+            return
+
         for blas in blas_order:
             if self._calc_info(blas):
                 return