![Alexandre PAYEN](/assets/img/avatar_default.png)
The numpy build system attempts to find lapack/blas at build time. It tries a lot of different implementations, e.g. lapack, openblas, atlas, ... It is possible to help this automatic discovery by specifying libraries to load in site.cfg and/or by setting environment variables BLAS and LAPACK. Unfortunately, the build system's logic is really hard to understand and it's fragile. For example, regardless of what is specified as libraries to load, it *will* try to find libblas.so and liblapack.so. However, when something is specified explicitly in site.cfg, it will use a different code path. It turns out that when we specified the blas and lapack libraries explicitly, as is done now, the build system logic will assume (without checking) that cblas is used. This causes calls to cblas_* to be linked in - again without checking, because numpy contains a copy of the header and it uses dlopen to load it. clapack, however, does *not* provide cblas (although it does provide a library libblas.so, but no libcblas.so). Therefore, when importing numpy at runtime, we get an error like: ImportError: /usr/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-arm-linux-gnueabihf.so: undefined symbol: cblas_sgemm The initial attempt to fix this added cblas to the libraries. This happens to work because apparently the entire libraries line is ignored when a non-existing library is added to it (remember, clapack does not provide libcblas). Another attempt was to set BLAS=None in the environment. This didn't have any effect. Setting both BLAS=None and LAPACK=None does disable lapack and blas, but then we don't use clapack at all. In fact, it is not necessary to provide a libraries line at all: the build system will attempt to find liblapack, libblas and libcblas without any help. Therefore, remove the libraries line from site.cfg and remove PYTHON_NUMPY_SITE_CFG_LIBS. Note that the paths to staging's /usr/include and /usr/lib need to be specified explicitly. Indeed, the numpy build system doesn't use the compiler to check the presence/absence of includes and libraries; it searches the paths itself. It also hardcodes paths to /usr/lib etc, but this is something that will be tackled in a separate commit. Note that there is another problem: both lapack and clapack provide libblas.so and liblapack.so. This will be handled in a later commit. Also, openblas provides a cblas implementation in libopenblas.so, so there should be a dependency on openblas to make sure numpy can find it. This part is not entirely clear yet, so it will also be handled in a separate commit. Runtime testing is essential to be able to track this kind of issue, so that is something that will be added in a separate commit as well. Fixes: http://lists.busybox.net/pipermail/buildroot/2019-June/252380.html Initial patch from Giulio Benetti : [v1] http://patchwork.ozlabs.org/patch/1100100/ [v2] http://patchwork.ozlabs.org/patch/1100208/ Signed-off-by: Alexandre PAYEN <alexandre.payen@smile.fr> Cc: Giulio Benetti <giulio.benetti@micronovasrl.com> Signed-off-by: Romain Naour <romain.naour@smile.fr> Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
39 lines
1.3 KiB
Makefile
39 lines
1.3 KiB
Makefile
################################################################################
|
|
#
|
|
# python-numpy
|
|
#
|
|
################################################################################
|
|
|
|
PYTHON_NUMPY_VERSION = 1.16.4
|
|
PYTHON_NUMPY_SOURCE = numpy-$(PYTHON_NUMPY_VERSION).tar.gz
|
|
PYTHON_NUMPY_SITE = https://github.com/numpy/numpy/releases/download/v$(PYTHON_NUMPY_VERSION)
|
|
PYTHON_NUMPY_LICENSE = BSD-3-Clause, BSD-2-Clause, PSF, Apache-2.0, MIT, Zlib
|
|
PYTHON_NUMPY_LICENSE_FILES = LICENSE.txt doc/sphinxext/LICENSE.txt \
|
|
doc/scipy-sphinx-theme/LICENSE.txt \
|
|
numpy/linalg/lapack_lite/LICENSE.txt \
|
|
tools/npy_tempita/license.txt \
|
|
numpy/core/src/multiarray/dragon4.c
|
|
PYTHON_NUMPY_SETUP_TYPE = setuptools
|
|
|
|
ifeq ($(BR2_PACKAGE_CLAPACK),y)
|
|
PYTHON_NUMPY_DEPENDENCIES += clapack
|
|
else
|
|
PYTHON_NUMPY_ENV += BLAS=None LAPACK=None
|
|
endif
|
|
|
|
PYTHON_NUMPY_BUILD_OPTS = --fcompiler=None
|
|
|
|
define PYTHON_NUMPY_CONFIGURE_CMDS
|
|
-rm -f $(@D)/site.cfg
|
|
echo "[DEFAULT]" >> $(@D)/site.cfg
|
|
echo "library_dirs = $(STAGING_DIR)/usr/lib" >> $(@D)/site.cfg
|
|
echo "include_dirs = $(STAGING_DIR)/usr/include" >> $(@D)/site.cfg
|
|
endef
|
|
|
|
# Some package may include few headers from NumPy, so let's install it
|
|
# in the staging area.
|
|
PYTHON_NUMPY_INSTALL_STAGING = YES
|
|
|
|
$(eval $(python-package))
|
|
$(eval $(host-python-package))
|