Fixes:
http://autobuild.buildroot.net/results/3b6/3b6280b0b7a9634b747db2865b21c6266007c725/
The PYTHON_KEEP_PY_FILES global variable conflicts with the per-package
<pkg>_KEEP_PY_FILES variable for the python package, causing make to
complain:
package/zlib/zlib.mk:7: *** Recursive variable 'PYTHON_KEEP_PY_FILES' references itself (eventually). Stop.
As a workaround, rename the global variable to KEEP_PYTHON_PY_FILES so it
cannot conflict with the per-package variable.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When BR2_PACKAGE_PYTHON{,3}_PYC_ONLY=y, we force remove all .py files
from the system, as they have all been byte-compiled into their .pyc
variants.
However, it turns out that some packages (e.g: OpenCV) do some funky
things with a few .py files: they pass them through Python's
execfile() facility, which only works with .py files and not .pyc
files. It is used by OpenCV for example to read two small
configuration files.
In order to support such use cases, this commit introduces a very
simple mechanism by which packages can exclude some path patterns from
the .py removal: a per-package <pkg>_KEEP_PY_FILES variable that is
collected into a global PYTHON_KEEP_PY_FILES variable, then used by
the python/python3 target-finalize hooks.
This variable is intentionally not documented, this is really a hack
that we ideally would like to see go away, and we'd rather not see its
usage spread too much.
This is necessary to be able to fix bug #12171.
[Peter: check if PYTHON_KEEP_PY_FILES contains non-white space]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Some python scripts may be ran in the custom scripts a user can define
in the config. Allow the user to enable host-python3 explicitly.
If any of those require ssl, they will fail with no possible fix.
Add an option to enable openssl as well. This is made optional because
openssl significantly increases the build time.
Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
python3 nowadays appends the triplet to the config-<version>m directory:
echo target/usr/lib/python3.7/config-*
target/usr/lib/python3.7/config-3.7m-powerpc-linux-gnu
Likewise, there is no longer a pyconfig.h:
ls target/usr/lib/python3.7/config-3.7m-powerpc-linux-gnu
config.c config.c.in install-sh libpython3.7m.a Makefile
makesetup python-config.py python.o Setup Setup.local
So adjust the removal logic to match. Use a wildcard rather than
$GNU_TARGET_NAME as buildroot and python3's idea of the triplet doesn't
always match (E.G. for musl/uclibc).
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Fixes:
http://autobuild.buildroot.net/results/cb4/cb49c539501342e45cbe5ade82e588fcdf51f05b
GCC commit 6834b83784dcf0364eb820e8 (multiarch support for non-glibc linux
systems), which is part of GCC 8+, changed the multiarch logic to use
$arch-linux-musl / $arch-linux-uclibc rather than $arch-linux-gnu.
This then causes the python3 configure script to error out:
checking for the platform triplet based on compiler characteristics... powerpc-linux-gnu
configure: error: internal configure error for the platform triplet, please file a bug report
http://autobuild.buildroot.net/results/cb4/cb49c539501342e45cbe5ade82e588fcdf51f05b
As it requires that the --print-multiarch output (if not empty) matches the
deduced triplet (which always uses -linux-gnu).
It isn't quite clear why --print-multiarch returns something for a
non-multiarch toolchain on some architectures (E.G. PowerPC), but as a
workaround, add a patch to rewrite the --print-multiarch output to match
older GCC versions to keep the configure script happy.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Fixes the following security issues:
- bpo-37463: ssl.match_hostname() no longer accepts IPv4 addresses with
additional text after the address and only quad-dotted notation without
trailing whitespaces. Some inet_aton() implementations ignore whitespace
and all data after whitespace, e.g. ‘127.0.0.1 whatever’.
- bpo-35907: CVE-2019-9948: Avoid file reading by disallowing local-file://
and local_file:// URL schemes in URLopener().open() and
URLopener().retrieve() of urllib.request.
- bpo-30458: Address CVE-2019-9740 by disallowing URL paths with embedded
whitespace or control characters through into the underlying http client
request. Such potentially malicious header injection URLs now cause an
http.client.InvalidURL exception to be raised.
- bpo-33529: Prevent fold function used in email header encoding from
entering infinite loop when there are too many non-ASCII characters in a
header.
- bpo-35755: shutil.which() now uses os.confstr("CS_PATH") if available and
if the PATH environment variable is not set. Remove also the current
directory from posixpath.defpath. On Unix, shutil.which() and the
subprocess module no longer search the executable in the current directory
if the PATH environment variable is not set.
Also remove the following upstreamed patches:
- 0033-bpo-36742-Fixes-handling-of-pre-normalization-charac.patch
- 0034-bpo-36742-Corrects-fix-to-handle-decomposition-in-us.patch
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
[Peter: mention security fixes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Globally change Adam Duskett's email address to aduskett@gmail.com.
Note that one or two of the patches may have been applied upstream with
the old email address, but in that case those patches will anyway be
removed when bumping.
Signed-off-by: Adam Duskett <Aduskett@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Fixes CVE-2019-10160: urlsplit does not handle NFKC normalization (2nd fix)
While the fix for CVE-2019-9936 is included in 3.7.3, the followup
regression fixes unfortunatly aren't.
https://bugs.python.org/issue36742
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Python3 at runtime identifies the uuid backend to use to implement safe
time-based UUID generation functions. When the python _uuid extension
module, /usr/lib/python3.7/lib-dynload/_uuid.cpython-37m-<arch>.so
is not found, the fall back is to use a pure python implementation.
- If uuid.h from util-linux is available at build time, the _uuid
module is built and used. A Buildroot patch allows us to disable this
build-time check and never build the _uuid module.
With the bump to python 3.7, an optional dependency on util-linux was
added to build the _uuid module. However, this may lead to circular
dependencies. To break this circular dependency, we can disable the
build of the _uuid module and always use the pure python implementation.
Ref:
http://lists.busybox.net/pipermail/buildroot/2018-October/233113.htmlhttp://lists.busybox.net/pipermail/buildroot/2018-September/231060.htmlhttp://lists.busybox.net/pipermail/buildroot/2018-October/233079.html
CC: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Tested-by: Carlos Santos <casantos@datacom.com.br>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When python3 depends on bluez5_utils-headers,
python3 is build with bluetooth support.
Signed-off-by: Grzegorz Blach <grzegorz@blach.pl>
[Arnout: select bluez5_utils-headers is bluez5_utils is enabled]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Other changes include:
- Refreshing all necessary patches for 3.7.0
- Add a hash for the license file.
- Python no longer has it's own internal libffi, as such, host-libffi
is now required to build host-python3, and is added as a
dependency.
- Drop PYTHON3_LIBTOOL_PATCH = NO, since there is no longer any
internal libffi copy that was causing the libtool patching process
to fail.
- A new core module "uuid" is now is added in the Config.in file, and
relies on util-linux's uuid library.
- Also, a new patch: 0030-Fix-cross-compiling-the-uuid-module.patch
is required to fix compiling the uuid module, because the include
directory search path for uuid.h is hardcoded to /usr/include/uuid,
which causes an "unsafe for cross-compilation" error during
compiling if the host pc has uuid headers installed.
- 0031-Add-an-option-to-disable-uuid-module.patch is added to allow
disabling the Python3 UUID module, so that when
BR2_PACKAGE_PYTHON3_UUID is disabled by the UUID library is
present, the uuid Python module is not built, as expected.
- 0032-fix-building-on-older-distributions.patch is added to change
os.replace by os.rename in the update_file.py script to fix
building on older Linux distributions that have older versions of
python that don't include os.replace.
os.rename acts in the same way as os.replace, but is cross-platform
compatible. Because Buildroot is guaranteed to be built in a POSIX
environment, it is safe to change replace to rename.
Tested on CentOS6 and Fedora28, All test results passed:
br-arm-full [1/6]: OK
br-arm-cortex-a9-glibc [2/6]: OK
br-arm-cortex-m4-full [3/6]: SKIPPED
br-x86-64-musl [4/6]: OK
br-arm-full-static [5/6]: SKIPPED
armv5-ctng-linux-gnueabi [6/6]: OK
6 builds, 2 skipped, 0 build failed, 0 legal-info failed
Signed-off-by: Adam Duskett <aduskett@gmail.com>
[Thomas:
- remove PYTHON3_LIBTOOL_PATCH = NO
- improve the solution in patch 0030-Fix-cross-compiling-the-uuid-module
- add patch 0031-Add-an-option-to-disable-uuid-module]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This reverts commit 233202597d, which
causes a lot of build failures. Part of the Python build process tries
to use os.replace(), which is only available since Python 3.3. It
should work if the host-python being built was used, but unfortunately
the system Python ends up being used, causing the build failure.
Fixes:
http://autobuild.buildroot.net/results/ed95a7ded6bd6c17bd0820b3a96862487b71eb2b/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit fixes the warnings reported by check-package on the help
text of all package Config.in files, related to the formatting of the
help text: should start with a tab, then 2 spaces, then at most 62
characters.
The vast majority of warnings fixed were caused by too long lines. A
few warnings were related to spaces being used instead of a tab to
indent the help text.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
... as reported by utils/check-package
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Adam Duskett <aduskett@gmail.com>
Cc: Yegor Yefremov<yegorslists@googlemail.com>
Cc: Matt Weber <matthew.weber@rockwellcollins.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When prefix is set to a path like /usr during crossbuild
the sed operations end up executing twice, once for the prefix
reassignment and another for includedir if it is set as a string
including the ${prefix} variable. This results in an issue
when the build directory is under /usr.
This patch updates the remaining location which uses the prefix
variable to also sed and update to use the real path.
Upstream bug report:
https://bugs.python.org/issue31713
Buildroot bug:
https://bugs.busybox.net/show_bug.cgi?id=10361
Fixes failures like the following:
dbus-python-1.2.4 | NOK | http://autobuild.buildroot.net/results/758858efa97b6273c1b470513f5492258a6d8853
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[Arnout: refer to autobuild failures that still exist]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Bump Python3 version to 3.6.2.
Patches dropped:
"Support PGEN_FOR_BUILD and FREEZE_IMPORTLIB_FOR_BUILD"
Rationale: With commit 9d02f562961efd12d3c8317a10916db7f77330cc, code
generation step of building CPython now became explicit (instead of
always performed as a part of 'make' invocation) and more granular. We
no longer need to use Parser/pgen at all and tricking the build system
into using different Programs/_freeze_importlib can be done as a part
of recipe.
Additional info about the build change can be found at
https://bugs.python.org/issue23404
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Tested-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This commit is similar to 350941e31d
("python: remove target Python packages from PYTHONPATH") but for
python3.
We currently have
$(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/site-packages/
inside the PYTHON3_PATH variable, which gets used to define
PYTHONPATH, passed to the host Python interpreter when
building/installing target packages.
However, this is terribly wrong, as it causes the host interpreter to
potentially import target Python packages. This is wrong for several
reasons:
- Some Python packages might need some Python modules to be installed
on the host (described in setup_requires in setup.py), but their
installation currently works because by luck the corresponding
Python module is installed for the target. Some of those cases were
happening for real, and fixed by previous patches.
- Some Python packages include some native code, therefore built for
a specific CPU architecture. When you point the host Python
interpreter to native libraries built for the target, you get nice
build failures, such as the one affecting the python-cffi related
packages.
This change fixes the following build failures:
http://autobuild.buildroot.net/results/9005b89407e46b537a54cac6cc0c69dcac4dc5ea/
(python-cryptography)
http://autobuild.buildroot.net/results/395682d33d02fdcaa39d3c0326355bd9ea3d6feb/
(python-pynacl)
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Tested-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Since things are no longer installed in $(HOST_DIR)/usr, the callers
should also not refer to it.
This is a mechanical change with
git grep -l '$(HOST_DIR)/usr/bin' | xargs sed -i 's%$(HOST_DIR)/usr/bin%$(HOST_DIR)/bin%g'
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The check-package script when ran gave warnings on only using
one space before backslashes on all of these makefiles.
This patch cleans up all warnings related to the one space before
backslashes rule in the make files in the package directory.
Signed-off-by: Adam Duskett <aduskett@codeblue.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
.pyc files include path to source .py file. This patch changes the way
`pycompile.py' is launched in order to only keep the part relative to
$TARGET_DIR.
This work was sponsored by `BA Robotic Systems'.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Whitespaces were searched using the following regex:
[ ]{1,}\t
and then manually removed in most of the cases. For
xserver_xorg-server.mk, tabs before backslashes were removed.
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Update all appropriate version numbers as well as SHAs and MD5s as well
as repbasing BR's patches on top of 3.6.1 codebase (new github repo
tree, v3.6.1 tag was used). Note that patch:
[PATCH] Change the install location of _sysconfigdata.py
was dropped due to the fact taht build system now adds platform
specific suffix to sysconfigdata's name, so each platform's file
should have a unique name and distutils now allows to specify which
sysconfigdata is used via _PYTHON_SYSCONFIGDATA_NAME
see:
c4b53afce492dec548ff
and patches:
[PATCH] distutils/sysconfig: use sysconfigdata
[PATCH] setup.py: do not add invalid header locations
[PATCH] Do not harcode invalid path to ncursesw headers
was dropped since it looks like it made it's way upstream, see:
409482251b1351c31aa9e13c3201fb
respectively.
Signed-off-by: Andrey Smirnov <andrew.smirnov@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
.pyc files contain the modification time of the corresponding .py
source. In order to make the build reproducible, we fix the modification
time of all .py before compiling .pyc files.
In addition, since pycompile relies on the modification time to know if
a file needs to be recompiled, it is safer to force recompilation of all
source files.
This work was sponsored by `BA Robotic Systems'.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
[Thomas: do not register PYTHON_FIX_TIME as a
PYTHON_TARGET_FINALIZE_HOOKS, instead call it inside
PYTHON_CREATE_PYC_FILES before doing the byte compilation.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The BR2_PACKAGE_PYTHON3_{READLINE,ZLIB,BZIP2,XZ} options were so far
only bringing in the necessary dependencies, relying on the Python
build system to automatically detect them.
However, this means that even if one of those option was disabled, if
their dependency was found, Python would build the corresponding module,
which is really not what the user would expect.
For example, if you have:
BR2_PACKAGE_READLINE=y
# BR2_PACKAGE_PYTHON3_READLINE is not set
Then you would still get the readline Python module built and installed.
This commit fixes that by adding new --{enable,disable} options, and use
them in python3.mk.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Commit fa62773826 ("python3: do not use the system OpenSSL in the host
variant") added a patch that allows to disable building the OpenSSL
related modules in Python, even if OpenSSL is found.
But in this commit, it was only used to unconditionally disable OpenSSL
support for the host python3.
This commit extends that to use the --disable-openssl option also for
the target python3, when BR2_PACKAGE_PYTHON3_SSL. This ensures that if
BR2_PACKAGE_PYTHON3_SSL is disabled, but BR2_PACKAGE_OPENSSL is enabled,
we still don't get the OpenSSL modules built, as the user would expect.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This refreshes the set of python3 patches so they apply cleanly on the
v3.5.2 tag of cpython Github repository.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
host-python3 currently detect if there is an usable OpenSSL installation
and conditionnaly compiles the 'ssl', '_ssl' and '_hashlib' modules.
This may break compilation if the system's OpenSSL has been updated to
1.1.0 because of a bug in python, see https://bugs.python.org/issue26470
for details.
Unlike Python 2.7, Python 3 unconditionnaly compiles fallbacks for
common hash algorithm, so disabling OpenSSL will still leave Python 3
with implementations of common hash algorithm.
This adds a patch to configure.ac patch to implement a --disable-openssl
option.
Signed-off-by: Nicolas Cavallari <nicolas.cavallari@green-communications.fr>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Python is not able to detect if compiler double representation is
compliant with IEE754:
checking whether C doubles are little-endian IEEE 754 binary64... no
checking whether C doubles are big-endian IEEE 754 binary64... no
checking whether C doubles are ARM mixed-endian IEEE 754 binary64... no
Accordingly 'legacy' mode isused. It is possible to check this at
runtime by check if 'sys.float_repr_style' contains 'short' or
'legacy'. Calculus correctness is not garanteed with 'legacy'.
Problem is better described here:
http://stackoverflow.com/questions/29920294/what-causes-pythons-float-repr-style-to-use-legacyhttps://bugs.python.org/issue7117
However, all gcc architecture use a representation compliant with
IEE754. So, we can enable it unconditionnaly.
Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
[Thomas: rework condition to not use strip, as suggested by Baruch.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Needed by the upcoming python-mwscrape2slob package.
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
[Thomas: move "select" after "bool".]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
During the execution of its configure script, Python 3 tries to find an
available "hg" installation, and if available, will try to use it to get
information from the version control system. To do this, it tries to
communicate over the network, potentially over ports that are blocked,
causing the build to halt. This was reported by a user as part of bug
7802.
To solve this, we simply make the Python script use /bin/false as the
"hg" program.
Fixes bug #7802 for the python3 package.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Register package-specific target-finalize hooks with the
newly-introduced <PKG>_TARGET_FINALIZE_HOOKS.
This incidentally fixes luarocks, which was registering target-finalize
hooks even when it was not enabled.
To be noted, the skeleton package is not converted, because it is not
optional, we always have it; so its hooks would always be registered
anyway. Besides, the followup patches would render this conversion moot
anyway, since those hooks would be spread across the various skeleton
packages.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, each python package (be it the python interpreter package
itself or external python modules) is responsible for compiling its
.py into .pyc files. Unfortunately, this is not ideal as some packages
only install .py files without compiling them into .pyc files. In this
case, if the Buildroot configuration specifies to keep only the .pyc
files, the .py files are removed and lost.
To address this, this commit changes the logic by making the
compilation of .pyc files a global operation: the python interpreter
packages register a target finalize hook that is in charge of
compiling all installed .py files.
The *.pyc generation on a per package basis is disabled in the
python-package infrastructure by passing the "--no-compile" option to
setup.py.
The *.pyc generation for the Python interpreter internal modules is
disabled through --disable-pyc-build configure option.
A small helper script is used to perform the compilation, the purpose
of this script is to abort the compilation process if one of the .py
file cannot be compiled. It has been provided by Samuel Martin and
integrated into this commit.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
[Thomas:
- rework for python 3.5
- integrate Samuel proposal that allows to detect compilation
failures.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As suggested by Samuel Martin, this commit adds the option
--no-run-if-empty xargs option to the "find ... | xargs ..." logic used
in the python and python3 target-finalize hooks to remove py/pyc/pyo
files. This ensures that the command doesn't fail if there are no files
matching the pattern.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The major changes in terms of Buildroot packaging are:
- Due to PEP488, Python no longer generates .pyc (unoptimized) and
.pyo (optimized) byte-code files. Instead, it generates <foo>.pyc,
<foo>.opt-1.pyc and <foo>.opt-2.pyc. Therefore, we removed the
--disable-pyo-build option and kept only the --disable-pyc-build
option, which completely disables building all .pyc files. In
addition, since the optimized .opt-X.pyc files don't work if the
corresponding un-optimized .pyc file is not present, we are for the
moment unconditionally removing the optimized ones (keeping both
the unoptimized and optimized ones doubles the required filesystem
size!). So basically we preserve the behavior we had before this
commit:
BR2_PACKAGE_PYTHON3_PY_ONLY -> only *.py
BR2_PACKAGE_PYTHON3_PYC_ONLY -> only non-optimized *.pyc
BR2_PACKAGE_PYTHON3_PY_PYC -> both the *.py and non-optimized *.pyc
To achieve this, the TARGET_FINALIZE_HOOKS are reworked:
PYTHON3_REMOVE_PY_FILES is responsible for removing *.py files in
the BR2_PACKAGE_PYTHON3_PYC_ONLY case.
PYTHON3_REMOVE_PYC_FILES is responsible for removing *.pyc files
in the BR2_PACKAGE_PYTHON3_PY_ONLY case.
PYTHON3_REMOVE_OPTIMIZED_PYC_FILES is responsible for removing the
optimized *.opt-1.pyc and *.opt-2.pyc files, which is done
unconditionally.
- The PEP3147 disabling patch had to be significantly reworked due to
the code having changed heavily. The code was moved into a
_bootstrap_external.py, which is a "frozen" Python module, i.e a
module generated into a .h file at compile time using the
_freeze_importlib program.
- Due to the above, we now need to regenerate importlib.h at build
time. Unfortunately, for the target Python _freeze_importlib is
built for the target, so we can't run it on the build machine. To
fix this, we copy the _freeze_importlib program from the
host-python in $(HOST_DIR), and then patch the target python to use
it. Since the same solution can be used for 'pgen', we do it, and
avoid having to touch the graminit.{c,h} files.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In preparation for the bump to Python 3.5.0, let's switch all the
patches to the Git format. This way, a Git repository of the Python
source code can be used to manage those patches, which makes it easier
to bump to newer Python versions.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Packages built with automake use a `py-compile` helper to byte-compile
Python source files. This script uses the "py_compile" module from the
standard library. In turn, the compile() function in the "py_compile"
module invokes the cache_from_source() function provided by importlib.
This commit adds a new patch named "020-importlib-no-pep3147.patch"
that changes cache_from_source() and source_from_cache() in importlib
to get rid of the "__pycache__" directory.
This commit fixes the following import error in kmod when the module
is built for Python 3:
>>> from kmod import Kmod
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name 'Kmod'
Moreover, this commit removes two patches that are no longer necessary
since modifying cache_from_source() and source_from_cache() disables
PEP 3147 for the standard library and distutils / setuptools.
* 004-old-stdlib-cache.patch
* 016-distutils-no-pep3147.patch
Signed-off-by: Christophe Vu-Brugier <cvubrugier@fastmail.fm>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>