Commit af494d92d3 (utils/genrandconfig: disable libopenssl without
atomics) intorduced a code-style issue that flake8 does not like:
$ make check-flake8
utils/genrandconfig:253:8: E713 test for membership should be 'not in'
1 E713 test for membership should be 'not in'
Fixes: af494d92d3https://gitlab.com/buildroot.org/buildroot/-/jobs/3045260108
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
libopenssl needs atomic or the build will fail (e.g. on sparcv8 without
libatomic):
${LDCMD:-/nvmedata/autobuild/instance-7/output-1/host/bin/sparc-buildroot-linux-uclibc-gcc} -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -O0 -g2 -g2 -L. \
-o apps/openssl apps/asn1pars.o apps/ca.o apps/ciphers.o apps/cms.o apps/crl.o apps/crl2p7.o apps/dgst.o apps/dhparam.o apps/dsa.o apps/dsaparam.o apps/ec.o apps/ecparam.o apps/enc.o apps/engine.o apps/errstr.o apps/gendsa.o apps/genpkey.o apps/genrsa.o apps/nseq.o apps/ocsp.o apps/openssl.o apps/passwd.o apps/pkcs12.o apps/pkcs7.o apps/pkcs8.o apps/pkey.o apps/pkeyparam.o apps/pkeyutl.o apps/prime.o apps/rand.o apps/rehash.o apps/req.o apps/rsa.o apps/rsautl.o apps/s_client.o apps/s_server.o apps/s_time.o apps/sess_id.o apps/smime.o apps/speed.o apps/spkac.o apps/srp.o apps/storeutl.o apps/ts.o apps/verify.o apps/version.o apps/x509.o \
apps/libapps.a -lssl -lcrypto -ldl
/nvmedata/autobuild/instance-7/output-1/host/lib/gcc/sparc-buildroot-linux-uclibc/10.3.0/../../../../sparc-buildroot-linux-uclibc/bin/ld: ./libssl.so: undefined reference to `__atomic_fetch_sub_4'
It should be noted that openssl3 has added OPENSSL_DEV_NO_ATOMICS but
"this is intended for internal development only, to check the
refcounting is properly coded. It should never become a configuration
option, hence the name of the macro.":
503d4745a1
Atomics are not available in Buildroot if:
- architecture is 32 bit and something other than ARM or xtensa, and
- GCC < 4.8 or no threads or FLAT.
The nothreads case can theoretically happen in many different
situations, but in practice nobody disables threads. So the only
interesting case is the FLAT case. Since ARM and RISC-V 64 both have
atomics intrinsics, that leaves just m68k NOMMU as FLAT. So this is
truly a corner case.
The proper solution would be to patch GCC to also provide libatomic in
those cases.
- For nothreads, atomics are in fact not needed, so libatomic can simply
be implemented as stubs.
- For FLAT, it's probably just a matter of having a match to uclinux in
libatomic/configure.tgt.
Again, though, this happens only in such niche cases that it's not worth
working on it.
Fixes:
- http://autobuild.buildroot.org/results/bce526d538f43a541fdfbc0c9b4a7cecebbbc539
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
From flake8:
utils/genrandconfig:429:21: E703 statement ends with a semicolon
1 E703 statement ends with a semicolon
Fixes: d3e029575c
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Add a custom case to make sure that a random configuration with an empty
version for aufs-util doesn't fail.
Fixes:
- http://autobuild.buildroot.org/results/e242cf66a02983bcf6e95b37f8e458bd18aee683
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Improve commit 541e794a95 by adding a
custom case to make sure that a random configuration with an empty
platform for arm-trusted-firmware doesn't fail:
make_helpers/plat_helpers.mk:15: *** "Error: Unknown platform. Please use PLAT=<platform name> to specify the platform". Stop.
Fixes:
- http://autobuild.buildroot.org/results/1b67220008223d1bcbe70b76d643f9d04362ba6b
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In case of an unexpected error, we currently only print the exception as
an str(). For example, the recent issue with the glibc version check
only reported:
TypeError: cannot use a string pattern on a bytes-like object
That does not help in fixing the issue; the exception text is also not
usually very user-friendly either anyway.
We change the reporting to print the traceback, which in the glibc
version check mentioned above, the error is reported as:
Traceback (most recent call last):
File "./utils/genrandconfig", line 740, in <module>
ret = gen_config(args)
File "./utils/genrandconfig", line 676, in gen_config
if not is_toolchain_usable(configfile, toolchainconfig):
File "./utils/genrandconfig", line 186, in is_toolchain_usable
if StrictVersion('2.14') > StrictVersion(glibc_version):
File "/usr/lib/python3.8/distutils/version.py", line 40, in __init__
self.parse(vstring)
File "/usr/lib/python3.8/distutils/version.py", line 135, in parse
match = self.version_re.match(vstring)
TypeError: cannot use a string pattern on a bytes-like object
With this, the error is much easier to pinpoint (it's the last one that
is not in a system module).
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Unless explicitly told otherwise, subprocess.check_output() returns
bytes objects [0].
When we try to check the C library version (to check the Linaro
toolchain is usable), genrandconfig currently fails with:
TypeError: cannot use a string pattern on a bytes-like object
So, as suggested in the python documentation, decocde() the output of
subprocess.check_output() before we can use it.
[0] https://docs.python.org/3/library/subprocess.html#subprocess.check_output
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add custom cases to make sure that a random configuration with an empty
git, tarball location or version for xenomai doesn't fail. It reverts to
BR2_PACKAGE_XENOMAI_LATEST_VERSION in these cases.
Fixes:
- http://autobuild.buildroot.org/results/19419759b4add0197b4e629d0b2216c2a07624b2
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Add a custom case to make sure that a random configuration with an empty
git location for refpolicy doesn't fail. It reverts to
BR2_PACKAGE_REFPOLICY_UPSTREAM_VERSION in that case.
Fixes:
- http://autobuild.buildroot.org/results/98da07a1f9b88e2e8a028a55c8cf76c6ebc28304
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a custom case to make sure that a random configuration with an
empty tarball location for OP-TEE OS doesn't fail. It reverts to
BR2_TARGET_OPTEE_OS_LATEST in that case.
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The support for this architecture has been removed from the upstream
Linux kernel, as of commit:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=aec499c75cf8e0b599be4d559e6922b613085f8f
Which states:
The nds32 architecture, also known as AndeStar V3, is a custom
32-bit RISC target designed by Andes Technologies. Support was added
to the kernel in 2016 as the replacement RISC-V based V5 processors
were already announced, and maintained by (current or former) Andes
employees.
As explained by Alan Kao, new customers are now all using RISC-V,
and all known nds32 users are already on longterm stable kernels
provided by Andes, with no development work going into mainline
support any more.
There has also been little to no maintenance done in Buildroot for
this architecture in recent times, so let's follow the Linux kernel
community decision and drop support for this CPU architecture.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
If we don't have a DTS name or path we need to also disable cuimage
to disable DTS support as BR2_LINUX_KERNEL_CUIMAGE requires DTS
support.
Fixes:
linux/linux.mk:591: *** No kernel device tree source specified, check your BR2_LINUX_KERNEL_INTREE_DTS_NAME / BR2_LINUX_KERNEL_CUSTOM_DTS_PATH settings. Stop.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
If the kconfig values being replaced are not empty we should not
try to fix them.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Internally genrandconfig will use gettimeofday when generating a
KCONFIG_SEED, since autobuild-run executes genrandconfig at the same
time for multiple autobuilder runners this could potentially result
in the same KCONFIG_SEED being generated for those test runs started
at the same time.
To ensure this doesn't happen set the KCONFIG_SEED using the urandom
entropy source.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This should have been BR2_TARGET_ROOTFS_EXT2 as the
BR2_TARGET_ROOTFS_EXT2_GEN variable is an integer variable used
to indicate the ext2/ext3/ext4 variant.
Fixes:
- http://autobuild.buildroot.net/results/5d7/5d7833212bd8a2b575945f848d8c91dabe7d3c30
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Until commit "arch/Config.in.sh: fixup MMU selection" in this series,
SH2A could either be used with BR2_USE_MMU disabled or BR2_USE_MMU
enabled.
The later made absolutely no sense, since SH2A does not have a MMU:
MMU support was introduced starting from SH3 according to
https://en.wikipedia.org/wiki/SuperH#SH-3
Also, since commit 22d5501e03 ("arch:
tidy up binary formats config"), which was merged in Buildroot
2015.05, the architecture tuple used when BR2_sh2a=y and BR2_USE_MMU
disabled is sh2a-buildroot-uclinux-uclibc, and this was already
unsupported back in the days of Buildroot 2015.08 and binutils 2.24,
causing the build to fail with:
*** BFD does not support target sh2a-buildroot-uclinux-uclibc.
just like it fails to build today with recent version of binutils.
So, this has been broken since 2015.08, and nobody complained. SH2A is
seldom used, so it's time to kill it.
It is worth mentioning that there had been an attempt at resurrecting
SH2 support around 2015 (see https://lwn.net/Articles/647636/) as part
of the J2 core. This effort led to the addition of FDPIC support for
SH2A in the musl C library (and therefore proper ELF binaries, with
shared libraries), but that was never supported in Buildroot. Now that
the J2 project is essentially dead, there is no reason to bother with
this.
Fixes:
http://autobuild.buildroot.net/results/63d01d33ae30f86b63b9f42a9fea116f2f3e9005/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently we only test a limited set of toolchains that are mostly
prebuilt, add a flag to allow using randconfig for randomizing
additional toolchain settings instead of randpackageconfig.
To avoid invalid configs we need to add additional config validation
filtering and fixups. Although the loop around 'make randconfig'
attempts does make sure we eventually end up with a valid configuration,
there is a bias towards e.g. disabling the kernel. It would be possible
to e.g. always force BR2_LINUX_KERNEL_USE_ARCH_DEFAULT_CONFIG=y, but
that removes the possibility for a toolchains-csv config to add a valid
custom defconfig. (Note that currently this isn't possible anyway since
'make randpackageconfig' is used if a toolchains-csv is provided, but
eventually we want to use 'make randconfig' also if a toolchains-csv is
provided.)
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The option was removed by commit 306839586b (package/python: drop target
package).
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
genrandconfig is used by the Buildroot autobuilders to generate
semi-random configurations that we build test. As part of this, we use
"make randpackageconfig" to randomize the selection of packages,
together with a KCONFIG_PROBABILITY value, which indicates the
probabibility for each option to be enabled. This probability is
itself randomized, between 1% and 30% for every build.
However, with our increasing number of packages (over 2900), when we
use a 30% probability for options to be enabled, it means a *lot* of
options are enabled, causing very large configurations to be
tested. These configurations are not very realistic, and they take
ages to build on our autobuilders: we have builds that take 4, 5 or
even 7 hours to build.
In order to test a larger number of configurations and therefore a
larger variety of configurations, this commit reduces the maximum
probability to 20%.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Python 2 is EOL sice 2020 [1], it's still available on distros, but may not
be installed by default (as being replaced by python3).
Thus remove compatibility imports:
from __future__ import print_function
from __future__ import absolute_import
Tested with python3 -m py_compile.
[1] https://www.python.org/doc/sunset-python-2/
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
... similar to BR2_ENABLE_DEBUG.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since 810ba387be, some form of these options are enable
by default. Specifically:
- Kept FORTIFY level 2 option as the default is now level 1.
- Removed all SSP options as the default now uses the best
option based on toolchain support.
- Similar to SSP, for RELRO, the default now uses the best
option based on toolchain support.
- Completely drop PIC PIE as it defaults =y
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Normally the kconfig stings would end up empty and cause a build
error. This patch provides test files to allow testing the creation
of uboot environment and script bin files from user provided txt files.
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Commit 9ea528f84b (package/python-nfc: bump to version 0.13.5) changed the
python-nfc package to download from github, so the package no longer needs
bzr on the host.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
With classpath removed, no packages select these symbols any more - So drop
them and their corresponding logic in dependencies.sh / genrandconfig.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Now that the support for per-package directories has been merged, it
is time to get some exposure for it in the autobuilders, so let's
build 1 out of 15 builds with this feature enabled, at least as an
initial step.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
With Python 3.7, genrandconfig fails with:
'str' object has no attribute 'decode'
We are already working on str objects, and there is no need to decode
them, so we drop the call to decode_byte_list() and its definition as
it was only used there.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This patch adds the remaining ssp and relro cases plus both of the
fortify options. The randomization was left consistant between the
options but the order of the conditions placed the most restrictive
hardening options with more priority.
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
After a few weeks of running reproducible builds in the autobuilders,
we found out that such builds without a diffoscope analysis are pretty
useless: the cmp on the tarballs doesn't help us fix the
reproducibility issue.
So, let's only do reproducible builds when diffoscope is available.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Atharva Lele <itsatharva@gmail.com>
Acked-by: Atharva Lele <itsatharva@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In a future patch, we are going to need a SystemInfo() instance in
gen_config(), so create the SystemInfo() instance there, and pass it
to fixup_config(), where we currently use it.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Atharva Lele <itsatharva@gmail.com>
Acked-by: Atharva Lele <itsatharva@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Initial patch to start reproducibility testing.
Also enable tar filesystem to compare images since diffoscope works well
with tar.
Signed-off-by: Atharva Lele <itsatharva@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Now that Qt4 is removed, these fixups can be removed as well.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Currently all random defconfigs which are used in autobuilder use size
optimizaion (-Os), since BR2_OPTIMIZE_S=y is the default.
Adding "-O2" optimization will give better test coverage.
In many cases software gets built with speed optimization rather than
size optimization. So let's add Level 2 optimizaion option to be
generated in random defconfigs, so we could be able to test how
packages are built with "-O2" in autobuilder.
Signed-off-by: Evgeniy Didin <Evgeniy.Didin@synopsys.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: arc-buildroot@synopsys.com
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When adding the custom BR2_WGET value in the configuration,
genrandconfig forgets to add a newline. Due to this, the next option
that is added is printed on the same line as BR2_WGET="", which causes
it to be ignored.
Due to this, in all builds, the line right after BR2_WGET was
ignored. It could have been BR2_ENABLE_DEBUG, BR2_INIT_BUSYBOX,
BR2_INIT_SYSTEMD, BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV,
BR2_STATIC_LIBS or BR2_PACKAGE_PYTHON_PY_ONLY depending on the
randomization.
Fix that by adding a proper newline at the end of the BR2_WGET option.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
A number of autobuilder failures are due to the fact that autobuilder
instances use old distributions, with old SSL certificates, and
therefore wget aborts with an error "The certificate of `xyz.org' is
not trusted.".
In order to avoid such failures that are not very interesting in the
context of the autobuilders, we pass --no-check-certificate to
wget. The integrity of the downloaded files is anyway verified by the
hashes, and this is only meant to be used in the context of
testing/CI, not in production.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Yann E. MORIN