Add a custom case to make sure that a random configuration with an
empty tarball location for OP-TEE OS doesn't fail. It reverts to
BR2_TARGET_OPTEE_OS_LATEST in that case.
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The support for this architecture has been removed from the upstream
Linux kernel, as of commit:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=aec499c75cf8e0b599be4d559e6922b613085f8f
Which states:
The nds32 architecture, also known as AndeStar V3, is a custom
32-bit RISC target designed by Andes Technologies. Support was added
to the kernel in 2016 as the replacement RISC-V based V5 processors
were already announced, and maintained by (current or former) Andes
employees.
As explained by Alan Kao, new customers are now all using RISC-V,
and all known nds32 users are already on longterm stable kernels
provided by Andes, with no development work going into mainline
support any more.
There has also been little to no maintenance done in Buildroot for
this architecture in recent times, so let's follow the Linux kernel
community decision and drop support for this CPU architecture.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
If we don't have a DTS name or path we need to also disable cuimage
to disable DTS support as BR2_LINUX_KERNEL_CUIMAGE requires DTS
support.
Fixes:
linux/linux.mk:591: *** No kernel device tree source specified, check your BR2_LINUX_KERNEL_INTREE_DTS_NAME / BR2_LINUX_KERNEL_CUSTOM_DTS_PATH settings. Stop.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
If the kconfig values being replaced are not empty we should not
try to fix them.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Internally genrandconfig will use gettimeofday when generating a
KCONFIG_SEED, since autobuild-run executes genrandconfig at the same
time for multiple autobuilder runners this could potentially result
in the same KCONFIG_SEED being generated for those test runs started
at the same time.
To ensure this doesn't happen set the KCONFIG_SEED using the urandom
entropy source.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This should have been BR2_TARGET_ROOTFS_EXT2 as the
BR2_TARGET_ROOTFS_EXT2_GEN variable is an integer variable used
to indicate the ext2/ext3/ext4 variant.
Fixes:
- http://autobuild.buildroot.net/results/5d7/5d7833212bd8a2b575945f848d8c91dabe7d3c30
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Until commit "arch/Config.in.sh: fixup MMU selection" in this series,
SH2A could either be used with BR2_USE_MMU disabled or BR2_USE_MMU
enabled.
The later made absolutely no sense, since SH2A does not have a MMU:
MMU support was introduced starting from SH3 according to
https://en.wikipedia.org/wiki/SuperH#SH-3
Also, since commit 22d5501e03 ("arch:
tidy up binary formats config"), which was merged in Buildroot
2015.05, the architecture tuple used when BR2_sh2a=y and BR2_USE_MMU
disabled is sh2a-buildroot-uclinux-uclibc, and this was already
unsupported back in the days of Buildroot 2015.08 and binutils 2.24,
causing the build to fail with:
*** BFD does not support target sh2a-buildroot-uclinux-uclibc.
just like it fails to build today with recent version of binutils.
So, this has been broken since 2015.08, and nobody complained. SH2A is
seldom used, so it's time to kill it.
It is worth mentioning that there had been an attempt at resurrecting
SH2 support around 2015 (see https://lwn.net/Articles/647636/) as part
of the J2 core. This effort led to the addition of FDPIC support for
SH2A in the musl C library (and therefore proper ELF binaries, with
shared libraries), but that was never supported in Buildroot. Now that
the J2 project is essentially dead, there is no reason to bother with
this.
Fixes:
http://autobuild.buildroot.net/results/63d01d33ae30f86b63b9f42a9fea116f2f3e9005/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently we only test a limited set of toolchains that are mostly
prebuilt, add a flag to allow using randconfig for randomizing
additional toolchain settings instead of randpackageconfig.
To avoid invalid configs we need to add additional config validation
filtering and fixups. Although the loop around 'make randconfig'
attempts does make sure we eventually end up with a valid configuration,
there is a bias towards e.g. disabling the kernel. It would be possible
to e.g. always force BR2_LINUX_KERNEL_USE_ARCH_DEFAULT_CONFIG=y, but
that removes the possibility for a toolchains-csv config to add a valid
custom defconfig. (Note that currently this isn't possible anyway since
'make randpackageconfig' is used if a toolchains-csv is provided, but
eventually we want to use 'make randconfig' also if a toolchains-csv is
provided.)
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
The option was removed by commit 306839586b (package/python: drop target
package).
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
genrandconfig is used by the Buildroot autobuilders to generate
semi-random configurations that we build test. As part of this, we use
"make randpackageconfig" to randomize the selection of packages,
together with a KCONFIG_PROBABILITY value, which indicates the
probabibility for each option to be enabled. This probability is
itself randomized, between 1% and 30% for every build.
However, with our increasing number of packages (over 2900), when we
use a 30% probability for options to be enabled, it means a *lot* of
options are enabled, causing very large configurations to be
tested. These configurations are not very realistic, and they take
ages to build on our autobuilders: we have builds that take 4, 5 or
even 7 hours to build.
In order to test a larger number of configurations and therefore a
larger variety of configurations, this commit reduces the maximum
probability to 20%.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Python 2 is EOL sice 2020 [1], it's still available on distros, but may not
be installed by default (as being replaced by python3).
Thus remove compatibility imports:
from __future__ import print_function
from __future__ import absolute_import
Tested with python3 -m py_compile.
[1] https://www.python.org/doc/sunset-python-2/
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
... similar to BR2_ENABLE_DEBUG.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since 810ba387be, some form of these options are enable
by default. Specifically:
- Kept FORTIFY level 2 option as the default is now level 1.
- Removed all SSP options as the default now uses the best
option based on toolchain support.
- Similar to SSP, for RELRO, the default now uses the best
option based on toolchain support.
- Completely drop PIC PIE as it defaults =y
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Normally the kconfig stings would end up empty and cause a build
error. This patch provides test files to allow testing the creation
of uboot environment and script bin files from user provided txt files.
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Commit 9ea528f84b (package/python-nfc: bump to version 0.13.5) changed the
python-nfc package to download from github, so the package no longer needs
bzr on the host.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
With classpath removed, no packages select these symbols any more - So drop
them and their corresponding logic in dependencies.sh / genrandconfig.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Now that the support for per-package directories has been merged, it
is time to get some exposure for it in the autobuilders, so let's
build 1 out of 15 builds with this feature enabled, at least as an
initial step.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
With Python 3.7, genrandconfig fails with:
'str' object has no attribute 'decode'
We are already working on str objects, and there is no need to decode
them, so we drop the call to decode_byte_list() and its definition as
it was only used there.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This patch adds the remaining ssp and relro cases plus both of the
fortify options. The randomization was left consistant between the
options but the order of the conditions placed the most restrictive
hardening options with more priority.
Cc: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
After a few weeks of running reproducible builds in the autobuilders,
we found out that such builds without a diffoscope analysis are pretty
useless: the cmp on the tarballs doesn't help us fix the
reproducibility issue.
So, let's only do reproducible builds when diffoscope is available.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Atharva Lele <itsatharva@gmail.com>
Acked-by: Atharva Lele <itsatharva@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In a future patch, we are going to need a SystemInfo() instance in
gen_config(), so create the SystemInfo() instance there, and pass it
to fixup_config(), where we currently use it.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Atharva Lele <itsatharva@gmail.com>
Acked-by: Atharva Lele <itsatharva@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Initial patch to start reproducibility testing.
Also enable tar filesystem to compare images since diffoscope works well
with tar.
Signed-off-by: Atharva Lele <itsatharva@gmail.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Now that Qt4 is removed, these fixups can be removed as well.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Currently all random defconfigs which are used in autobuilder use size
optimizaion (-Os), since BR2_OPTIMIZE_S=y is the default.
Adding "-O2" optimization will give better test coverage.
In many cases software gets built with speed optimization rather than
size optimization. So let's add Level 2 optimizaion option to be
generated in random defconfigs, so we could be able to test how
packages are built with "-O2" in autobuilder.
Signed-off-by: Evgeniy Didin <Evgeniy.Didin@synopsys.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: arc-buildroot@synopsys.com
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
When adding the custom BR2_WGET value in the configuration,
genrandconfig forgets to add a newline. Due to this, the next option
that is added is printed on the same line as BR2_WGET="", which causes
it to be ignored.
Due to this, in all builds, the line right after BR2_WGET was
ignored. It could have been BR2_ENABLE_DEBUG, BR2_INIT_BUSYBOX,
BR2_INIT_SYSTEMD, BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV,
BR2_STATIC_LIBS or BR2_PACKAGE_PYTHON_PY_ONLY depending on the
randomization.
Fix that by adding a proper newline at the end of the BR2_WGET option.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
A number of autobuilder failures are due to the fact that autobuilder
instances use old distributions, with old SSL certificates, and
therefore wget aborts with an error "The certificate of `xyz.org' is
not trusted.".
In order to avoid such failures that are not very interesting in the
context of the autobuilders, we pass --no-check-certificate to
wget. The integrity of the downloaded files is anyway verified by the
hashes, and this is only meant to be used in the context of
testing/CI, not in production.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
In commit ea9669fffa ("core: kill
DEPENDENCIES_HOST_PREREQ"), the core-dependencies make target was
removed, and is now named just "dependencies".
This broke the utils/genrandconfig script, and this commit intends to
fix that.
Since this script is part of the Buildroot tree, it is provided
together with Buildroot, so we don't need to support the legacy
core-dependencies target. Someone checking out an older Buildroot will
have a working setup, with support/dependencies exposing
core-dependencies and utils/genrandconfig using core-dependencies. The
only broken situation will be between
ea9669fffa and this commit, but that's
not a sufficient reason to add some backward compatibility code.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Fix these warnings:
E201 whitespace after '['
E202 whitespace before ']'
E501 line too long (138 > 132 characters)
-> isolate the common part of the external toolchain url in a variable
to make the long lines shorter and more readable.
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
When generating a random configuration, also check the host
dependencies for that configuration and error out if it fails.
This is mainly intended for the autobuilders, so they don't try to
build a configuration for which the necessary host packages are not
installed.
Instead of erroring out, we could also generate a different random
config. However:
- When called for the autobuilders, the autobuild-run script will
anyway just attempt another build. The only difference is that a
warning will be printed in this autobuild log now. This warning
is in fact quite useful to hint that the autobuilder could benefit
from intalling this additional host package.
- When called manually, the user may in fact want to install the
missing host package and continue with the saved random config.
Fixes:
http://autobuild.buildroot.net/results/d7c/d7c12e873a71fef1e0b5e19cfad45024050614e1http://autobuild.buildroot.net/results/a0a/a0a105c1ec2d0517112b02932a85831d4b597b9fhttp://autobuild.buildroot.net/results/96c/96cc6cedc64c777e993671204809599a1498472a
and many others...
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In preparation for the addition of comments in the CSV file listing
toolchain configurations, we filter out such lines when reading the
CSV file in utils/genrandconfig.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now we have the toolchain config fragments in the buildroot directory
itself, it is no longer necessary to fetch it from the toolchain URL.
The --toolchains-url option is renamed to --toolchains-csv.
The paths in the toolchains_csv file should be either absolute, or
relative to buildrootdir.
After this change, the script should be called from autobuild-run as:
subprocess.call([os.path.join(srcdir, "utils/genrandconfig"),
"-o", outputdir, "-b", srcdir,
"--toolchains-csv", kwargs['toolchains_csv']],
stdout=devnull, stderr=log)
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This has a number of side-effects which must be handled.
The lines in minimal.config may be overridden by the random lines added
by amending the configuration, so is_toolchain_usable() shouldn't take
those into account, or indeed the random lines added. Therefore, make
a copy of the config before appending minimal.config and the random
lines. While we're at it, rename the variable to the more appropriate
toolchainconfig.
minimal.config sets BR2_INIT_NONE=y, but we really also want to test
with BR2_INIT_BUSYBOX=y. Therefore, add a random line to use the
busybox init system. We set its probability rather high. The
probabilities of systemd and eudev are increased since they're now
in the else branch of BR2_INIT_BUSYBOX, which halves the probability
that we even get there.
We now also generate configurations without busybox. Previously,
busybox was almost always selected due to BR2_INIT_BUSYBOX=y. Only if
systemd is selected there was a chance to build without busybox.
We now set BR2_SYSTEM_BIN_SH_NONE=y, the other /bin/sh options are
never tested. However, this is not really something that is relevant
to test in the autobuilders.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When outputdir is 'output' (the default), genrandconfig didn't work
correctly because it expects the configfile in outputdir, while
Buildroot puts it in the buildroot directory.
Fix this by explicitly checking if outputdir == buildrootdir/output.
Because abspath is used for both paths, string comparison works
reliably.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The path to the .config file is calculated in several places - replace
it with a single calculation, and pass configfile as an argument
to is_toolchain_usable and fixup_config. These functions also don't
need outputdir any more.
This makes it easier to fix the case when configfile is not in
outputdir.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The --instance argument is just an artifact of genrandconfig's
history as part of autobuild-run. It is much more logical to pass
the output directory and the buildroot directory as arguments, with
sane defaults.
This also allows us to remove the hack of creating a symlink in the
instance directory if it doesn't exist yet.
Note that the default outputdir 'output' doesn't work yet, because in
that case Buildroot will put the config file in the buildroot directory
instead of the output directory. This will be fixed in a follow-up
patch.
After this change, the script should be called from autobuild-run as:
subprocess.call([os.path.join(srcdir, "utils/genrandconfig"),
"-o", outputdir, "-b", srcdir,
"--toolchains-url", kwargs['toolchains_url']],
stdout=devnull, stderr=log)
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This prepares for passing buildrootdir as an argument.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This prepares for passing outputdir as an argument.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The output of genrandconfig is currently very terse, which is annoying
for debugging the script or generally seeing what is going on. Also the
timing information added by log_write isn't very useful when the script
is used stand-alone.
In the new setup, (verbose) output goes to stdout and error output goes
to stderr. Also the "INFO: generate the configuration" message is
eliminated - it should go in the autobuild-run script.
We also add an explicit message when a toolchain can't be used after
the first defconfig, otherwise autobuild-run will just silently
restart.
Note that, since the output of make is no longer redirected to
/dev/null, we get one more message on stderr that will be recorded in
the autobuilder's log file: KCONFIG_SEED=0xXXXXXXXX.
This approach allows us to optimise the error handling to use
exceptions, where appropriate, which can be caught at the top level and
converted to an error message to stderr. This, in turn, allows us to use
subprocess.check_call, which eliminates a lot of conditions.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
