Commit Graph

1569 Commits (9f0acac3f7608a17489c9a40ad646dbaea8788b5)

Author SHA1 Message Date
Peter Korsgaard 71ddf1a084 Update for 2022.11.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-03-01 17:01:34 +01:00
Yann E. MORIN 2f8b83f5e4 support/download: catch post-process errors
Fixes:
http://autobuild.buildroot.net/results/12a/12a63ae177fe3ed0c9a1ef2fa01870f334f36b0f/

Currently, when the post-process helper fails while downloading from
upstream, there is no fallback to the backup mirror.

In case the post-process helper fails, we must consider that to be a
download failure, so we must bail out as if the download backend itself
did fail, but we fail to do so.

Duplicate the logic we have for the download helper: if the post-process
helper fails, remove the downloaded stuff, and continue on to the next
URI, which will ultimately hit the backup mirror (if one has been
configured).

Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit a39116b5db)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-02-28 22:12:21 +01:00
Peter Korsgaard 57a66d1f60 support/dependencies/dependencies.sh: require diagnostics perl package for mosh
Fixes:
http://autobuild.buildroot.net/results/820/820e98b1c126469b1f180f078d102ded43b9c40e/

scripts/Makefile.am of mosh-1.4.0 needs the perl diagnostics module on the host:

make[3]: Entering directory '/home/buildroot/autobuild/instance-2/output-1/build/mosh-1.4.0/scripts'
perl -Mdiagnostics -c ./mosh.pl
Can't locate diagnostics.pm in @INC (you may need to install the diagnostics module) (@INC contains: /home/buildroot/autobuild/instance-2/output-1/host/lib/perl /usr/local/lib64/perl5/5.36 /usr/local/share/perl5/5.36 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5).
BEGIN failed--compilation aborted.

So add a check for it in dependencies.sh similar to the other perl modules.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 5be2d9bb62)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-02-28 21:57:18 +01:00
Peter Korsgaard 004660e84f support/dependencies/dependencies.sh: ensure git is >= 2.0.0 for vendoring
The go vendoring fails on CentOS 7 (which uses git 1.8.3.1) with errors
related to shallow clones:

make docker-compose-source
..
go: downloading github.com/matttproud/golang_protobuf_extensions v1.0.4
github.com/docker/compose/v2/pkg/mocks imports
        github.com/theupdateframework/notary/client imports
        github.com/docker/go/canonical/json: github.com/docker/go@v1.5.1-1.0.20160303222718-d30aec9fd63c: invalid pseudo-version: git fetch --unshallow -f origin in /home/jacmet/source/buildroot-mirror/output/host/share/go-path/pkg/mod/cache/vcs/48fbd2dfabec81f4c93170677bfc89087d4bec07a2d08f6ca5ce3d17962677ee: exit status 128:
        fatal: git fetch-pack: expected shallow list
make[1]: *** [/home/jacmet/source/buildroot-mirror/output/build/docker-compose-2.15.1/.stamp_downloaded] Error 1

It works with git 2.0.0 (released May 2014, included in Debian 8), so check
for >= 2.0.0 with logic similar to the GNU patch version check.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit f229564c36)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-02-28 16:52:23 +01:00
Peter Korsgaard 1174407c5f support/dependencies/dependencies.sh: silence gcc plugin test
The gcc plugin test was not using the -q option to grep causing it to print
the line to stdout, so fix that.

While we're at it, adjust the locale check to use grep -q instead of
redirecting to /dev/null for consistency with the other checks.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 4fbd2f6d2c)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-02-28 16:28:31 +01:00
Peter Korsgaard df702ada10 support/dependencies/dependencies.sh: correct check for open perl module
Commit 4cdd99190e (support/dependencies/dependencies.sh: require open perl
package for libxcrypt) added a check for the "open" perl module for
libxcrypt, but it does not work as "open" cannot be directly used with
"require" as an argument is needed:

perl -e "require open"
Not enough arguments for open at -e line 1, at EOF
Execution of -e aborted due to compilation errors.

So special case the check to instead check with "use open ':std'".

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit b63e155e5f)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-02-23 20:03:02 +01:00
Peter Korsgaard a007e20b33 support/dependencies/dependencies.sh: require open perl package for libxcrypt
Fixes:
http://autobuild.buildroot.net/results/16c/16cba783be11cc5024f9c56bb0a7abb0acf13ef8/

The configure script of libxcrypt needs the open perl module on the host:

Can't locate open.pm in @INC (you may need to install the open module) (@INC
contains: /home/buildroot/autobuild/instance-2/output-1/host/lib/perl
/usr/local/lib64/perl5/5.36 /usr/local/share/perl5/5.36
/usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5
/usr/share/perl5) at ./build-aux/scripts/expand-selected-hashes line 20.

So add a check for it in dependencies.sh similar to how it is done for mpv.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 4cdd99190e)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-02-22 21:02:05 +01:00
Peter Korsgaard 4a17c925e0 support/dependencies/dependencies.sh: require English perl package for host-libxml-parser-perl
Fixes http://autobuild.buildroot.net/results/166/1667b4ebd8e16e7d8b47bc3ca128a76daf27b4d9/

The configure script of host-libxml-parser-perl needs the English perl
module on the host:

ERROR from evaluation of
/home/buildroot/autobuild/instance-1/output-1/build/host-libxml-parser-perl-2.46/Expat/Makefile.PL:
Can't locate English.pm in @INC (you may need to install the English module)
(@INC contains:
/home/buildroot/autobuild/instance-1/output-1/build/host-libxml-parser-perl-2.46/inc
/home/buildroot/autobuild/instance-1/output-1/host/lib/perl
/usr/local/lib64/perl5/5.36 /usr/local/share/perl5/5.36
/usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5
/usr/share/perl5 .) at ./Makefile.PL line 3.

So add a check for it in dependencies.sh similar to the existing check for
ExtUtils::MakeMaker.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 8f8085e63a)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-02-22 20:56:28 +01:00
Peter Korsgaard 579945a3ba support/dependencies/dependencies.sh: require FindBin perl package for libopenssl
Fixes:
http://autobuild.buildroot.net/results/5d1/5d10f4f545dccf126e3f5b5efce777a393c5e7bc/
http://autobuild.buildroot.net/results/7c2/7c2df31c6eb34b68e460e092d8c262bdc6c8f25a/

The configure script of (host-)libopenssl needs the FindBin perl module on the host:

Can't locate FindBin.pm in @INC (you may need to install the FindBin module)
(@INC contains: /home/buildroot/autobuild/instance-2/output-1/host/lib/perl
/usr/local/lib64/perl5/5.36 /usr/local/share/perl5/5.36
/usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5
/usr/share/perl5) at ./Configure line 15.  BEGIN failed--compilation aborted
at ./Configure line 15.

As this is needed for both libopenssl and host-libopenssl (which does not
have a corresponding config symbol in the .config), we have to require it
unconditionally.

Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 5aab8c59d5)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-02-22 20:46:56 +01:00
Ricardo Martincoski feebcb90a2 support/scripts/generate-gitlab-ci-yml: improve test-pkg support
Following the example of test-pkg config described in commit
"12c7a05da1 utils/test-pkg: add gitlab-ci support" to test a defconfig
fragment that contains a disabled option is currently possible, but
it do requires one to change the git config core.commentChart so the
lines starting with "#" are not discarded by git when creating/editing
the commit message.

For instance, without the indentation the 3rd line below would be
excluded from the commit message when the editor is closed:
    test-pkg config:
    SOME_OPTION=y
    # OTHER_OPTION is not set
    SOME_VARIABLE="some value"

Requiring to change git configs is not very nice.
So make the developer's life easier by changing the sed expression to
remove indentation with spaces from a defconfig fragment found on a
commit message.
For instance these lines become valid and generate a defconfig fragment
without the indentation of one space to be tested in GitLab CI:
test-pkg config:
 SOME_OPTION=y
 # OTHER_OPTION is not set
 SOME_VARIABLE="some value"

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 0ad3ae14c5)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-02-22 17:37:09 +01:00
Peter Korsgaard d48a8beb39 Update for 2022.11.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-01-18 09:04:16 +01:00
Yann E. MORIN d475cb6056 support/tests: print failed command and output on assertRunOK error
Currently, when asserting that a command succeeded, we just capture the
return code of the command. If that is not zero, the assertion fails,
but the error message is not very splicit:
    AssertionError: 1 != 0

Replace the error message with an explicit message that dumps the failed
command, the error code, and the resulting output.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
(cherry picked from commit 44161560dd)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-01-01 19:34:23 +01:00
Yann E. MORIN 4da09740c6 support/testing: remove leftover kernel config fragment
Commit 86d32208b6 (support/testing/tests/init/test_systemd.py: use
downloaded kernel) stopped building a custom kernel for the systemd
tests, but forgot to drop the associated kernel config fragment.

That fragment is now not used in any test case, so we can drop it.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
(cherry picked from commit 0627cb0e8a)
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2023-01-01 15:32:03 +01:00
Peter Korsgaard 40bd4a32aa Update for 2022.11
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-12-05 08:23:19 +01:00
Peter Korsgaard 12a33b54b3 Update for 2022.08.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 25680e6aa8)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-11-17 08:24:23 +01:00
Thomas Petazzoni b9dd9ee857 support/testing/tests/package/test_gdb.py: drop version-specific tests
Back when support/testing/tests/package/test_gdb was introduced, there
was a significant difference in how gdb < 10 and gdb >= 10 were
handled in gdb.mk, which explained why we were testing both gdb 9.x
and gdb 11.x.

However, support for gdb 9.x has now been dropped, and we only support
gdb >= 10.x, so testing gdb 9.x and 11.x separately no longer make
much sense. In addition:

 - other GDB tests in the same file already test the default version,
 which is now 11.x, meaning we in fact have duplicated tests between
 the ones testing the default version and the ones testing 11.x
 specifically

 - GDB 9.x has been removed, which means all the tests testing GDB 9.x
 are failing, with a Config.in.legacy build error.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828456 (TestGdbHostOnly9x)
  https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828454 (TestGdbHostGdbserver9x)
  https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828451 (TestGdbHostGdbTarget9x)

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-03 23:08:10 +01:00
Thomas Petazzoni e7930708a3 support/testing/tests/package/test_python_crossbar: use ext2 instead of cpio
The CPIO filesystem generated by the test_python_crossbar test is too
large, and doesn't fit as an initramfs in the 256MB of RAM available
in the versatilepb machine. This causes a "Initramfs unpacking failed:
write error" when booting, and many files being missing from the root
filesystem, ultimately causing the test to fail.

It would make sense to switch all test cases to use ext2 + a
hard-drive, but for now, let's fix the few test cases that are causing
problems.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828587

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-03 23:07:58 +01:00
Thomas Petazzoni 3884cfc057 support/testing/tests/package/test_python_flask*: increase time after server startup
It seems like on Gitlab CI, the runners are quite slow, and the Flask
server does not startup in the 15 seconds we give it. So increase this
to 30 seconds before trying to contact the Flask server.

Hopefully fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828594

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-03 23:07:54 +01:00
Christian Stewart 5d8371a3fe support/testing/tests/package/test_docker-compose: update kernel to 4.19.262
Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-01 19:15:01 +01:00
Christian Stewart f486521b27 support/testing/tests/package/test_docker-compose: quiet overly verbose output
Docker compose up outputs terminal control characters intended for
interactive output viewing.

Wget similarly can use the -q option to produce quieter logs.

Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-01 19:14:57 +01:00
Christian Stewart 685d8a0318 support/testing/tests/package/test_docker-compose: fix docker compose container name
Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-01 19:14:46 +01:00
Christian Stewart 2e99a233bd support/testing/tests/package/test_docker-compose: fix docker compose binary name
Docker compose is now invoked as "docker compose" not "docker-compose."

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/3249828442

Signed-off-by: Christian Stewart <christian@paral.in>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-11-01 19:14:29 +01:00
Simon Richter 04154a6517 support/download/cargo-post-process: cargo output for vendor config
Use the output of `cargo vendor` to generate the vendor configuration.

Fixes the need to patch the generated configuration if there are
non-crates.io dependencies.

Note:
  `cargo vendor` currently prints a newline before it prints the
  needed configuration.

  This is fixed in +nightly, will end up in +stable soon and must
  be considered when updating cargo.
  See: https://github.com/rust-lang/cargo/pull/11273

  Until then it is needed to remove this first line to make sure
  that the contents of .cargo/config will be the same as they were
  generated with the earlier version of the script. Thus, the
  hashes of the packages that use this script remain the same.

Signed-off-by: Simon Richter <simon.richter@ptwdosimetry.com>
[yann.morin.1998@free.fr: add comment in rust-bin.mk and rust.mk]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-10-31 10:06:17 +01:00
yann.morin@orange.com 9d948e1b34 toolchain: support gconv modules from glibc >= 2.34
Startig with glibc 2.34, the gconv modules description has been split in
two:
  - a common definition in the old location, /usr/lib/gconv/gconv-modules
  - specific definitions in a subdirectory, /usr/lib/gconv/gconv-modules.d/

This is done so as to simplify the handling of glibc gconv modules, and
eventually to segregate those outside of glibc, and so that third-parties
may also provide their own gconv converters and their definitions.

And starting with that same glibc version, most of the gconv modules
definitions are moved to an extra configuration file in that
sub-directory.

It is thus no longer possible to use special code pages, like cp850,
which are very useful to access FAT-formatted devices.

Add support for this new gconv layout, while keeping support for older
glibc versions. Note that the modules themselves are not moved or
renamed, just the definition files have changed.

Instead of passing the one old gonv modules definitions file on stdin,
we pass the base directory to that file, and move into the script the
responsibility to find all the gconv definition files.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Romain Naour <romain.naour@gmail.com>
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Cc: Giulio Benetti <giulio.benetti@benettiengineering.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-10-21 21:02:40 +02:00
yann.morin@orange.com 822cc1ebc4 support/scripts: don't require gawk to generate glibc gconv modules
When only a subset of the glibc gconv modules are installed, we need to
generate a trimmed-down list of available modules. We currently use gawk
for that.

However, we are not using any GNU extension in that awk script, and it
happens to work as expected when using mawk (which has no GNU
extension).

Commit 11c1076db9 (toolchain: add option to copy the gconv libraries)
did not explain why it used gawk explicitly, and given the age for that
commit, we doubt we'd be able to have the involved participants recall
anything from that period...

Besides, gawk is not a requirement for Buildroot.

Switch over to using plain awk.

Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-10-21 21:01:25 +02:00
José Luis Salvador Rufo 1b77c2170d support/testing/tests/package/test_zfs: increase timeout for the ZFS tests
Newest versions requires a bit more time to finish.

Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-10-06 20:46:12 +02:00
Peter Korsgaard 1648c2ec36 Update for 2022.08.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 25d865996d)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-10-03 08:49:17 +02:00
Michael Klein a54a7bf805 support/scripts/size-stats: count compiled python (.pyc) files
Any .pyc files generated by the pycompile script during target
finalization are currently counted in the "Unknown" package,
because packages-file-list.txt only contains the source .py file.

If a .py file is added to filesdict, add the corresponding .pyc
file as well.

Signed-off-by: Michael Klein <m.klein@mvz-labor-lb.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-09-24 10:34:30 +02:00
Lang Daniel 2040961b48 package/polkit: test /usr/share/polkit-1/rules.d
Polkit has two directories that are used to store rules.
Add the second directory to the existing tests, to ensure
that both work in the future.

Signed-off-by: Daniel Lang <d.lang@abatec.at>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-09-17 23:36:10 +02:00
Yann E. MORIN c25b445488 support/tests/cpio: extend runtime tests
Check that dependencies that are DT_NEEDED by a program are
automatically copied by dracut. We use cramfs, the package,
as it is small and just depends on libz.

Test more than one dracut config file.

Reported-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thierry Bultel <thierry.bultel@linatsea.fr>
Cc: Adam Duskett <aduskett@gmail.com>
[Arnout: remove test for features that haven't been merged yet]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2022-09-17 22:13:23 +02:00
Yann E. MORIN c8a618cff7 support/test/cpio: test for pv already done by listing the cpio archive
Since we do not pivot_root/switch_root from the cpio, whatever we get
in the rootfs was exactly what we got by listing the cpio archive.

Drop the test for the presence of pv, it's redundant.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thierry Bultel <thierry.bultel@linatsea.fr>
Cc: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2022-09-17 22:13:22 +02:00
Julien Olivain 680ad2d44b support/testing/tests/package/test_hwloc.py: new runtime test
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-09-17 15:06:08 +02:00
Thomas Petazzoni 43fc826d82 support/config-fragments/autobuild/bootlin-armv7m-uclibc.config: specify configuration more precisely
Due to a bug in the CodeSourcery ARM toolchain packaging, this
toolchain currently appears as available even for noMMU
configurations, which is obviously wrong. Due to this, the
bootlin-armv7m-uclibc.config fragment ends up using the CodeSourcery
ARM toolchain, which is obviously wrong for an ARM noMMU
configuration, causing a build failure when matching the toolchain
capabilities with the configuration.

Even though we will separately fix the CodeSourcery ARM toolchain
packaging, it makes sense to ensure that the
bootlin-armv7m-uclibc.config fragment explicitly selects the Bootlin
toolchain.

Reported-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-09-17 14:45:47 +02:00
Peter Korsgaard be914b97ad Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-09-11 09:57:08 +02:00
Peter Korsgaard 0003fdbed3 Update for 2022.08
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-09-10 17:00:09 +02:00
Marcus Hoffmann 54e5a64200 support/testing/tests/package/test_python_paho_mqtt: new runtime test
This tests valdates that we can publish a message and read it back.

Signed-off-by: Marcus Hoffmann <marcus.hoffmann@othermo.de>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[yann.morin.1998@free.fr:
  - don't manually start mosquitto, there's a startup script for that
  - don't pass custom timeout
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-31 21:37:50 +02:00
Peter Korsgaard 41b1b65c5a Update for 2022.05.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit bf0d8c9659)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2022-08-31 21:06:05 +02:00
Arnout Vandecappelle 6c7061f3a8 support/scripts/pkg-stats: fix flake8 errors
Fixes flake8 errors:
support/scripts/pkg-stats:1013:133: E501 line too long (164 > 132 characters)
support/scripts/pkg-stats:1018:36: F541 f-string is missing placeholders
support/scripts/pkg-stats:1110:199: E261 at least two spaces before inline comment

For the first and the last one, we chose to split the long lines rather
than adding noqa: 501. Indeed, the long lines make it very unreadable,
and there are relatively natural places where the line can be broken.
Also split a line just below the second one in a similar way.

The f-string on 1018 doesn't need to be an f-string.

Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
2022-08-27 23:02:38 +02:00
Thomas Petazzoni d68628a538 support/scripts/pkg-stats: remove remaining double quote escaping
This is done either by switching to single quoted f-strings, triple
double quoted f-strings when needed, or simply single-quoted strings.

The renderer HTML is exactly identical before/after this commit.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-By: Sen Hastings <sen@phobosdpl.com>
Acked-By: Sen Hastings <sen@phobosdpl.com>
2022-08-24 13:36:12 +02:00
Thomas Petazzoni 98a3fba940 support/scripts/pkg-stats: remove useless escaping of double quotes
Within single-quoted f-strings, and within triple double quoted
strings, escaping all the double quotes is completely useless and
makes the code more difficult to read. Get rid of all this useless
escaping.

The renderer HTML is exactly identical before/after this commit.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-By: Sen Hastings <sen@phobosdpl.com>
Acked-By: Sen Hastings <sen@phobosdpl.com>
2022-08-24 13:35:49 +02:00
James Hilliard fc429c4dc6 package/pkg-python: clean conflicting pep517 packages before install
The python installer package isn't able to overwrite files of packges
that already exist, this causes problems when doing a rebuild or
update without a full clean.

To fix this we can use functionality from importlib to identify and
remove any conflicting python package files before installation.

We also need to use internals from python-installer, as we want to use
the same logic as pyinstaller uses internally for getting the scheme so
that we ensure we clean the correct package scheme (we want it to be the
same as the one we're installing)

Fixes:
Traceback (most recent call last):
  File "/home/buildroot/buildroot/support/scripts/pyinstaller.py", line 69, in <module>
    main()
  File "/home/buildroot/buildroot/support/scripts/pyinstaller.py", line 61, in main
    install(
  File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/_core.py", line 109, in install
    record = destination.write_file(
  File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/destinations.py", line 207, in write_file
    return self.write_to_fs(scheme, path_, stream, is_executable)
  File "/home/buildroot/buildroot/output/host/lib/python3.10/site-packages/installer/destinations.py", line 167, in write_to_fs
    raise FileExistsError(message)
FileExistsError: File already exists: /home/buildroot/buildroot/output/target/usr/lib/python3.10/site-packages/tinycss2/__init__.py

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Tested-by: Marcus Hoffmann <marcus.hoffmann@othermo.de>
[yann.morin.1998@free.fr:
  - extend commit log about the use of the installer internals (the
    symbols prefixed with '_')
  - check path.files against explicitly None
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-22 11:54:25 +02:00
James Hilliard d752f73b0c package/pkg-python: use pyinstaller.py for host python packages
The python installer cli isn't able to overwrite files of packages
that already exist, this causes problems when doing a rebuild or
update without a full clean.

Since we need to add functionality to our pyinstaller.py script to fix
this issue we must also use pyinstaller.py for host python packages.

Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Tested-by: Marcus Hoffmann <marcus.hoffmann@othermo.de>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-22 11:39:27 +02:00
Fabrice Fontaine ba2659401f support/dependencies: check for find and xargs
Commit 7652817c93 updated the
documentation but forgot to update support/dependencies

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-21 15:50:15 +02:00
Thierry Bultel bd0b2db231 support/testing/tests/fs/test_cpio.py: new runtime test
It includes a simple test for the full cpio image, and a test of the
dracut image. To validate that the dracut image is a subset of the full
image, 'pv' is added to the image, and the test verifies that pv is not
part of the image. Note that the real rootfs is not mounted at the
moment, so pv is never available in the running image.

Systemd and other init systems are currently untested.

Signed-off-by: Thierry Bultel <thierry.bultel@linatsea.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-18 22:49:53 +02:00
Thomas Petazzoni 7ad5e6cede support/testing/tests/package/test_python_s3transfer: use ext2 instead of cpio
The CPIO filesystem generated by the test_python_s3transfer test is
too large, and doesn't fit as an initramfs in the 256MB of RAM
available in the versatilepb machine. This causes a "Initramfs
unpacking failed: write error" when booting, and many files being
missing from the root filesystem, ultimately causing the test to fail.

It would make sense to switch all test cases to use ext2 + a
hard-drive, but for now, let's fix the few test cases that are causing
problems.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/2884635126

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
  - drop superfluous# BR2_TARGET_ROOTFS_TAR is not set
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-16 22:56:46 +02:00
Thomas Petazzoni 0813ec1aa0 support/testing/tests/package/test_python_botocore: use ext2 instead of cpio
The CPIO filesystem generated by the test_python_botocore test is too
large, and doesn't fit as an initramfs in the 256MB of RAM available
in the versatilepb machine. This causes a "Initramfs unpacking failed:
write error" when booting, and many files being missing from the root
filesystem, ultimately causing the test to fail.

It would make sense to switch all test cases to use ext2 + a
hard-drive, but for now, let's fix the few test cases that are causing
problems.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/2884635042

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
  - drop superfluous# BR2_TARGET_ROOTFS_TAR is not set
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-16 22:55:50 +02:00
Thomas Petazzoni a9df206190 support/testing/tests/package/test_python_boto3: use ext2 instead of cpio
The CPIO filesystem generated by the test_python_boto3 test is too
large, and doesn't fit as an initramfs in the 256MB of RAM available
in the versatilepb machine. This causes a "Initramfs unpacking failed:
write error" when booting, and many files being missing from the root
filesystem, ultimately causing the test to fail.

It would make sense to switch all test cases to use ext2 + a
hard-drive, but for now, let's fix the few test cases that are causing
problems.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/2884635041

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
  - drop superfluous# BR2_TARGET_ROOTFS_TAR is not set
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-16 22:55:19 +02:00
Thomas Petazzoni 0214ee94e0 support/testing/tests/fs/test_f2fs: fix test after f2fs-tools bump
In commit
9267b0f14d ("package/f2fs-tools: bump to
version 1.15.0"), f2fs-tools was bumped from 1.14.0 to 1.15.0.

It turns out that this version bump causes the output of dump.f2fs to
slightly change.

In version 1.14.0, it looked like this:

Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors = 262144 (128 MB)
Info: MKFS version
  "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022"
Info: FSCK version
  from "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 20
22"
    to "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 20
22"
Info: superblock features = 0 :
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
Info: total FS sectors = 262144 (128 MB)
Info: CKPT version = 70c101c3
Info: checkpoint state = 181 :  trimmed nat_bits unmount

In version 1.15.0, it looked like this:

Info: MKFS version
  "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022"
Info: FSCK version
  from "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 20
22"
    to "Linux version 5.4.0-124-generic (buildd@lcy02-amd64-089) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 20
22"
Info: superblock features = 0 :
Info: superblock encrypt level = 0, salt = 00000000000000000000000000000000
Info: Segments per section = 1
Info: Sections per zone = 1
Info: total FS sectors = 262144 (128 MB)
Info: CKPT version = b89f8bb
Info: checkpoint state = 181 :  trimmed nat_bits unmount

You will notice that the message "Info: total sectors = 262144 (128
MB)" is no longer present, and only "Info: total FS sectors =
262144 (128 MB)" is not present.

Except our test case was precisely looking for this "Info: total
sectors" string in the output, causing the test to fail.

We fix this by simply matching on "Info: total FS sectors" now.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/2884634814

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2022-08-16 18:08:02 +02:00
Julien Olivain 4b291f1bac support/testing: octave: increase test timeout
Octave package test can occasionally fail due to timeout while testing
some octave modules. This commit slightly increase the timeout value
to reduce those failures.

Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-08-08 23:07:23 +02:00
Emile Cormier bfafb08c62 package/python-crossbar: adjust dependencies based on requirements-min.txt
This commit makes sure that the python-crossbar package pulls in the
right dependencies, based on the requirements-min.txt.

It does so by:

 - Changing the 0002-Remove-idna-requirement patch by a more thorough
   patch that drops all indirect dependencies from
   requirements-min.txt, making it easier to have a 1:1 mapping
   between lines in requirements-min.txt and Buildroot selects.

 - Changing the
   0003-crossbar-webservice-wap-use-markupsafe-instead-of-we patch to
   update requirements-min.txt to indicate the new MarkupSafe
   dependency. Here again, to have a 1:1 mapping between lines in
   requirements-min.txt and Buildroot selects.

 - Updating the Buildroot selects to match requirements-min.txt, with
   relevant comments when it does not.

 - Fixing up the Crossbar test case to no longer force autobahn to use
   umsgpack. Instead, we now use the default of msgpack that is
   expected by autobahn.

Fixes bug #14556, https://bugs.busybox.net/show_bug.cgi?id=14556.

Signed-off-by: Emile Cormier <emile.cormier.jr@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2022-08-08 22:27:35 +02:00