Commit graph

696 commits

Author SHA1 Message Date
Yann E. MORIN 3ab56f8a63 support/dependencies: check for system-provided bison and flex
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-08-20 16:01:42 +02:00
Ricardo Martincoski 5f201de0dc support/testing: add test for file capabilities
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-08-10 08:25:51 +02:00
Damien Thébault d8c0faa980 support/misc/toolchainfile.cmake.in: use TARGET_LDFLAGS for shared and module libraries
With cmake packages, we are only using TARGET_LDFLAGS for executables
and not for shared libraries.

This patch adds CMAKE_SHARED_LINKER_FLAGS and
CMAKE_MODULE_LINKER_FLAGS to the cmake toolchain file so that
buildroot TARGET_LDFLAGS are used for shared and module libraries.

Signed-off-by: Damien Thébault <damien.thebault@vitec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-07-19 09:33:37 +02:00
David De Grave (Essensium/Mind) d0505b849f support/dependencies/dependencies.sh: add details for Redhat/Fedora
Add the i686 package list to install when using pre-built 32 bits
binaries with a redhat/fedora host distribution (glibc.i686 and
zlib.i686).

Signed-off-by: David De Grave (Essensium/Mind) <david.degrave@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-07-02 22:43:56 +02:00
Thomas Petazzoni edca2cd086 support/config-fragments/autobuild: update pre-built Buildroot toolchains
All pre-built Buildroot toolchains have been rebuilt with Buildroot
2018.05, so this commit updates the corresponding configuration
fragments to make sure the autobuilders use the new toolchains.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-06-30 18:00:30 +02:00
Ofer Heifetz 78968330e3 support/config-fragments/autobuild: add Linaro AArch64 BE support
Signed-off-by: Ofer Heifetz <oferh@marvell.com>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-06-28 22:15:57 +02:00
Yann E. MORIN 567a8476ad support/docker: use the distro-provided flake8
Currently, we install flake8 and its dependencies via pip. We
tried to be reproducible by pinning the version of those python
packages, but we did forget quite a few of them, and thus some
dependencies for flake8 are installed as uncontrolled versions.

Furthermore, before we install flake8 and its dependencies, we
forcibly update pip, setuptools, and wheels packages to their
latest versions. This explicitly breaks reproducibility.

While we could enforce a specific version of all those packages
and still grab them from PyPI, we can simply grab them from the
distribution-provided packages instead.

Since we're using a pinned version of stretch, this already
guarantees we'll reproducibly get the same versions over and
over again. Besides, we just need to list flake8 as a package to
install to automatically get all its dependencies (again, in a
reproducible way).

This has the slight unfortunate drawback of downgrading flake8
to version 3.2.1, from version 3.5.0, as well as downgrading a
few of flake8's dependencies, as noticed by Ricardo:
    http://lists.busybox.net/pipermail/buildroot/2018-May/222376.html

However, as Ricardo said, there isn't "any serious limitation of
this old version, the release notes for a version in the between
mentions 'Dramatically improve the performance' but we have a
limited number of scripts and running on Gitlab for all of them
still takes less than 5 minutes".

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-06-17 20:54:02 +02:00
Yann E. MORIN afc1ec7073 support/docker: sort the list of installed packages
As suggested in the docker best practices [0], order the package list
alphabetically, and list only one package per line.

This will be much usefull later, we need to update the list of installed
packages, like adding new ones for example.

[0] https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#sort-multi-line-arguments

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-06-17 20:53:53 +02:00
Yann E. MORIN 99723554c0 support/testing: fix python syntax
Fix three issues with code style in our test infra:
  - 'print' is now a function,
  - exceptions need to be caught-assigned with the 'as' keyword,
  - old-style "%s"%() formatting is deprecated.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Thomas: drop indices in format strings.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-06-10 15:56:25 +02:00
Fabrice Fontaine dc5553d2ce dependencies.sh: add perl autodie for whois
Add dependency on perl autodie if whois package is selected

Fixes:
 - http://autobuild.buildroot.net/results/4d2ca14580cbc73f94e86ee834fac6d2541298ec

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-06-05 22:20:18 +02:00
Robert Beckett a5ba72946e dl-wrapper: Fix support for URIs containing '+'
'+' is a valid character in a url. The current dl-wrapper gets the
URI scheme by dropping everything after the last '+' character, with
the intension of finding 'git' from e.g. 'git+https://uri'.

If a uri has a '+' anywhere in it, it ends up using too much of the
string as a scheme, and fails to match the handler properly.

An example of where this form of URI is used is when using deploy tokens
in gitlab. It uses a form like https://<username>:<password>@gitlab.com/<group>/<repo.git>
where username for deploy token is of the form 'gitlab+deploy-token-<number>'.

Use the %% operator to search backwards until the last '+' character when
dropping the rest of the string as we know that the first '+'
in the string should be the scheme.

Signed-off-by: Robert Beckett <bbeckett@netvu.org.uk>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-06-04 21:59:55 +02:00
Yann E. MORIN c4f844ff32 support/docker: run apt-get update and apt-get install in two RUNs
In commit 7517aef4d (support/docker: limit the number of layers),
we reduced the number of layers by coalescing multiple RUN commands
into less commands.

In doing so, we especially coalesced "apt-get update" with "apt-get
install".

However, the distribution we used is a pinned version of stretch, so
we know that running apt-get update will always yield the same apt
database.

If we split the two apt-get commands, then we can re-use any local
intermediate image when we need to update the list of packages to
install; this helps quite a bit when testing the docker files over
and over again, with just slight variants in the packages list.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-06-03 20:31:20 +02:00
Peter Korsgaard 8b0fd3cb49 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-06-02 11:21:20 +02:00
Peter Korsgaard f3d114a1ef Update for 2018.05
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-06-01 22:22:57 +02:00
Arnout Vandecappelle (Essensium/Mind) 66f40a7242 download/cvs: add a 10 minute timeout
Apparently, CVS servers can be deadlocked and in that case clients will
retry connecting to them indefinitely. Cfr.
http://autobuild.buildroot.net/results/23d/23d1034b33d0354de15de2ec4a8ccd0603e8db78/build-end.log
Apparently, the sf.net CVS server got in such a deadlock on 2018-05-18,
and almost 2 weeks later it is still not fixed.

Instead of just hanging, we should fall back on BR2_SECONDARY_SITE. To
achieve this, it's sufficient to add a timeout to the CVS command.

The timeout value is of course arbitrary. However, we can assume that
nobody will be putting large projects under CVS any more. So if the
download takes more than 5 minutes, it's probably broken. Let's put the
timeout at 10 minutes then.

Fixes:
http://autobuild.buildroot.net/results/db3/db33d4fa507fb3b4132423cd0a7e25a1fe6e4105
http://autobuild.buildroot.net/results/b6d/b6d927dcc73ac8d754422577dacefff4ff918a5c
http://autobuild.buildroot.net/results/23d/23d1034b33d0354de15de2ec4a8ccd0603e8db78
http://autobuild.buildroot.net/results/127/1272a3aa3077e434c9805ec3034f35e6fcc330d4

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-05-31 19:32:45 +02:00
Matt Weber eb04235783 support/scripts/pkg-stats: use parenthesis for print
Use Python 3 style print calls, in order to make pkg-stats Python 3
compliant.

Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-19 13:48:21 +02:00
Angelo Compagnucci 43ca1987e6 support/download/file: remove set -x
Signed-off-by: Angelo Compagnucci <angelo@amarulasolutions.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-13 22:26:43 +02:00
Angelo Compagnucci a9bb361435 support/download/file: fix file:// protocol handling
Since the rework of the download infrastructure, the "file" download
helper gets passed an URL that starts with file://, but forgets to
strip it before passing it to "cp", causing a failure as the "cp"
program isn't prepared for file paths starting with file://. This is
fixed by stripping the file:// at the beginning of the URL.

In addition, the path passed to cp lacked a slash between the
directory path and the filename part of the url. This is fixed by
adding a slash at the appropriate places.

Fixes the following build failure when the "file" download method is
used:

cp: cannot stat 'file:///home/angelo/DEV/TOOLCHAINSarmv7-eabihf--glibc--bleeding-edge-2017.11-1.tar.bz2': No such file or directory

Signed-off-by: Angelo Compagnucci <angelo@amarulasolutions.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-13 22:23:13 +02:00
Petr Vorel 04918ca72f support/kconfig: Add missing target to README.buildroot
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-13 22:09:01 +02:00
Yann E. MORIN 72703d02b9 support/dependencies: check that PATH does not contain CWD
A person on IRC reported a build failure with the util-linux package,
looking like this:

for I in uname26 linux32 linux64        ; do \
	cd /home/aep/consulting/chargery/tracker/output/target/usr/bin && ln -sf setarch $I ; \
done
[...]
/bin/sh: line 1: ./ln: cannot execute binary file: Exec format error
/bin/sh: line 1: ./ln: cannot execute binary file: Exec format error
/bin/sh: line 1: ./ln: cannot execute binary file: Exec format error

The issue was an empty path in the PATH variable, which means "current
working directory", causing a "ln" binary built by util-linux for the
target to be used instead of the system-provided "ln".

We already check a number of things in the PATH and LD_LIBRARY_PATH
variables in support/dependencies/dependencies.sh, but we were not
checking that PATH did not contain an empty path.

This commit fixes that and takes this opportunity to simplify the test
code for PATH and LD_LIBRARY_PATH.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Thomas: improve commit log.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-09 23:26:18 +02:00
Yann E. MORIN 4f54c959dc download/git: always do full-clone
We currently attempt a shallow clone, as tentative to save bandwidth and
download time.

However, now that we keep the git tree as a cache, it may happen that we
need to checkout an earlier commit, and that would not be present with a
shallow clone.

Furthermore, the shallow fetch is already really broken, and just
happens to work by chance. Consider the following actions, which are
basically what happens today:

    mkdir git
    git init git
    cd git
    git remote add origin https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
    git fetch origin --depth 1 v4.17-rc1
    if ! git fetch origin v4.17-rc1:v4.17-rc1 ; then
        echo "warning"
    fi
    git checkout v4.17-rc1

The checkout succeeds just because of the git-fetch in the if-condition,
which is initially there to fetch the special refs from github PRs, or
gerrit reviews. That fails, but we just print a warning. If we were to
ever remove support for special refs, then the checkout would fail.

The whole purpose of the git cache is to actually save bandwidth and
download time, but in the long run. For one-offs, people would
preferably use a wget download (e.g. with the github macro) instead of
a git clone.

We switch to always doing a full clone. It is more correct, and pays off
in the long run...

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-01 21:22:28 +02:00
Yann E. MORIN 80d8bc6e46 download/git: ensure we can checkout repos with submodule conversions
When a git tree has had sub-dir <-> sub-module conversions, or has had
submodules added or removed over the course of time, checking out a
changeset across those conversions/additions/removals may leave
untracked files, or may fail because of a conflict of type.

So, before we checkout the new changeset, we forcibly remove the
submodules. The new set of submodules, if any, will be restored later.

Ideally, we would use a native git command: git submodule deinit --all.
However, that was only introduced in git 1.8.3 which, while not being
recent by modern standards, is still too old for some enterprise-grade
distributions (RHEL6 only has git-1.7.1).

So, instead, we just use git submodule foreach, to rm -rf the submodules
directory.

Again, we would ideally use 'cd $toplevel && rm -rf $path', but
$toplevel was only introduced in git 1.7.2. $path has always been there.

So, instead, we just cd back one level, and remove the basename of the
directory.

Eventually, we need to get rid of now-empty and untracked directories,
that were parents of a removed submodule. For example. ./foo/bar/ was a
submodule, so ./foo/bar/ was removed, which left ./foo/ around.

Yet again, recent-ish git versions would have removed it during the
forced checkout, but old-ish versions (e.g. 1.7.1) do not remove it with
the forced checkout.

Instead we rely on the already used forced-forced clean of directories,
untracked, and ignored content, to really get rid of extra stuff we are
not interested in.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-01 21:22:28 +02:00
Yann E. MORIN 3d2e018889 download/git: ensure we checkout to a clean state
Force the checkout to ignore and throw away any local changes. This
allows recovering from a previous partial checkout (e.g. killed by
the user, or by a CI job...)

git checkout -f has been supported since the inception of git, so we
can use it without any second thought.

Also do a forced-forced clean, to really get rid of all untracked stuff.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-01 21:22:28 +02:00
Yann E. MORIN b7efb43e86 download/git: try to recover from utterly-broken repositories
In some cases, the repository may be in a state we can't automatically
recover from, especially since we must still support oldish git versions
that do not provide the necessary commands or options thereof.

As a last-ditch recovery, delete the repository and recreate the cache
from scratch.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-01 21:22:28 +02:00
Yann E. MORIN 428a0649ff download/git: quickly exit when the cset does not exist
Check that the given cset is indeed something we can checkout. If not,
then exit early.

This will be useful when a later commit will trap any failing git
command to try to recover the repository by doing a clone from scratch:
when the cset is not a commit, it does not mean the repository is broken
or what, and re-cloning from scratch would not help, so no need to trash
a good cache.

Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-01 21:22:28 +02:00
Yann E. MORIN 577315687f download/git: run all git commands in the current directory
That way, we can pushd earlier, which will help with last-ditch recovery
in a followup commit.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-01 21:22:28 +02:00
Yann E. MORIN f1eb192e26 download/git: add warning not to use our git cache
We really want the user not to use our git cache manually, or their
changes (committed or not) may eventually get lost.

So, add a warning file, not unlike the one we put in the target/
directory, to warn the user not to use the git tree.

Ideally, we would have carried this file in support/misc/, but the git
backend does not have access to it: the working directory is somewhere
unknown, and TOPDIR is not exported in the environment.

So, we have to carry it in-line in the backend instead.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-01 21:22:28 +02:00
Eric Le Bihan 09a5eb427a support/testing: set $USER in rust tests
When the run-time tests to build rust and rust-bin packages are run via Docker,
the $USER environment variable is not set, which makes cargo fail when
initializing the test project.

So add it to make cargo happy.

Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-29 09:57:32 +02:00
Thomas Petazzoni ea7f7aaa6d support/scripts: drop xorg-release script
This script causes a large number of flake8 warnings, is rarely used
(but even never used), and is going to be replaced at some point by
the improved pkg-stats that will give details about the upstream
version available for all packages, not just X.org packages.

Therefore, let's drop the xorg-release script in order to silence all
those flake8 warnings:

support/scripts/xorg-release:36:1: E302 expected 2 blank lines, found 1
support/scripts/xorg-release:58:27: E201 whitespace after '{'
support/scripts/xorg-release:58:44: E203 whitespace before ':'
support/scripts/xorg-release:58:54: E202 whitespace before '}'
support/scripts/xorg-release:63:1: E305 expected 2 blank lines after class or function definition, found 1
support/scripts/xorg-release:64:15: E261 at least two spaces before inline comment
support/scripts/xorg-release:67:32: E261 at least two spaces before inline comment
support/scripts/xorg-release:86:1: E302 expected 2 blank lines, found 1
support/scripts/xorg-release:95:1: E302 expected 2 blank lines, found 1
support/scripts/xorg-release:107:1: E302 expected 2 blank lines, found 1
support/scripts/xorg-release:115:20: W601 .has_key() is deprecated, use 'in'
support/scripts/xorg-release:123:34: E201 whitespace after '{'
support/scripts/xorg-release:124:46: E203 whitespace before ':'
support/scripts/xorg-release:124:50: E202 whitespace before '}'
support/scripts/xorg-release:127:1: E302 expected 2 blank lines, found 1
support/scripts/xorg-release:141:15: W601 .has_key() is deprecated, use 'in'
support/scripts/xorg-release:146:21: W601 .has_key() is deprecated, use 'in'
support/scripts/xorg-release:176:1: E305 expected 2 blank lines after class or function definition, found 1
support/scripts/xorg-release:180:1: W391 blank line at end of file

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-27 22:50:36 +02:00
Thomas Petazzoni ac260a2ace support/testing: fix Marvell ATF source code
The version of the ARM Trusted Firmware from Marvell was a Git branch,
not a Git commit, leading to unreproducible results. So let's use a
Git commit instead, which is the latest available from the branch that
was previously used.

More specifically, this branch has recently seen a fix that is needed
for ATF to build properly with recent gcc versions:

  c96ec59f8b

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-25 21:28:47 +02:00
Thomas Petazzoni e74a7cd1e0 support/testing: fix ATF Vexpress test case
This test case currently fails to build with:

./build/juno/release/bl1/context_mgmt.o: In function `cm_prepare_el3_exit':
context_mgmt.c:(.text.cm_prepare_el3_exit+0x54): undefined reference to `cm_set_next_context'
context_mgmt.c:(.text.cm_prepare_el3_exit+0x54): relocation truncated to fit: R_AARCH64_JUMP26 against undefined symbol `cm_set_next_context'

This issue has been fixed upstream in commit
10c252c14b7f446c0b49ef1aafbd5d37804577dd, available since v1.3. So
while we bump, let's bump to the latest version of ATF, v1.5.

Fixes:

  https://gitlab.com/buildroot.org/buildroot/-/jobs/64360659

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-25 21:28:18 +02:00
Thomas Petazzoni 34d32e911f support/scripts/check-uniq-files: remove csv module import
Since commit 5563a1c6a4
("support/check-uniq-files: support weird locales and filenames"), the
'csv' Python module is no longer used by the check-uniq-files.

Due to this, flake8 complains with:

support/scripts/check-uniq-files:4:1: F401 'csv' imported but unused

Fix this by dropping the useless csv import.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-25 21:19:22 +02:00
Thomas Petazzoni 8469054fff support/config-fragments/autobuild: update ARC pre-built toolchain
The toolchain has been rebuilt to fix the RPATH of the binutils programs.

Fixes:

  http://autobuild.buildroot.net/results/b2562b05d397d4e1ffe0f8d2f4ce4c84ab6feae1/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-22 12:28:41 +02:00
Thomas Petazzoni 473819bbdc support/config-fragments/autobuild: update OpenRISC pre-built toolchain
The toolchain has been rebuilt after commit
7fb6fe883a, which adds more fcntl()
definitions in uClibc for OpenRISC.

Fixes:

  http://autobuild.buildroot.net/results/664e7ee968e46b7ac0054ce4d5d4f9a82201550e/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-22 12:28:40 +02:00
Thomas Petazzoni 04a052d908 support/config-fragments/autobuild: update all Buildroot pre-built toolchains
This commit updates all the pre-built Buildroot toolchains, which have
all been rebuilt with Buildroot as of commit 046c5e2. The initial
motivation for this update is that an upcoming bump of procps-ng uses
fopencookie(), which has only been introduced in musl 1.1.19, which
itself started being used in Buildroot after the 2018.02 release.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-21 14:52:56 +02:00
Yann E. MORIN 6e2f5d0254 download/git: fix transform regexp for older tar versions
Older versions of tar (e.g. 1.27.1) incorrectly interpret the escaping
of the regexp separator, and generate broken tarballs.

For example, given the following transform expression:
    --transform="s/^\.\//squashfs-e38956b92f738518c29734399629e7cdb33072d3\//"

the resulting paths in the generated tarball would be:
    squashfs-e38956b92f738518c29734399629e7cdb33072d3\/

i.e. a directory which last character is indeed a '\'.

We fix that by using a separator which is very unlikely to occur in a
filename.

Fixes:
    http://autobuild.buildroot.org/results/742/7427f34e5c9f6d043b0fe6ad2c66cc0f31d2b24f/

and probably a slew of others as well...

Take this opportunity to fix indentation on the following line
(leading spaces, not TABs).

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-20 10:48:27 +02:00
Ricardo Martincoski 41a7c72c21 support/testing: add test for syslog-ng
Check:
 - the daemon is started;
 - a message is collected;
 - the daemon does not issue a warning message on startup.

When the .conf file version does not match the package version a warning
message shows up on serial on every boot. This message is generated by
syslog-ng before it is running, so it is not logged to
/var/log/messages. So in order to test the message is generated, restart
the server. It makes the message appears on /var/log/messages (since the
server is already running) where its existence can be easily tested
using grep.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@datacom.ind.br>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-19 23:16:15 +02:00
Yann E. MORIN e2b71e852d download/git: be sure we do fetch something
The different versions of git will behave in different ways when
fetching remote references, as summarised by the table below:

              |     ancient git         | new git
--------------------------------------------------------------------
git fetch     | fetch all refs but tags | fetches all refs but tags
git fetch -t  | fetches only tags       | fetch all refs and tags

(git-fetch may still fetch tags, but only if reachable from a branch)

So, to cover all the bases, we do a simple fetch, to be sure we have
branches, followed by the existing fetch -t, to get extra tags.

Fixes:
    http://autobuild.buildroot.net/results/0a2/0a238a7f55ea56c33b639ad03ed5796143426889/build-end.log

Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-19 23:10:24 +02:00
Yann E. MORIN d48a1b0d5e download/git: ensure we have a sane repository
There are cases where a repository might be broken, e.g. when a previous
operation was killed or otherwise failed unexpectedly.

We fix that by always initialising the repository, as suggested by
Ricardo. git-init is safe on an otherwise-healthy repository:

    Running git init in an existing repository is safe. It will not
    overwrite things that are already there. [...]

Using git-init will just ensure that we have the strictly required files
to form a sane tree. Any blob that is still missing would get fetched
later on.

Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-19 22:32:28 +02:00
Yann E. MORIN a07da16e78 download/git: ensure we always work in the expected repository
git always look directories up until it finds a repository. In case
the git cache is broken, it may no longer be identified as a repository,
and git will look higher in the directories until it finds one.

In the default conditions, this would be Buildroot's own git tree
(because DL_DIR is a subdir of Buildroot), but in some situations may
very well be any repository the user has Buildroot in, like a
br2-external tree...

So, we force git to use our git cache and never look elsewhere, as
Suggested by Ricardo.

Use GIT_DIR, as it has been there for ages now, while --git-dir was
only introduced later (even if most distros ship an later version),
as suggested by Arnout.

Also fix the one call to git that was not using the wrapper.

Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-19 22:30:50 +02:00
Thomas Petazzoni d13c3d178c support/config-fragments/autobuild: remove Blackfin configuration
In preparation for the removal of the Blackfin architecture, drop the
autobuilder toolchain configuration that was testing Blackfin.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-15 22:03:18 +02:00
Thomas Petazzoni 2679444980 support/download/dl-wrapper: pass the correct -N option
${raw_name} is never defined in dl-wrapper, and therefore the value
passed to the -N option is always empty. This causes a problem for the
'cvs' backend, which uses the value of this option as the CVS module
to be downloaded.

If the name of the CVS module is omitted, all the CVS modules from
that CVS repository are downloaded, which creates a tarball with a lot
more contents, and the actual useful contents in a sub-directory,
obviously breaking patches that should be applied, and the entire
build process that follows.

Fixes:

  http://autobuild.buildroot.net/results/fcee0e3d7eeeb373313b1794092c729b1b052348/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-12 22:54:00 +02:00
Thomas Petazzoni 0cf077a209 support/download/dl-wrapper: fix passing remaining options to helper scripts
When calling the backend-specific helper scripts, the remaining
options are in ${@}. However, in order to let the helper script know
that those remaining options should not be parsed, but instead passed
as-is to the download tool, they must be separated from the main
options by "--".

Without this, packages that use <pkg>_DL_OPTS, such as the
amd-catalyst package, cannot download their tarball anymore.

Fixes:

  http://autobuild.buildroot.net/results/de818f6e4c8e63d5e8a49c445d10c34eccc40410/

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-12 22:53:55 +02:00
Thomas Petazzoni 184cb52f6d support/scripts/fix-rpath: exclude /lib/firmware in the target
The /lib/firmware directory contains random firmware for various
devices. It happens that some of them might be or appear to be ELF
files, but they shouldn't be checked by fix-rpath. For example, one of
the Qualcomm VPU firmware file appears to be an ELF file, but patchelf
isn't happy about it:

$ ./output/host/bin/patchelf --print-rpath output/target/lib/firmware/qcom/venus-4.2/venus.b00
patchelf: patchelf.cc:387: void ElfFile<Elf_Ehdr, Elf_Phdr, Elf_Shdr, Elf_Addr, Elf_Off, Elf_Dyn, Elf_Sym>::parse() [with Elf_Ehdr = Elf32_Ehdr; Elf_Phdr = Elf32_Phdr; Elf_Shdr = Elf32_Shdr; Elf_Addr = unsigned int; Elf_Off = unsigned int; Elf_Dyn = Elf32_Dyn; Elf_Sym = Elf32_Sym]: Assertion `shstrtabIndex < shdrs.size()' failed.
Aborted (core dumped)

Even though patchelf definitely shouldn't crash, it anyway doesn't
make sense to check ELF files in /lib/firmware, so let's exclude this
directory from our check.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-12 21:15:49 +02:00
Yann E. MORIN 6dfaa33d13 download/git: more resilient in case of kill
In case the git backend gets killed right in-between it finished
initialising the repository, but before it could add the remote,
we'd end up with a repository without the 'origin' remote, so we
would not be able to change its URL.

Another case that may happen (like in the build failure, below),
is that the repository was initialised with a previous version
of Buildroot, before the commit e17719264b (download/git: don't
require too-recent git) was applied, and that trepository was
still lying around...

Fixes:
    http://autobuild.buildroot.org/results/25a/25aae054634368fadb265b97ebe4dda809deff6f/

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-10 09:28:43 +02:00
Yann E. MORIN e17719264b download/git: don't require too-recent git
git has supported -C only since 1.8.5, and some distros have not yet
caught up after more than 4 years...

Fall back to entering the directory.

Fixes:
    http://autobuild.buildroot.net/results/35f9f7a4adc6c2cad741079e4afdf1408c94703b

Reported-by: André Hentschel <nerv@dawncrow.de>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: André Hentschel <nerv@dawncrow.de>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-04-08 21:46:36 +02:00
Yann E. MORIN 54785851ee download/git: fix transform-name
When a package contains a relative symlink which first component is '..'
(thus pointing one directory higher), for example package 'meh' contains
this symlink:

    foo/bar -> ../buz

then it would be stored as 'meh-version./buz' because of the
transform-name pattern replacement.

Fix it to only match the leading './'.

Reported-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Tested-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-04-08 17:40:16 +02:00
Yann E. MORIN 585c5dc35a core/download fix local backend
Since c8ef0c03b0 (download: put most of the infra in dl-wrapper), the
backend for local files is now named after the scheme, which is 'file'
for a local file.

>From the same commit on, the directory part and the basename are now
passed separately, to let the backend reconstruct the full path when it
needs to do so, which is the case for the 'file' backend too.

Finaly, ff559846fd (support/download: Add support to pass options
directly to downloaders) introduced a nasty error, as it made use of
"${@}" when calling its internal function. Revert that mess now...

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-04-06 12:43:47 +02:00
Ricardo Martincoski 573ea2c7d4 download/git: fix basename for files inside tarballs
Commit "6d938bcb52 download: git: introduce cache feature" introduced a
typo that makes the tarball to contain files without the package
basename:
$ tar -tvf good-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
-rw-r--r-- 0/0               8 2017-10-14 02:10 ./file

Historically, all tarballs are generated with the basename:
$ tar -tvf good-a238b1dfcd825d47d834af3c5223417c8411d90d.tar.gz
-rw-r--r-- 0/0               8 2017-10-14 02:10 good-a238b1dfcd825d47d834af3c5223417c8411d90d/file

The hashes in the tree were calculated with the basename.

In the most common scenario, after the download ends the tarball is
generated, the hash mismatches and the download mechanism falls back to
use the tarball from http://sources.buildroot.net .

The problem can be reproduced by forcing the download of any git package
PKG that has a hash file to check against:
$ make defconfig
$ ./utils/config --set-str BR2_BACKUP_SITE ""
$ BR2_DL_DIR=$(mktemp -d) make PKG-dirclean PKG-source

Fix the typo so the basename is really added to the files, that was
clearly the intention of the code.

Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-04-05 08:15:45 +02:00
Thomas Petazzoni f955a4eac0 support/scripts/pkg-stats: replace with new Python version
Rename pkg-stats-new to pkg-stats.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-04 22:05:33 +02:00