Running foo-source-check on packages retrieved from git fails.
This is because there is no associated stamp file, so we do not have a
rule-assignment that sets PKG for foo-source-check.
But it does not make sense to have a stamp file at all, because
source-check is not supposed to change anything: the status after is
exactly the same as before; nothing is downlaoded, so there is no
progress (whatsoever) to memorise.
Fix that by just defining PKG in the source-check rule definition.
Fixes#9796.
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When debugging hidden dependencies, the build order is very important.
Most notably, it is interesting to identify potential culprits.
Add a new top-level rule, show-biuld-order, that dumps all the packages
in the order they would get built.
Note that there are a few differences with show-targets:
- more packages are reported, becasue show-targets does not report
host packages that have no prompt;
- the output is line-based, because we're using $(info $(1)); getting
a single output line like show-targets would require we use an
actual command, like printf '%s ' $(1); but that takes a lot of
time, while $(info $(1)) is almost instantaneous (the time to parse
the Makefiles);
- rootfs targets are not reported.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Whitespaces were searched using the following regex:
[ ]{1,}\t
and then manually removed in most of the cases. For
xserver_xorg-server.mk, tabs before backslashes were removed.
Signed-off-by: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
As shown recently by the firejail example, it is easy to miss that a
package builds and installs binaries without actually cross-compiling
them: they are built for the host architecture instead of the target
architecture.
This commit adds a small helper script, check-bin-arch, called as a
GLOBAL_INSTRUMENTATION_HOOKS at the end of the target installation of
each package, to verify that the files installed by this package have
been built for the correct architecture.
Being called as a GLOBAL_INSTRUMENTATION_HOOKS allows the build to error
out right after the installation of the faulty package, and therefore
get autobuilder error detection properly assigned to this specific
package.
Example output with the firejail package enabled, when building for an
ARM target:
ERROR: architecture for ./usr/lib/firejail/libconnect.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firejail is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/libtrace.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/libtracelog.so is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/ftee is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/lib/firejail/faudit is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firemon is Advanced Micro Devices X86-64, should be ARM
ERROR: architecture for ./usr/bin/firecfg is Advanced Micro Devices X86-64, should be ARM
Many thanks to Yann E. Morin and Arnout Vandecappelle for their reviews
and suggestions.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Don't special case $(XZCAT) when constructing DL_TOOLS_DEPENDENCIES. The next
commit will introduce another extractor that automatically builds when not
installed. Introduce EXTRACTOR_DEPENDENCY_PRECHECKED_EXTENSIONS that lists
archive extensions for which the extractor is already checked in
support/dependencies/check-host-foo.mk. Use this in the newly introduced
extractor-dependency to populate DL_TOOLS_DEPENDENCIES.
Cc: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
[Thomas: add missing space after "firstword", as noticed by Thomas DS.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This will allow packages to define their pre-rsync hooks which will be
guaranteed to run even if the source is missing.
Signed-off-by: Tal Shorer <tal.shorer@gmail.com>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Some packages misbehave, and install files in either $(STAGING_DIR)/$(O)
or in $(TARGET_DIR)/$(O) .
One common reason for that is that pkgconf now prepends the sysroot path
to all the paths it returns. Other reasons vary, but are mostly due to
poorly writen generic-packages.
And a new step hooks to check that no file gets installed in either
location, called after the install-target and install-staging steps.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now that we can dump the reverse dependencies of a package, add the
ability to graph those.
It does not make sense to do a full reverse graph, as it would be
semantically equivalent to the direct graph. So we only provide a
per-package reverse graph.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Finding the packages that select another one in a specific configuration
is not very trivial:
- when optional, the dependency is not expressed in Kconfig
- looking at the .mk files is not very nice.
Introduce a way to dump reverse dependencies of packages, i.e. the list
of packages that directly depend on that package. Like for direct
dependencies, we limit the list to the first-order reverse dependencies.
Document it in the main help; use the opportunity to also document
foo-show-depends.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we only support at most one br2-external tree. Being able
to use more than one br2-external tree can be very useful.
A use-case would be for having a br2-external to contain the basic
packages, basic board defconfigs and board files, provided by one team
responsible for the "board-bringup", while other teams consume that
br2-external as a base, and complements it each with their own set of
packages, defconfigs and extra board files.
Another use-case would be for third-parties to provide their own
Buildroot packaging in a br2-external tree, along-side the archives for
their stuff.
Finally, another use-case is to be able to add FLOSS packages in a
br2-external tree, and proprietary packages in another. This allows
to not touch the Buildroot tree at all, and still be able to get in
compliance by providing only that br2-external tree(s) that contains
FLOSS packages, leaving aside the br2-external tree(s) with the
proprietary bits.
What we do is to treat BR2_EXTERNAL as a colon-separated (space-
separated also work, and we use that internally) list of paths, on which
we iterate to construct:
- the list of all br2-external names, BR2_EXTERNAL_NAMES,
- the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which
point each to the actual location of the corresponding tree,
- the list of paths to all the external.mk files, BR2_EXTERNAL_MKS,
- the space-separated list of absolute paths to the external trees,
BR2_EXTERNAL_DIRS.
Once we have all those variables, we replace references to BR2_EXTERNAL
with either one of those.
This cascades into how we display the list of defconfigs, so that it is
easy to see what br2-external tree provides what defconfigs. As
suggested by Arnout, tweak the comment from "User-provided configs" to
"External configs", on the assumption that some br2-external trees could
be provided by vendors, so not necessarily user-provided. Ditto the menu
in Kconfig, changed from "User-provided options" to "External options".
Now, when more than one br2-external tree is used, each gets its own
sub-menu in the "User-provided options" menu. The sub-menu is labelled
with that br2-external tree's name and the sub-menu's first item is a
comment with the path to that br2-external tree.
If there's only one br2-external tree, then there is no sub-menu; there
is a single comment that contains the name and path to the br2-external
tree.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This unique NAME is used to construct a per br2-external tree variable,
BR2_EXTERNAL_$(NAME)_PATH, which contains the path to the br2-external
tree.
This variable is available both from Kconfig (set in the Kconfig
snippet) and from the .mk files.
Also, display the NAME and its path as a comment in the menuconfig.
This will ultimately allow us to support multiple br2-external trees at
once, with that NAME (and thus BR2_EXTERNAL_$(NAME)) uniquely defining
which br2-external tree is being used.
The obvious outcome is that BR2_EXTERNAL should now no longer be used to
refer to the files in the br2-external tree; that location is now known
from the BR2_EXTERNAL_$(NAME)_PATH variable instead. This means we no
longer need to expose, and must stop from from exposing BR2_EXTERNAL as
a Kconfig variable.
Finally, this also fixes a latent bug in the pkg-generic infra, where we
would so far always refer to BR2_EXTERNAL (even if not set) to filter
the names of packages (to decide whether they are a bootloader, a
toolchain or a simple package).
Note: since the variables in the Makefile and in Kconfig are named the
same, the one we computed early on in the Makefile will be overridden by
the one in .config when we have it. Thus, even though they are set to
the same raw value, the one from .config is quoted and, being included
later in the Makefile, will take precedence, so we just re-include the
generated Makefile fragment a third time before includeing the
br2-external's Makefiles. That's unfortunate, but there is no easy way
around that as we do want the two variables to be named the same in
Makefile and Kconfig (and we can't ask the user to un-quote that variable
himself either), hence this little dirty triple-inclusion trick.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This is stored in the manifest file; it is perfectly possible to have
empty fields in there.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
By aggregating all the license files into a single big text-only file
means we have no way to use license files that are binary blobs (e.g.
pdf, rtf...).
Just do not generate that big file; if the user still wants it, it is
very easy to create it afterwards.
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, packages using target finalize hooks must remember that they
need to register their hook in TARGET_FINALIZE_HOOKS
conditionally (otherwise their hook will be triggered even if the
package is disabled).
In order to avoid this potential mistake, this commit introduces a
per-package target-finalize hook variable, in which packages can
register their target-finalize hooks, with the guarantee that they will
only be triggered if the package is enabled.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
[Thomas: rework commit log.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, when an user forgets to define a SITE, he gets some fairly
weird download failure. In order to make things easier to diagnose, this
commit adds a check in the package infrastructure that verifies that if
SOURCE has a value, SITE is not empty.
Cc: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, when the package defines no _SOURCE and no _VERSION
variables, we end up trying to download <PACKAGE>-undefined.tar.gz.
This is utterly pointless.
Change the logic to not attempt a download when neither _SOURCE
nor _VERSION is set:
- do not set an undefined _VERSION to "undefined"
- do not append an empty (or "undefined") version to the package
rawname
Consequently, for packages that have no _VERSION, the build
directory will be just the package name (which to some may look
nicer than the current "package-undefined").
Furthermore, the message trace is also a bit leaner (to the same
people that find "package" nicer than "package-undefined").
Signed-off-by: Alex Suykov <alex.suykov@gmail.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This feature consists in automatically deducing dependencies of a host
package from the dependencies of the target variant.
However, it causes some issues, and many host packages need different
dependencies than their target variants.
Now that host dependencies are explicitly set for all packages, we can
remove it.
Signed-off-by: Julien Floret <julien.floret@6wind.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add a new package variable that packages can set to specify that they
need git submodules.
Only accept this option if the download method is git, as we can not get
submodules via an http download (via wget).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Aleksandar Simeonov <aleksandar@barix.com>
Tested-by: Matt Weber <matt@thewebers.ws>
Reviewed-by: Matt Weber <matt@thewebers.ws>
Tested-By: Nicolas Cavallari <nicolas.cavallari@green-communications.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Almost all packages which are saved for legal-info have their source
archives downloaded as part of 'make source', which makes an off-line
build completely possible [0].
However, for the pre-configured external toolchains, the source tarball
is different, as the main tarball is a binary package. And that source
tarball is only downloaded during the legal-info phase, which makes it
inconvenient for full off-line builds.
We fix that by adding a new rule, $(1)-legal-source which only
$(1)-all-source depends on, so that we only download it for a top-level
'make source', not as part of the standard download mechanism (i.e. only
what is really needed to build).
This new rule depends, like the normal download mechanism, on a stamp
file, so that we do not emit a spurious hash-check message on successive
runs of 'make source'.
This way, we can do a complete [0] off-line build and are still able to
generate legal-info, while at the same time we do not incur any download
overhead during a simple build.
Also, we previously downloaded the _ACTUAL_SOURCE_TARBALL when it was
not empty. However, since _ACTUAL_SOURCE_TARBALL defaults to the value
of _SOURCE, it can not be empty when _SOURCE is not. Thus, we'd get a
spurious report of a missing hash for the tarball, since it was not in
a standard package rule (configure, build, install..) and thus would
miss the PKG and PKGDIR variables to find the .hash file.
We fix that in this commit as well, by:
- setting PKG and PKGDIR just for the -legal-source rule;
- only downloading _ACTUAL_SOURCE_TARBALL if it is not empty *and* not
the same as _SOURCE (to avoid a second report about the hash).
[0] Save for nodejs which invarriably wants to download stuff at build
time. Sigh... :-( Fixing that is work for another time...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Move the declarations of _ACTUAL_SOURCE and _ACTUAL_SITE earlier, so
that they are close to where _SOURCE and _SITE are handled.
This looks so far like a purely cosmetic change, but makes more sense
with the follow-up patch, where we'll need them earlier in the file.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Some packages, like perl, download extra files that end up as part of
the source that Buildroot builds. Up until now, those files were not
saved in the legal-info output.
Add those files to the legal-info output.
The unfortunate side-effect is that we will also save the secondary
archive for the external blackfin toolchains; however, we already do
save the binary release of some external toolchains when they do not
provide actual source archives.
This is inherently bad, as those are not source archives, but solving
this is a bigger concern, for another series...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, the legal-info infra only saves the source archive of a
package. However, that's not enough as we may apply some patches on
packages sources.
We do suggest users to also redistribute the Buildroot sources as part
of their compliance distribution, so the patches bundled in Buildroot
would indeed be included in the compliance distribution.
However, that's still not enough, since we may download some patches, or
the user may use a global patch directory. Patches in there might not
end up in the compliance distribution, and there are risks of
non-conformity.
So, always include patches alongside the source archive.
To ensure reproducibility, we also generate a series file, so patches
can be re-applied in the correct order.
We get the list of patches to include from the list of patches that were
applied by the package infrastructure (via the apply-patches support
script). So, we need to get packages properly extracted and patched
before we can save their legal-info, not just in the case they define
_LICENSE_FILES.
Update the legal-info header accordingly.
Note: this means that, when a package is not patched and defines no
LICENSE_FILES, we will extract and patch it for nothing. There is no
easy way to know whether we have to patch a package or not. We can only
either duplicate the logic to detect patches (bad) or rely on the infra
actually patching the package. Also, a vast majority of packages are
either patched, or define _LICENSE_FILES, so it is best and easiest to
always extract and patch them prior to legal-info.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Now that we save the source archives in a directory named after the
package and its version, do the same for the license files, for
consistency.
It has a not-so-bad side-effect of also saving the version string in
the all-licenses list.
The only (small) side-effect, is that the warnings about undefined
_LICENSE_FILES now contains the version string, too. That's unavoidable,
since that's what is stored in the legal report.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Add a package-variable to store the package-specific make rules.
Although this variable would be seldom used, we still document it.
However, we make sure the documentation explicitly states that this
variable should not be used (if it needs to be, the submitter of a
package will be told so during reviews).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we put all source archives side-by-side in the same
directory.
Since we're about to also save individual patches that were applied
on those sources, we don't want to make that directory a complete
mess of unassorted files.
So, we install each source archive in its own sub-directory, where
we'll later store the patches too. Store that location in a variable,
so it can be re-used later on (to install patches in a future commit).
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Acked-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Introduce a new per-package variable to store the 'rawname-version'
tuple, instead of computing it every time we need it.
Currently, it's only a single location, but follow-up patches will
introduce more use of it.
Reported-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
.. and use $(Q) instead of a hard-coded @.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This comment is outdated since the assignment of $(2)_MANIFEST_TARBALL
and $(2)_MANIFEST_SITE have been removed, which happened in
b9f4727ff4.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
[Thomas: adjust comment as suggested by Yann E. MORIN.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The ifneq ($$($(2)_SITE_METHOD),override) test will always match
because SITE_METHOD is never set to "override". The result is that
some packages are downloaded and extracted unnecessarily, although
they are (correctly) not saved in output/legal-info/.
Fix by simply testing $(2)_OVERRIDE_SRCDIR instead: when empty, it
indicates that the packages is neither a "local" site-method package
nor a package that has its source directory overridden.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Reported-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we generate the dependency graph in a single command, piping
the stdout of support/scripts/.graph-depends to the stdin of dot.
Unfortunately, this means we can't catch a failure of graph-depends, as
the shell can only treturn the exit code of the last command in a pipe.
Still, we do want to keep the output of graph-depends, and we in fact do
keep it by mean of a tee.
graph-depends has just gained the ability to generate its output to a
file, so we break the pipe in two differnet commands, so we can bail out
on graph-depends errors.
Do that for the two call sites.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, the check that packages we build are indeed enabled is done
at the time a package is configured.
This can come quite late in the build process, and does not provide
direct knowledge of the real culprit for the incorrect dependency.
However, we can improve these two issues quite easily, albeit at the
expense of a very slightly more complicated make code.
First, the check can not be done at the time we define the package, i.e.
in the inner-generic-pacakge, because all its dependencies might have
not been parsed yet, so we can't yet know whether it is enabled or not
(because we can't match the package name of the dependency to its
Kconfig variable yet).
But then, we know we have all packages definitions after we scanned the
the bundled packages, kernel, bootloaders and toolchains, as well as the
br2-external tree (if any).
So, at this location, we iterate through the list of enabled packages,
and check that the packages they each depend on are indeed enabled.
This allows us to:
1- do the check very early, before any build action,
2- report on the exact offending package very easily.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Virtual packages do not have a Kconfig symbol, so we must not check that
they are enabled.
Fixes:
http://autobuild.buildroot.org/results/287/28713478cc6edf8e5d5c3e830fee86a42f0afa8d/
... and numerous similar failures ...
[Thomas:
- use "$($(PKG)_IS_VIRTUAL)" != "YES" instead of !
"$($(PKG)_IS_VIRTUAL)" = "YES"]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
In Buildroot, the selection of a package from a Config.in level and
from a Makefile level are completely disconnected. This can lead to
issues where the build of a package is triggered at the Makefile level
due to the package being listed in another package <pkg>_DEPENDENCIES
variable, even if that package is not enabled in the configuration.
This has for example been the case recently with python-can having
'python' in its <pkg>_DEPENDENCIES, while python-can could be enabled
when Python 3.x is used, in which case the 'python' package should not
be built.
To detect such issues more easily, this patch adds a check in the
package infrastructure. When the build process of a package is being
triggered, we verify that the package is enabled in the
configuration. We do this check in the "configure" step, since this
step is the first common step between the normal download case and the
"local site method" / "package override" case.
[Thomas: adjust the error message as suggested by Yann, to take into
account the fact that the problem might not be in a direct reverse
dependency, but in something higher up in the tree of reverse
dependencies.]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Commit 5538e476 (from 2013!) removed the use of the NAMEVER variable in
.stamp_patched, but it was still defined. So remove the definition now.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When we build our host programs, and they depend on a host library we
also build, we want to ensure that program actually uses that library at
runtime, and not the one from the system.
We currently ensure that in two ways:
- we add a RPATH tag that points to our host library directory,
- we export LD_LIBRARY_PATH to point to that same directory.
With these two in place, we're pretty much confident that our host
libraries will be used by our host programs.
However, it turns our that not all the host programs we build end up
with an RPATH tag:
- some packages do not use our $(HOST_LDFLAGS)
- some packages' build system are oblivious to those LDFLAGS
In this case, there are two situations:
- the program is not linked to one of our host libraries: it in fact
does not need an RPATH tag [0]
- the program actually uses one of our host libraries: in that case it
should have had an RPATH tag pointing to the host directory.
For libraries, they only need an RPATH if they depend on another library
that is not installed in the standard library path. However, any system
library will already be in the standard library path, and any library we
install ourselves is in $(HOST_DIR)/usr/lib so already in RPATH.
We add a new support script that checks that all ELF executables have
a proper DT_RPATH (or DT_RUNPATH) tag when they link to our host
libraries, and reports those file that are missing an RPATH. If a file
missing an RPATH is an executable, the script aborts; if only libraries
are are missing an RPATH, the script does not abort.
[0] Except if it were to dlopen() it, of course, but the only program
I'm aware of that does that is openssl, and it has a correct RPATH tag.
[Peter: reworded as suggested by Arnout, fix HOT_DIR typo in comment]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Currently, we touch our stamp files before we call the step hooks.
This means a step hook can not properly fail a step, since the stamp
file exists even if the hook exits in error, thus a subsequent 'make'
would not try to redo that step.
Fix that by calling the hooks before we touch the stamp files.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Cc: Peter Seiderer <ps.report@gmx.net>
Cc: Romain Naour <romain.naour@openwide.fr>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
One of the selling points for br2-external is to provide a mean to add
new packages. However, it is not supported that a package be defined by
Buildroot and then redefined in a br2-external tree.
This situation may occur without the user noticing or even willing to
redefine the package, for example:
- br2-external is first created against a version of Buildroot
- a package (missing in Buildroot) is added to that br2-external tree
- upstream Buildroot adds this package
- user updates to the new Buildroot
In this case, the result in undefined, and we can't make any guarantee
on the result (working or not).
Add a sanity check so that a package redefinition gets caught.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, packages that need to exclude parts of the archives when
extracting (e.g. to gain space), like gcc or toolchain-external, have to
provide custom extract commands, just for the sake of adding a bunch of
--exclude directives when calling tar.
Add a new variable that packages may set, to provide a space-separated
list of patterns to exclude.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Reviewed-by: Romain Naour <romain.naour@openwide.fr>
Reviewed-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This patch adds a global instrumentation hook that collects the list
of files installed in $(TARGET_DIR) by each package, and stores this
list into a file called $(BUILD_DIR)/packages-file-list.txt. It can
later be used to determine the size contribution of each package to
the target root filesystem.
Note that in order to detect if a file installed by one package is
later overriden by another package, we calculate the md5 of installed
files and compare them at each installation of a new package.
Collecting the list of files installed by each package is done
unconditionally, as tests have shown that the performance impact of
doing this is negligible.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: Ryan Barnett <ryanbarnett3@gmail.com>
Tested-by: Ryan Barnett <ryanbarnett3@gmail.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The FOO_SITE/FOO_SOURCE variables usually point to a tarball containing
source code.
For the downloaded external toolchains this is not true, the "source"
tarball actually contains binaries. This is fine for making Buildroot
work, but for legal-info we really want to ship real source code, not
binaries.
Luckily, some (hopefully all) toolchain vendors publish a downloadable
tarball containing the source code counterpart for their binary
packages.
Here we allow the user to declare the URL of this other tarball in the
pair of variables FOO_ACTUAL_SOURCE_TARBALL (by default equal to
FOO_SOURCE) and FOO_ACTUAL_SOURCE_SITE (by default equal to FOO_SITE).
If the "actual source" package can be downloaded from the same
directory as the binary package, then only FOO_ACTUAL_SOURCE_TARBALL
needs to be set.
Note this change is not strictly toolchain-specific: it might be useful
for other packages that happen to ship binaries in the same way.
[Thomas:
- remove "the source code has not been saved" warning that could
never be triggered due to how the conditions were
organized. Discussed with Luca live during the meeting.]
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Not needed anymore since the URLs have been cleared, the manual states
they are illegal and a check has been introduced to notice future
mistakes.
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
A trailing slash in FOO_SITE is useless, since Buildroot automatically adds
a slash between FOO_SITE and the filename as appropriate.
Moreover it is potentially harmful, which led to introducing a workaround
to strip them:
commit 1cbffbd015
Author: Shawn J. Goff <shawn7400@gmail.com>
Date: Fri Apr 12 09:40:30 2013 +0000
eliminate double slashes caused by FOO_SITE ending in a slash
When a FOO_SITE variable ends in a slash and gets joined with a
FOO_SOURCE variable like $(FOO_SITE)/$(FOO_SOURCE), the resulting URI
has a double slash. While double-slashes are fine in unix paths, they
are reserved in URIs - the part following '//' must be an authority.
So let's ban trailing slashes entirely. They have all been removed in
a 7b0e757fb8, now add a check to error out loudly in case a new one
is added.
Example commands to test this check:
$ make busybox-dirclean busybox-source
rm -Rf /home/murray/devel/buildroot/output/build/busybox-1.23.2
busybox-1.23.2.tar.bz2: OK (md5: 7925683d7dd105aabe9b6b618d48cc73)
busybox-1.23.2.tar.bz2: OK (sha1: 7f37193cb249f27630e0b2a2c6c9bbb7b1d24c16)
$
$ make BUSYBOX_SITE=http://www.busybox.net/downloads/ busybox-dirclean busybox-source
rm -Rf /home/murray/devel/buildroot/output/build/busybox-1.23.2
BUSYBOX_SITE (http://www.busybox.net/downloads/) cannot have a trailing slash
make[1]: *** [/home/murray/devel/buildroot/output/build/busybox-1.23.2/.stamp_downloaded] Error 1
make: *** [_all] Error 2
$
Signed-off-by: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
When a package has both a target and a host variants, and there is an
override-srcdir set for the target variant, the host variant should
inherit the target's override-srcdir, unless explicitly set, like we do
for all other target-variant properties.
However, do not override it if expressly set to empty (i.e. when the
user wants to override only the target variant).
Also, as we do for the other variables, the target variant never
inherits from the host variant.
Reported-by: Mike <mikez@OpenPlayer.org>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Tested-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Currently, we're only fixing libtool's .la files for autotools packages,
and even more so, only for those that do not define their own
_INSTALL_STAGING_CMDS.
However, .la files might also be installed by non-autotools packages,
like the NVidia GPU binary blob (libGL.la).
So, move that code into the global build rule, so all packages get their
.la files fixed, whether they be autotools packages or not, and whether
they define their own _INSTALL_STAGING_CMDS or not.
Also print a message.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Create a proper package for the skeleton.
The main Makefile is modified to remove the skeleton support.
The 'dirs' target, will create the $(TARGET_DIR).
The file 'output/target/.root' doesn't exists anymore, as there's no
Make rule to statisfy.
The infrastructure are modified to filter host-skeleton.
It's needed becauses the host-dependencies are derived from the
dependencies of the target package where 'host-' is preprended to the
depedency name.
In the pkg-generic we add skeleton as a dependency to every package.
The whole system/system.mk is now removed at the profit of
package/skeleton/skeleton.mk
[Thomas:
- rebase on top of master and fix some minor conflicts
- remove the 'select BR2_PACKAGE_SKELETON' in
BR2_ROOTFS_SKELETON_DEFAULT and BR2_ROOTFS_SKELETON_CUSTOM, since
anyway the skeleton package is always enabled.
- fixup a few mistakes in the getty handling due to misnamed
variables.]
Signed-off-by: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reset permissions for rsynced packages (when using OVERRIDE_SRCDIR) to
755/644. We do this under the assumption that source files shouldn't
care about their permissions, except possibly for the exec bit.
This guarantees that if a package uses 'rsync -a' or 'cp -p' to copy
a file from its build dir to the target, it'll end up with the same
permissions on the target every time.
Signed-off-by: Guido Martínez <guido@vanguardiasur.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
If a package is based on "generic-package", pkg-generic.mk will compute
the name of the Kconfig variable to use for checking if this package has
been selected by the user.
Unfortunately, this mechanism does not take into account the case where
a bootloader is declared in a $(BR2_EXTERNAL)/boot directory.
So, even if the bootloader has been selected, it will not be added to
$(TARGETS) and will not be built.
This patch fixes this issue.
[Arnout: use single filter line, also apply this to toolchain]
Signed-off-by: Eric Le Bihan <eric.le.bihan.dev@free.fr>
Cc: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The introduction of <pkg>_STRIP_COMPONENTS broke the build of the
target tar package, because support/dependencies/check-host-tar.mk
defines TAR_STRIP_COMPONENTS to --strip-components. Which leads to
have the package infrastructure do:
$$(TAR_STRIP_COMPONENTS)=$$($(2)_STRIP_COMPONENTS)
which for the tar package evaluates to:
$$(TAR_STRIP_COMPONENTS)=$$(TAR_STRIP_COMPONENTS)
which evalutes to:
--strip-components=--strip-components
Which obviously doesn't work really well. And in fact the
TAR_STRIP_COMPONENTS definition in
support/dependencies/check-host-tar.mk is no longer necessary: it was
needed in the days where we were trying to support old tar versions
that did not support --strip-components. But nowadays, when such an
old tar version is encountered, we build our own host-tar which
supports --strip-components.
Fixes:
http://autobuild.buildroot.org/results/ae2/ae20df67f99f75b1ba5d5b7316ad265d66f3aa66/
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
