2009-11-08 10:37:36 -07:00
|
|
|
################################################################################
|
|
|
|
|
# Generic package infrastructure
|
|
|
|
|
#
|
|
|
|
|
# This file implements an infrastructure that eases development of
|
2012-04-17 08:45:20 -06:00
|
|
|
# package .mk files. It should be used for packages that do not rely
|
|
|
|
|
# on a well-known build system for which Buildroot has a dedicated
|
|
|
|
|
# infrastructure (so far, Buildroot has special support for
|
|
|
|
|
# autotools-based and CMake-based packages).
|
2009-11-08 10:37:36 -07:00
|
|
|
#
|
|
|
|
|
# See the Buildroot documentation for details on the usage of this
|
|
|
|
|
# infrastructure
|
|
|
|
|
#
|
|
|
|
|
# In terms of implementation, this generic infrastructure requires the
|
|
|
|
|
# .mk file to specify:
|
|
|
|
|
#
|
2014-07-24 12:07:02 -06:00
|
|
|
# 1. Metadata information about the package: name, version,
|
2009-11-08 10:37:36 -07:00
|
|
|
# download URL, etc.
|
|
|
|
|
#
|
|
|
|
|
# 2. Description of the commands to be executed to configure, build
|
|
|
|
|
# and install the package
|
|
|
|
|
################################################################################
|
|
|
|
|
|
2013-11-11 08:03:27 -07:00
|
|
|
################################################################################
|
|
|
|
|
# Helper functions to catch start/end of each step
|
|
|
|
|
################################################################################
|
|
|
|
|
|
|
|
|
|
# Those two functions are called by each step below.
|
|
|
|
|
# They are responsible for calling all hooks defined in
|
|
|
|
|
# $(GLOBAL_INSTRUMENTATION_HOOKS) and pass each of them
|
|
|
|
|
# three arguments:
|
|
|
|
|
# $1: either 'start' or 'end'
|
|
|
|
|
# $2: the name of the step
|
|
|
|
|
# $3: the name of the package
|
|
|
|
|
|
|
|
|
|
# Start step
|
|
|
|
|
# $1: step name
|
|
|
|
|
define step_start
|
|
|
|
|
$(foreach hook,$(GLOBAL_INSTRUMENTATION_HOOKS),$(call $(hook),start,$(1),$($(PKG)_NAME))$(sep))
|
|
|
|
|
endef
|
|
|
|
|
|
|
|
|
|
# End step
|
|
|
|
|
# $1: step name
|
|
|
|
|
define step_end
|
|
|
|
|
$(foreach hook,$(GLOBAL_INSTRUMENTATION_HOOKS),$(call $(hook),end,$(1),$($(PKG)_NAME))$(sep))
|
|
|
|
|
endef
|
|
|
|
|
|
2013-11-11 08:03:28 -07:00
|
|
|
#######################################
|
|
|
|
|
# Actual steps hooks
|
|
|
|
|
|
|
|
|
|
# Time steps
|
|
|
|
|
define step_time
|
|
|
|
|
printf "%s:%-5.5s:%-20.20s: %s\n" \
|
2018-09-21 07:31:16 -06:00
|
|
|
"$$(date +%s.%N)" "$(1)" "$(2)" "$(3)" \
|
2013-11-11 08:03:28 -07:00
|
|
|
>>"$(BUILD_DIR)/build-time.log"
|
|
|
|
|
endef
|
|
|
|
|
GLOBAL_INSTRUMENTATION_HOOKS += step_time
|
|
|
|
|
|
core: check host executables have appropriate RPATH
When we build our host programs, and they depend on a host library we
also build, we want to ensure that program actually uses that library at
runtime, and not the one from the system.
We currently ensure that in two ways:
- we add a RPATH tag that points to our host library directory,
- we export LD_LIBRARY_PATH to point to that same directory.
With these two in place, we're pretty much confident that our host
libraries will be used by our host programs.
However, it turns our that not all the host programs we build end up
with an RPATH tag:
- some packages do not use our $(HOST_LDFLAGS)
- some packages' build system are oblivious to those LDFLAGS
In this case, there are two situations:
- the program is not linked to one of our host libraries: it in fact
does not need an RPATH tag [0]
- the program actually uses one of our host libraries: in that case it
should have had an RPATH tag pointing to the host directory.
For libraries, they only need an RPATH if they depend on another library
that is not installed in the standard library path. However, any system
library will already be in the standard library path, and any library we
install ourselves is in $(HOST_DIR)/usr/lib so already in RPATH.
We add a new support script that checks that all ELF executables have
a proper DT_RPATH (or DT_RUNPATH) tag when they link to our host
libraries, and reports those file that are missing an RPATH. If a file
missing an RPATH is an executable, the script aborts; if only libraries
are are missing an RPATH, the script does not abort.
[0] Except if it were to dlopen() it, of course, but the only program
I'm aware of that does that is openssl, and it has a correct RPATH tag.
[Peter: reworded as suggested by Arnout, fix HOT_DIR typo in comment]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-11-13 14:48:51 -07:00
|
|
|
# This hook checks that host packages that need libraries that we build
|
|
|
|
|
# have a proper DT_RPATH or DT_RUNPATH tag
|
|
|
|
|
define check_host_rpath
|
|
|
|
|
$(if $(filter install-host,$(2)),\
|
core: implement per-package SDK and target
This commit implements the core of the move to per-package SDK and
target directories. The main idea is that instead of having a global
output/host and output/target in which all packages install files, we
switch to per-package host and target directories, that only contain
their explicit dependencies.
There are two main benefits:
- Packages will now see only the dependencies they explicitly list in
their <pkg>_DEPENDENCIES variable, and the recursive dependencies
thereof.
- We can support top-level parallel build properly, because a package
only "sees" its own host directory and target directory, isolated
from the build of other packages that can happen in parallel.
It works as follows:
- A new output/per-package/ directory is created, which will contain
one sub-directory per package, and inside it, a "host" directory
and a "target" directory:
output/per-package/busybox/target
output/per-package/busybox/host
output/per-package/host-fakeroot/target
output/per-package/host-fakeroot/host
This output/per-package/ directory is PER_PACKAGE_DIR.
- The global TARGET_DIR and HOST_DIR variable now automatically point
to the per-package directory when PKG is defined. So whenever a
package references $(HOST_DIR) or $(TARGET_DIR) in its build
process, it effectively references the per-package host/target
directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it
is handled as well.
- Of course, packages have dependencies, so those dependencies must
be installed in the per-package host and target directories. To do
so, we simply rsync (using hard links to save space and time) the
host and target directories of the direct dependencies of the
package to the current package host and target directories.
We only need to take care of direct dependencies (and not
recursively all dependencies), because we accumulate into those
per-package host and target directories the files installed by the
dependencies. Note that this only works because we make the
assumption that one package does *not* overwrite files installed by
another package.
This is done for "extract dependencies" at the beginning of the
extract step, and for "normal dependencies" at the beginning of the
configure step.
This is basically enough to make per-package SDK and target work. The
only gotcha is that at the end of the build, output/target and
output/host are empty, which means that:
- The filesystem image creation code cannot work.
- We don't have a SDK to build code outside of Buildroot.
In order to fix this, this commit extends the target-finalize step so
that it starts by populating output/target and output/host by
rsync-ing into them the target and host directories of all packages
listed in the $(PACKAGES) variable. It is necessary to do this
sequentially in the target-finalize step and not in each
package. Doing it in package installation means that it can be done in
parallel. In that case, there is a chance that two rsyncs are creating
the same hardlink or directory at the same time, which makes one of
them fail.
This change to per-package directories has an impact on the RPATH
built into the host binaries, as those RPATH now point to various
per-package host directories, and no longer to the global host
directory. We do not try to rewrite such RPATHs during the build as
having such RPATHs is perfectly fine, but we still need to handle two
fallouts from this change:
- The check-host-rpath script, which verifies at the end of each
package installation that it has the appropriate RPATH, is modified
to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is
a correct RPAT.
- The fix-rpath script, which mungles the RPATH mainly for the SDK
preparation, is modified to rewrite the RPATH to not point to
per-package directories. Indeed the patchelf --make-rpath-relative
call only works if the RPATH points to the ROOTDIR passed as
argument, and this ROOTDIR is the global host directory. Rewriting
the RPATH to not point to per-package host directories prior to
this is an easy solution to this issue.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 09:46:40 -07:00
|
|
|
$(if $(filter end,$(1)),support/scripts/check-host-rpath $(3) $(HOST_DIR) $(PER_PACKAGE_DIR)))
|
core: check host executables have appropriate RPATH
When we build our host programs, and they depend on a host library we
also build, we want to ensure that program actually uses that library at
runtime, and not the one from the system.
We currently ensure that in two ways:
- we add a RPATH tag that points to our host library directory,
- we export LD_LIBRARY_PATH to point to that same directory.
With these two in place, we're pretty much confident that our host
libraries will be used by our host programs.
However, it turns our that not all the host programs we build end up
with an RPATH tag:
- some packages do not use our $(HOST_LDFLAGS)
- some packages' build system are oblivious to those LDFLAGS
In this case, there are two situations:
- the program is not linked to one of our host libraries: it in fact
does not need an RPATH tag [0]
- the program actually uses one of our host libraries: in that case it
should have had an RPATH tag pointing to the host directory.
For libraries, they only need an RPATH if they depend on another library
that is not installed in the standard library path. However, any system
library will already be in the standard library path, and any library we
install ourselves is in $(HOST_DIR)/usr/lib so already in RPATH.
We add a new support script that checks that all ELF executables have
a proper DT_RPATH (or DT_RUNPATH) tag when they link to our host
libraries, and reports those file that are missing an RPATH. If a file
missing an RPATH is an executable, the script aborts; if only libraries
are are missing an RPATH, the script does not abort.
[0] Except if it were to dlopen() it, of course, but the only program
I'm aware of that does that is openssl, and it has a correct RPATH tag.
[Peter: reworded as suggested by Arnout, fix HOT_DIR typo in comment]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2015-11-13 14:48:51 -07:00
|
|
|
endef
|
|
|
|
|
GLOBAL_INSTRUMENTATION_HOOKS += check_host_rpath
|
|
|
|
|
|
2016-10-24 10:38:52 -06:00
|
|
|
define step_check_build_dir_one
|
|
|
|
|
if [ -d $(2) ]; then \
|
|
|
|
|
printf "%s: installs files in %s\n" $(1) $(2) >&2; \
|
|
|
|
|
exit 1; \
|
|
|
|
|
fi
|
|
|
|
|
endef
|
|
|
|
|
|
|
|
|
|
define step_check_build_dir
|
|
|
|
|
$(if $(filter install-staging,$(2)),\
|
|
|
|
|
$(if $(filter end,$(1)),$(call step_check_build_dir_one,$(3),$(STAGING_DIR)/$(O))))
|
|
|
|
|
$(if $(filter install-target,$(2)),\
|
|
|
|
|
$(if $(filter end,$(1)),$(call step_check_build_dir_one,$(3),$(TARGET_DIR)/$(O))))
|
|
|
|
|
endef
|
|
|
|
|
GLOBAL_INSTRUMENTATION_HOOKS += step_check_build_dir
|
|
|
|
|
|
2013-11-11 08:03:29 -07:00
|
|
|
# User-supplied script
|
2014-09-14 19:31:11 -06:00
|
|
|
ifneq ($(BR2_INSTRUMENTATION_SCRIPTS),)
|
2013-11-11 08:03:29 -07:00
|
|
|
define step_user
|
|
|
|
|
@$(foreach user_hook, $(BR2_INSTRUMENTATION_SCRIPTS), \
|
2014-03-10 14:51:33 -06:00
|
|
|
$(EXTRA_ENV) $(user_hook) "$(1)" "$(2)" "$(3)"$(sep))
|
2013-11-11 08:03:29 -07:00
|
|
|
endef
|
|
|
|
|
GLOBAL_INSTRUMENTATION_HOOKS += step_user
|
|
|
|
|
endif
|
|
|
|
|
|
2019-11-05 09:46:42 -07:00
|
|
|
#######################################
|
|
|
|
|
# Helper functions
|
|
|
|
|
|
|
|
|
|
# Make sure .la files only reference the current per-package
|
|
|
|
|
# directory.
|
|
|
|
|
|
|
|
|
|
# $1: package name (lower case)
|
|
|
|
|
# $2: staging directory of the package
|
|
|
|
|
ifeq ($(BR2_PER_PACKAGE_DIRECTORIES),y)
|
|
|
|
|
define fixup-libtool-files
|
|
|
|
|
$(Q)find $(2)/usr/lib* -name "*.la" | xargs --no-run-if-empty \
|
package/pkg-generic.mk, support/scripts/fix-rpath: fix per-package regexp
Commit c4e6d5c8be6ada8e7c60950e3b499c55d48761cb ("core: implement
per-package SDK and target") had a mistake on the regexp that is used
to match $(PER_PACKAGE_DIR)/<something>/, and due to this, the regexp
was never matched.
The + sign in [^/]+ which was suggested by Yann E. Morin during the
review of the per-package patch series (instead of [^/]*) needs to be
escaped to be taken into account correctly. Without this, the regexp
doesn't match, and the replacement is not done, causing:
(1) For the libtool fixup in pkg-generic.mk, the lack of replacement
causes libtool .la files to not be tweaked as expected, which it
turn causes build failures reported by the autobuilder.
(2) For the fix-rpath, the RPATH of host binaries in the SDK were not
correct.
Interestingly, we have the same regexp in
support/scripts/check-host-rpath, but here the + sign does not need to
be escaped.
Fixes:
http://autobuild.buildroot.net/results/d4d996f3923699e266afd40cc7180de0f7257d99/ (libsvg-cairo)
http://autobuild.buildroot.net/results/56330f86872f67a2ce328e09b4c7b12aa835a432/ (bind)
http://autobuild.buildroot.net/results/9e0fc42d2c9f856b92954b08019b83ce668ef289/ (ibrcommon)
and probably a number of other similar issues
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-12-11 05:25:00 -07:00
|
|
|
$(SED) "s:$(PER_PACKAGE_DIR)/[^/]\+/:$(PER_PACKAGE_DIR)/$(1)/:g"
|
2019-11-05 09:46:42 -07:00
|
|
|
endef
|
|
|
|
|
endif
|
|
|
|
|
|
2020-04-30 03:52:44 -06:00
|
|
|
# Functions to collect statistics about installed files
|
|
|
|
|
|
|
|
|
|
# $(1): base directory to search in
|
|
|
|
|
# $(2): suffix of file (optional)
|
|
|
|
|
define pkg_size_before
|
|
|
|
|
cd $(1); \
|
|
|
|
|
LC_ALL=C find . -not -path './$(STAGING_SUBDIR)/*' \( -type f -o -type l \) -printf '%T@:%i:%#m:%y:%s,%p\n' \
|
|
|
|
|
| LC_ALL=C sort > $($(PKG)_DIR)/.files-list$(2).before
|
|
|
|
|
endef
|
|
|
|
|
|
|
|
|
|
# $(1): base directory to search in
|
|
|
|
|
# $(2): suffix of file (optional)
|
|
|
|
|
define pkg_size_after
|
|
|
|
|
cd $(1); \
|
|
|
|
|
LC_ALL=C find . -not -path './$(STAGING_SUBDIR)/*' \( -type f -o -type l \) -printf '%T@:%i:%#m:%y:%s,%p\n' \
|
|
|
|
|
| LC_ALL=C sort > $($(PKG)_DIR)/.files-list$(2).after
|
|
|
|
|
LC_ALL=C comm -13 \
|
|
|
|
|
$($(PKG)_DIR)/.files-list$(2).before \
|
|
|
|
|
$($(PKG)_DIR)/.files-list$(2).after \
|
|
|
|
|
| sed -r -e 's/^[^,]+/$($(PKG)_NAME)/' \
|
|
|
|
|
> $($(PKG)_DIR)/.files-list$(2).txt
|
|
|
|
|
rm -f $($(PKG)_DIR)/.files-list$(2).before
|
|
|
|
|
rm -f $($(PKG)_DIR)/.files-list$(2).after
|
|
|
|
|
endef
|
|
|
|
|
|
|
|
|
|
define check_bin_arch
|
|
|
|
|
support/scripts/check-bin-arch -p $($(PKG)_NAME) \
|
|
|
|
|
-l $($(PKG)_DIR)/.files-list.txt \
|
|
|
|
|
$(foreach i,$($(PKG)_BIN_ARCH_EXCLUDE),-i "$(i)") \
|
|
|
|
|
-r $(TARGET_READELF) \
|
|
|
|
|
-a $(BR2_READELF_ARCH_NAME)
|
|
|
|
|
endef
|
|
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
################################################################################
|
|
|
|
|
# Implicit targets -- produce a stamp file for each step of a package build
|
|
|
|
|
################################################################################
|
|
|
|
|
|
|
|
|
|
# Retrieve the archive
|
|
|
|
|
$(BUILD_DIR)/%/.stamp_downloaded:
|
2018-04-28 14:53:04 -06:00
|
|
|
@$(call step_start,download)
|
core: implement per-package SDK and target
This commit implements the core of the move to per-package SDK and
target directories. The main idea is that instead of having a global
output/host and output/target in which all packages install files, we
switch to per-package host and target directories, that only contain
their explicit dependencies.
There are two main benefits:
- Packages will now see only the dependencies they explicitly list in
their <pkg>_DEPENDENCIES variable, and the recursive dependencies
thereof.
- We can support top-level parallel build properly, because a package
only "sees" its own host directory and target directory, isolated
from the build of other packages that can happen in parallel.
It works as follows:
- A new output/per-package/ directory is created, which will contain
one sub-directory per package, and inside it, a "host" directory
and a "target" directory:
output/per-package/busybox/target
output/per-package/busybox/host
output/per-package/host-fakeroot/target
output/per-package/host-fakeroot/host
This output/per-package/ directory is PER_PACKAGE_DIR.
- The global TARGET_DIR and HOST_DIR variable now automatically point
to the per-package directory when PKG is defined. So whenever a
package references $(HOST_DIR) or $(TARGET_DIR) in its build
process, it effectively references the per-package host/target
directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it
is handled as well.
- Of course, packages have dependencies, so those dependencies must
be installed in the per-package host and target directories. To do
so, we simply rsync (using hard links to save space and time) the
host and target directories of the direct dependencies of the
package to the current package host and target directories.
We only need to take care of direct dependencies (and not
recursively all dependencies), because we accumulate into those
per-package host and target directories the files installed by the
dependencies. Note that this only works because we make the
assumption that one package does *not* overwrite files installed by
another package.
This is done for "extract dependencies" at the beginning of the
extract step, and for "normal dependencies" at the beginning of the
configure step.
This is basically enough to make per-package SDK and target work. The
only gotcha is that at the end of the build, output/target and
output/host are empty, which means that:
- The filesystem image creation code cannot work.
- We don't have a SDK to build code outside of Buildroot.
In order to fix this, this commit extends the target-finalize step so
that it starts by populating output/target and output/host by
rsync-ing into them the target and host directories of all packages
listed in the $(PACKAGES) variable. It is necessary to do this
sequentially in the target-finalize step and not in each
package. Doing it in package installation means that it can be done in
parallel. In that case, there is a chance that two rsyncs are creating
the same hardlink or directory at the same time, which makes one of
them fail.
This change to per-package directories has an impact on the RPATH
built into the host binaries, as those RPATH now point to various
per-package host directories, and no longer to the global host
directory. We do not try to rewrite such RPATHs during the build as
having such RPATHs is perfectly fine, but we still need to handle two
fallouts from this change:
- The check-host-rpath script, which verifies at the end of each
package installation that it has the appropriate RPATH, is modified
to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is
a correct RPAT.
- The fix-rpath script, which mungles the RPATH mainly for the SDK
preparation, is modified to rewrite the RPATH to not point to
per-package directories. Indeed the patchelf --make-rpath-relative
call only works if the RPATH points to the ROOTDIR passed as
argument, and this ROOTDIR is the global host directory. Rewriting
the RPATH to not point to per-package host directories prior to
this is an easy solution to this issue.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 09:46:40 -07:00
|
|
|
$(call prepare-per-package-directory,$($(PKG)_FINAL_DOWNLOAD_DEPENDENCIES))
|
2014-05-05 07:04:20 -06:00
|
|
|
$(foreach hook,$($(PKG)_PRE_DOWNLOAD_HOOKS),$(call $(hook))$(sep))
|
2009-11-08 10:37:36 -07:00
|
|
|
# Only show the download message if it isn't already downloaded
|
2015-04-26 03:51:11 -06:00
|
|
|
$(Q)for p in $($(PKG)_ALL_DOWNLOADS); do \
|
2018-04-02 07:09:26 -06:00
|
|
|
if test ! -e $($(PKG)_DL_DIR)/`basename $$p` ; then \
|
2015-03-29 11:33:18 -06:00
|
|
|
$(call MESSAGE,"Downloading") ; \
|
|
|
|
|
break ; \
|
|
|
|
|
fi ; \
|
|
|
|
|
done
|
2019-04-15 13:47:26 -06:00
|
|
|
$(foreach p,$($(PKG)_ALL_DOWNLOADS),$(call DOWNLOAD,$(p),$(PKG))$(sep))
|
2011-07-05 13:54:11 -06:00
|
|
|
$(foreach hook,$($(PKG)_POST_DOWNLOAD_HOOKS),$(call $(hook))$(sep))
|
2009-11-08 10:37:36 -07:00
|
|
|
$(Q)mkdir -p $(@D)
|
2018-04-28 14:53:04 -06:00
|
|
|
@$(call step_end,download)
|
2009-11-08 10:37:36 -07:00
|
|
|
$(Q)touch $@
|
|
|
|
|
|
core/legal-info: ensure legal-info works in off-line mode
Almost all packages which are saved for legal-info have their source
archives downloaded as part of 'make source', which makes an off-line
build completely possible [0].
However, for the pre-configured external toolchains, the source tarball
is different, as the main tarball is a binary package. And that source
tarball is only downloaded during the legal-info phase, which makes it
inconvenient for full off-line builds.
We fix that by adding a new rule, $(1)-legal-source which only
$(1)-all-source depends on, so that we only download it for a top-level
'make source', not as part of the standard download mechanism (i.e. only
what is really needed to build).
This new rule depends, like the normal download mechanism, on a stamp
file, so that we do not emit a spurious hash-check message on successive
runs of 'make source'.
This way, we can do a complete [0] off-line build and are still able to
generate legal-info, while at the same time we do not incur any download
overhead during a simple build.
Also, we previously downloaded the _ACTUAL_SOURCE_TARBALL when it was
not empty. However, since _ACTUAL_SOURCE_TARBALL defaults to the value
of _SOURCE, it can not be empty when _SOURCE is not. Thus, we'd get a
spurious report of a missing hash for the tarball, since it was not in
a standard package rule (configure, build, install..) and thus would
miss the PKG and PKGDIR variables to find the .hash file.
We fix that in this commit as well, by:
- setting PKG and PKGDIR just for the -legal-source rule;
- only downloading _ACTUAL_SOURCE_TARBALL if it is not empty *and* not
the same as _SOURCE (to avoid a second report about the hash).
[0] Save for nodejs which invarriably wants to download stuff at build
time. Sigh... :-( Fixing that is work for another time...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-07 10:14:38 -06:00
|
|
|
# Retrieve actual source archive, e.g. for prebuilt external toolchains
|
|
|
|
|
$(BUILD_DIR)/%/.stamp_actual_downloaded:
|
2018-04-28 14:53:04 -06:00
|
|
|
@$(call step_start,actual-download)
|
2019-04-15 13:47:26 -06:00
|
|
|
$(call DOWNLOAD,$($(PKG)_ACTUAL_SOURCE_SITE)/$($(PKG)_ACTUAL_SOURCE_TARBALL),$(PKG))
|
core/legal-info: ensure legal-info works in off-line mode
Almost all packages which are saved for legal-info have their source
archives downloaded as part of 'make source', which makes an off-line
build completely possible [0].
However, for the pre-configured external toolchains, the source tarball
is different, as the main tarball is a binary package. And that source
tarball is only downloaded during the legal-info phase, which makes it
inconvenient for full off-line builds.
We fix that by adding a new rule, $(1)-legal-source which only
$(1)-all-source depends on, so that we only download it for a top-level
'make source', not as part of the standard download mechanism (i.e. only
what is really needed to build).
This new rule depends, like the normal download mechanism, on a stamp
file, so that we do not emit a spurious hash-check message on successive
runs of 'make source'.
This way, we can do a complete [0] off-line build and are still able to
generate legal-info, while at the same time we do not incur any download
overhead during a simple build.
Also, we previously downloaded the _ACTUAL_SOURCE_TARBALL when it was
not empty. However, since _ACTUAL_SOURCE_TARBALL defaults to the value
of _SOURCE, it can not be empty when _SOURCE is not. Thus, we'd get a
spurious report of a missing hash for the tarball, since it was not in
a standard package rule (configure, build, install..) and thus would
miss the PKG and PKGDIR variables to find the .hash file.
We fix that in this commit as well, by:
- setting PKG and PKGDIR just for the -legal-source rule;
- only downloading _ACTUAL_SOURCE_TARBALL if it is not empty *and* not
the same as _SOURCE (to avoid a second report about the hash).
[0] Save for nodejs which invarriably wants to download stuff at build
time. Sigh... :-( Fixing that is work for another time...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-07 10:14:38 -06:00
|
|
|
$(Q)mkdir -p $(@D)
|
2018-04-28 14:53:04 -06:00
|
|
|
@$(call step_end,actual-download)
|
core/legal-info: ensure legal-info works in off-line mode
Almost all packages which are saved for legal-info have their source
archives downloaded as part of 'make source', which makes an off-line
build completely possible [0].
However, for the pre-configured external toolchains, the source tarball
is different, as the main tarball is a binary package. And that source
tarball is only downloaded during the legal-info phase, which makes it
inconvenient for full off-line builds.
We fix that by adding a new rule, $(1)-legal-source which only
$(1)-all-source depends on, so that we only download it for a top-level
'make source', not as part of the standard download mechanism (i.e. only
what is really needed to build).
This new rule depends, like the normal download mechanism, on a stamp
file, so that we do not emit a spurious hash-check message on successive
runs of 'make source'.
This way, we can do a complete [0] off-line build and are still able to
generate legal-info, while at the same time we do not incur any download
overhead during a simple build.
Also, we previously downloaded the _ACTUAL_SOURCE_TARBALL when it was
not empty. However, since _ACTUAL_SOURCE_TARBALL defaults to the value
of _SOURCE, it can not be empty when _SOURCE is not. Thus, we'd get a
spurious report of a missing hash for the tarball, since it was not in
a standard package rule (configure, build, install..) and thus would
miss the PKG and PKGDIR variables to find the .hash file.
We fix that in this commit as well, by:
- setting PKG and PKGDIR just for the -legal-source rule;
- only downloading _ACTUAL_SOURCE_TARBALL if it is not empty *and* not
the same as _SOURCE (to avoid a second report about the hash).
[0] Save for nodejs which invarriably wants to download stuff at build
time. Sigh... :-( Fixing that is work for another time...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-07 10:14:38 -06:00
|
|
|
$(Q)touch $@
|
|
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
# Unpack the archive
|
|
|
|
|
$(BUILD_DIR)/%/.stamp_extracted:
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_start,extract)
|
2009-11-08 10:37:36 -07:00
|
|
|
@$(call MESSAGE,"Extracting")
|
core: implement per-package SDK and target
This commit implements the core of the move to per-package SDK and
target directories. The main idea is that instead of having a global
output/host and output/target in which all packages install files, we
switch to per-package host and target directories, that only contain
their explicit dependencies.
There are two main benefits:
- Packages will now see only the dependencies they explicitly list in
their <pkg>_DEPENDENCIES variable, and the recursive dependencies
thereof.
- We can support top-level parallel build properly, because a package
only "sees" its own host directory and target directory, isolated
from the build of other packages that can happen in parallel.
It works as follows:
- A new output/per-package/ directory is created, which will contain
one sub-directory per package, and inside it, a "host" directory
and a "target" directory:
output/per-package/busybox/target
output/per-package/busybox/host
output/per-package/host-fakeroot/target
output/per-package/host-fakeroot/host
This output/per-package/ directory is PER_PACKAGE_DIR.
- The global TARGET_DIR and HOST_DIR variable now automatically point
to the per-package directory when PKG is defined. So whenever a
package references $(HOST_DIR) or $(TARGET_DIR) in its build
process, it effectively references the per-package host/target
directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it
is handled as well.
- Of course, packages have dependencies, so those dependencies must
be installed in the per-package host and target directories. To do
so, we simply rsync (using hard links to save space and time) the
host and target directories of the direct dependencies of the
package to the current package host and target directories.
We only need to take care of direct dependencies (and not
recursively all dependencies), because we accumulate into those
per-package host and target directories the files installed by the
dependencies. Note that this only works because we make the
assumption that one package does *not* overwrite files installed by
another package.
This is done for "extract dependencies" at the beginning of the
extract step, and for "normal dependencies" at the beginning of the
configure step.
This is basically enough to make per-package SDK and target work. The
only gotcha is that at the end of the build, output/target and
output/host are empty, which means that:
- The filesystem image creation code cannot work.
- We don't have a SDK to build code outside of Buildroot.
In order to fix this, this commit extends the target-finalize step so
that it starts by populating output/target and output/host by
rsync-ing into them the target and host directories of all packages
listed in the $(PACKAGES) variable. It is necessary to do this
sequentially in the target-finalize step and not in each
package. Doing it in package installation means that it can be done in
parallel. In that case, there is a chance that two rsyncs are creating
the same hardlink or directory at the same time, which makes one of
them fail.
This change to per-package directories has an impact on the RPATH
built into the host binaries, as those RPATH now point to various
per-package host directories, and no longer to the global host
directory. We do not try to rewrite such RPATHs during the build as
having such RPATHs is perfectly fine, but we still need to handle two
fallouts from this change:
- The check-host-rpath script, which verifies at the end of each
package installation that it has the appropriate RPATH, is modified
to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is
a correct RPAT.
- The fix-rpath script, which mungles the RPATH mainly for the SDK
preparation, is modified to rewrite the RPATH to not point to
per-package directories. Indeed the patchelf --make-rpath-relative
call only works if the RPATH points to the ROOTDIR passed as
argument, and this ROOTDIR is the global host directory. Rewriting
the RPATH to not point to per-package host directories prior to
this is an easy solution to this issue.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 09:46:40 -07:00
|
|
|
$(call prepare-per-package-directory,$($(PKG)_FINAL_EXTRACT_DEPENDENCIES))
|
2014-05-05 07:04:20 -06:00
|
|
|
$(foreach hook,$($(PKG)_PRE_EXTRACT_HOOKS),$(call $(hook))$(sep))
|
2009-11-08 10:37:36 -07:00
|
|
|
$(Q)mkdir -p $(@D)
|
2011-07-05 13:53:52 -06:00
|
|
|
$($(PKG)_EXTRACT_CMDS)
|
2009-11-08 10:37:36 -07:00
|
|
|
# some packages have messed up permissions inside
|
2011-07-07 02:53:18 -06:00
|
|
|
$(Q)chmod -R +rw $(@D)
|
2009-11-08 10:37:36 -07:00
|
|
|
$(foreach hook,$($(PKG)_POST_EXTRACT_HOOKS),$(call $(hook))$(sep))
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_end,extract)
|
2015-11-03 12:06:03 -07:00
|
|
|
$(Q)touch $@
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2011-09-29 13:57:37 -06:00
|
|
|
# Rsync the source directory if the <pkg>_OVERRIDE_SRCDIR feature is
|
|
|
|
|
# used.
|
|
|
|
|
$(BUILD_DIR)/%/.stamp_rsynced:
|
2018-04-28 14:53:04 -06:00
|
|
|
@$(call step_start,rsync)
|
2011-09-29 13:57:37 -06:00
|
|
|
@$(call MESSAGE,"Syncing from source dir $(SRCDIR)")
|
Makefile: rework main directory creation logic
In the current code, the creation of the main output directories
(BUILD_DIR, STAGING_DIR, HOST_DIR, TARGET_DIR, etc.) is done by a
global "dirs" target. While this works fine in the current situation,
it doesn't work well in a context where per-package host and target
directories are used.
For example, with the current code and per-package host directories,
the output/staging symbolic link ends up being created as a link to
the per-package package sysroot directory of the first package being
built, instead of the global sysroot.
This commit reworks the creation of those directories by having the
package/pkg-generic.mk code ensure that the build directory, target
directory, host directory, staging directory and binaries directory
exist before they are needed.
Two new targets, host-finalize and staging-finalize are added in the
main Makefile to create the compatibility symlinks for host and
staging directories. They will be extended later with additional logic
for per-package directories.
Thanks to those changes, the global "dirs" target is entirely removed.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-23 07:58:08 -07:00
|
|
|
@mkdir -p $(@D)
|
2014-05-05 07:04:20 -06:00
|
|
|
$(foreach hook,$($(PKG)_PRE_RSYNC_HOOKS),$(call $(hook))$(sep))
|
2016-12-11 11:12:45 -07:00
|
|
|
@test -d $(SRCDIR) || (echo "ERROR: $(SRCDIR) does not exist" ; exit 1)
|
2019-06-10 03:27:16 -06:00
|
|
|
rsync -au --chmod=u=rwX,go=rX $($(PKG)_OVERRIDE_SRCDIR_RSYNC_EXCLUSIONS) $(RSYNC_VCS_EXCLUSIONS) $(call qstrip,$(SRCDIR))/ $(@D)
|
2013-11-07 03:41:21 -07:00
|
|
|
$(foreach hook,$($(PKG)_POST_RSYNC_HOOKS),$(call $(hook))$(sep))
|
2018-04-28 14:53:04 -06:00
|
|
|
@$(call step_end,rsync)
|
2011-09-29 13:57:37 -06:00
|
|
|
$(Q)touch $@
|
|
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
# Patch
|
|
|
|
|
#
|
2011-07-05 13:53:57 -06:00
|
|
|
# The RAWNAME variable is the lowercased package name, which allows to
|
|
|
|
|
# find the package directory (typically package/<pkgname>) and the
|
|
|
|
|
# prefix of the patches
|
2013-12-18 03:25:01 -07:00
|
|
|
#
|
|
|
|
|
# For BR2_GLOBAL_PATCH_DIR, only generate if it is defined
|
2014-02-05 02:44:01 -07:00
|
|
|
$(BUILD_DIR)/%/.stamp_patched: PATCH_BASE_DIRS = $(PKGDIR)
|
2013-12-18 03:25:01 -07:00
|
|
|
$(BUILD_DIR)/%/.stamp_patched: PATCH_BASE_DIRS += $(addsuffix /$(RAWNAME),$(call qstrip,$(BR2_GLOBAL_PATCH_DIR)))
|
2009-11-08 10:37:36 -07:00
|
|
|
$(BUILD_DIR)/%/.stamp_patched:
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_start,patch)
|
2013-11-07 09:26:53 -07:00
|
|
|
@$(call MESSAGE,"Patching")
|
2011-09-19 14:10:52 -06:00
|
|
|
$(foreach hook,$($(PKG)_PRE_PATCH_HOOKS),$(call $(hook))$(sep))
|
2018-04-02 07:09:26 -06:00
|
|
|
$(foreach p,$($(PKG)_PATCH),$(APPLY_PATCHES) $(@D) $($(PKG)_DL_DIR) $(notdir $(p))$(sep))
|
2009-11-08 10:37:36 -07:00
|
|
|
$(Q)( \
|
rework patch model
At the Buildroot Developers Meeting (4-5 February 2013, in Brussels) a change
to the patch logic was discussed. See
http://elinux.org/Buildroot:DeveloperDaysFOSDEM2013
for details. In summary:
* For patches stored in the package directory, if
package/<pkg>/<version>/ does exist, apply package/<pkg>/<version>/*.patch,
otherwise, apply package/<pkg>/*.patch
* For patches stored in the global patches directory, if
$(GLOBAL_PATCH_DIR)/<pkg>/<version>/ does exist, apply
$(GLOBAL_PATCH_DIR)/<pkg>/<version>/*.patch, otherwise, apply
$(GLOBAL_PATCH_DIR)/<pkg>/*.patch
This patch adds the new BR2_GLOBAL_PATCH_DIR configuration item, and reworks
the generic package infrastructure to implement the new patch logic.
[Peter: fixup doc nits as pointed out by Thomas]
Signed-off-by: Simon Dawson <spdawson@gmail.com>
Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
2013-03-17 17:13:47 -06:00
|
|
|
for D in $(PATCH_BASE_DIRS); do \
|
|
|
|
|
if test -d $${D}; then \
|
|
|
|
|
if test -d $${D}/$($(PKG)_VERSION); then \
|
2014-10-22 10:20:10 -06:00
|
|
|
$(APPLY_PATCHES) $(@D) $${D}/$($(PKG)_VERSION) \*.patch \*.patch.$(ARCH) || exit 1; \
|
rework patch model
At the Buildroot Developers Meeting (4-5 February 2013, in Brussels) a change
to the patch logic was discussed. See
http://elinux.org/Buildroot:DeveloperDaysFOSDEM2013
for details. In summary:
* For patches stored in the package directory, if
package/<pkg>/<version>/ does exist, apply package/<pkg>/<version>/*.patch,
otherwise, apply package/<pkg>/*.patch
* For patches stored in the global patches directory, if
$(GLOBAL_PATCH_DIR)/<pkg>/<version>/ does exist, apply
$(GLOBAL_PATCH_DIR)/<pkg>/<version>/*.patch, otherwise, apply
$(GLOBAL_PATCH_DIR)/<pkg>/*.patch
This patch adds the new BR2_GLOBAL_PATCH_DIR configuration item, and reworks
the generic package infrastructure to implement the new patch logic.
[Peter: fixup doc nits as pointed out by Thomas]
Signed-off-by: Simon Dawson <spdawson@gmail.com>
Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
2013-03-17 17:13:47 -06:00
|
|
|
else \
|
2014-10-22 10:20:10 -06:00
|
|
|
$(APPLY_PATCHES) $(@D) $${D} \*.patch \*.patch.$(ARCH) || exit 1; \
|
2009-11-08 10:37:36 -07:00
|
|
|
fi; \
|
|
|
|
|
fi; \
|
rework patch model
At the Buildroot Developers Meeting (4-5 February 2013, in Brussels) a change
to the patch logic was discussed. See
http://elinux.org/Buildroot:DeveloperDaysFOSDEM2013
for details. In summary:
* For patches stored in the package directory, if
package/<pkg>/<version>/ does exist, apply package/<pkg>/<version>/*.patch,
otherwise, apply package/<pkg>/*.patch
* For patches stored in the global patches directory, if
$(GLOBAL_PATCH_DIR)/<pkg>/<version>/ does exist, apply
$(GLOBAL_PATCH_DIR)/<pkg>/<version>/*.patch, otherwise, apply
$(GLOBAL_PATCH_DIR)/<pkg>/*.patch
This patch adds the new BR2_GLOBAL_PATCH_DIR configuration item, and reworks
the generic package infrastructure to implement the new patch logic.
[Peter: fixup doc nits as pointed out by Thomas]
Signed-off-by: Simon Dawson <spdawson@gmail.com>
Acked-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Tested-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: Samuel Martin <s.martin49@gmail.com>
Signed-off-by: Peter Korsgaard <jacmet@sunsite.dk>
2013-03-17 17:13:47 -06:00
|
|
|
done; \
|
2009-11-08 10:37:36 -07:00
|
|
|
)
|
|
|
|
|
$(foreach hook,$($(PKG)_POST_PATCH_HOOKS),$(call $(hook))$(sep))
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_end,patch)
|
2015-11-03 12:06:03 -07:00
|
|
|
$(Q)touch $@
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2014-10-21 02:42:54 -06:00
|
|
|
# Check that all directories specified in BR2_GLOBAL_PATCH_DIR exist.
|
|
|
|
|
$(foreach dir,$(call qstrip,$(BR2_GLOBAL_PATCH_DIR)),\
|
|
|
|
|
$(if $(wildcard $(dir)),,\
|
|
|
|
|
$(error BR2_GLOBAL_PATCH_DIR contains nonexistent directory $(dir))))
|
|
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
# Configure
|
|
|
|
|
$(BUILD_DIR)/%/.stamp_configured:
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_start,configure)
|
2009-11-08 10:37:36 -07:00
|
|
|
@$(call MESSAGE,"Configuring")
|
2020-04-30 03:52:41 -06:00
|
|
|
$(Q)mkdir -p $(HOST_DIR) $(TARGET_DIR) $(STAGING_DIR) $(BINARIES_DIR)
|
core: implement per-package SDK and target
This commit implements the core of the move to per-package SDK and
target directories. The main idea is that instead of having a global
output/host and output/target in which all packages install files, we
switch to per-package host and target directories, that only contain
their explicit dependencies.
There are two main benefits:
- Packages will now see only the dependencies they explicitly list in
their <pkg>_DEPENDENCIES variable, and the recursive dependencies
thereof.
- We can support top-level parallel build properly, because a package
only "sees" its own host directory and target directory, isolated
from the build of other packages that can happen in parallel.
It works as follows:
- A new output/per-package/ directory is created, which will contain
one sub-directory per package, and inside it, a "host" directory
and a "target" directory:
output/per-package/busybox/target
output/per-package/busybox/host
output/per-package/host-fakeroot/target
output/per-package/host-fakeroot/host
This output/per-package/ directory is PER_PACKAGE_DIR.
- The global TARGET_DIR and HOST_DIR variable now automatically point
to the per-package directory when PKG is defined. So whenever a
package references $(HOST_DIR) or $(TARGET_DIR) in its build
process, it effectively references the per-package host/target
directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it
is handled as well.
- Of course, packages have dependencies, so those dependencies must
be installed in the per-package host and target directories. To do
so, we simply rsync (using hard links to save space and time) the
host and target directories of the direct dependencies of the
package to the current package host and target directories.
We only need to take care of direct dependencies (and not
recursively all dependencies), because we accumulate into those
per-package host and target directories the files installed by the
dependencies. Note that this only works because we make the
assumption that one package does *not* overwrite files installed by
another package.
This is done for "extract dependencies" at the beginning of the
extract step, and for "normal dependencies" at the beginning of the
configure step.
This is basically enough to make per-package SDK and target work. The
only gotcha is that at the end of the build, output/target and
output/host are empty, which means that:
- The filesystem image creation code cannot work.
- We don't have a SDK to build code outside of Buildroot.
In order to fix this, this commit extends the target-finalize step so
that it starts by populating output/target and output/host by
rsync-ing into them the target and host directories of all packages
listed in the $(PACKAGES) variable. It is necessary to do this
sequentially in the target-finalize step and not in each
package. Doing it in package installation means that it can be done in
parallel. In that case, there is a chance that two rsyncs are creating
the same hardlink or directory at the same time, which makes one of
them fail.
This change to per-package directories has an impact on the RPATH
built into the host binaries, as those RPATH now point to various
per-package host directories, and no longer to the global host
directory. We do not try to rewrite such RPATHs during the build as
having such RPATHs is perfectly fine, but we still need to handle two
fallouts from this change:
- The check-host-rpath script, which verifies at the end of each
package installation that it has the appropriate RPATH, is modified
to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is
a correct RPAT.
- The fix-rpath script, which mungles the RPATH mainly for the SDK
preparation, is modified to rewrite the RPATH to not point to
per-package directories. Indeed the patchelf --make-rpath-relative
call only works if the RPATH points to the ROOTDIR passed as
argument, and this ROOTDIR is the global host directory. Rewriting
the RPATH to not point to per-package host directories prior to
this is an easy solution to this issue.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 09:46:40 -07:00
|
|
|
$(call prepare-per-package-directory,$($(PKG)_FINAL_DEPENDENCIES))
|
package/pkg-generic.mk: rework pkg_size logic with the "installed" step
This commits reworks the pkg_size logic to no longer use the
GLOBAL_INSTRUMENTATION_HOOKS mechanism, but instead be directly
implemented within the configure step and install step.
The problem with the current implementation in the
GLOBAL_INSTRUMENTATION_HOOKS is that we only capture what is installed
in $(HOST_DIR) during the "host installation step", what is installed
in $(TARGET_DIR) during the "target installation step" and what is
installed in "$(STAGING_DIR)" during the staging installation step.
While this seems reasonable in principle, it is in fact not completely
true. For example, "toolchain", which is a target package, installs
tons of files in $(HOST_DIR). "qt5base", which is also a target
package, also installs things in $(HOST_DIR). Same with the "syslinux"
package.
The idea behind this patch is pretty simple:
- At the beginning of the configure step, right after the per-package
directories have been populated (if BR2_PER_PACKAGE_DIRECTORIES=y),
we capture the state of the HOST_DIR, TARGET_DIR and STAGING_DIR.
- At the end of all install steps (which is possible thanks to the
newly introduced "install" step), we capture again the state of
HOST_DIR, TARGET_DIR and STAGING_DIR, and compare it to what we
have saved at the configure step.
So regardless of whether a file has been installed in $(HOST_DIR)
during the target or staging installation steps of a target package,
or if a host package has installed a file in $(TARGET_DIR), we will
detect it.
The pkg_size_before and pkg_size_after macros are intentionally left
where they are (even if they now fall in the middle of the
GLOBAL_INSTRUMENTATION_HOOKS implementations) to minimize the diffstat
and facilitate review.
Note that we also have to change check_bin_arch to be explicitly
called from the install step rather than through a
GLOBAL_INSTRUMENTATION_HOOKS as it depends on the .files-list.txt file
produced by the pkg_size_after function.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-30 03:52:42 -06:00
|
|
|
@$(call pkg_size_before,$(TARGET_DIR))
|
|
|
|
|
@$(call pkg_size_before,$(STAGING_DIR),-staging)
|
|
|
|
|
@$(call pkg_size_before,$(HOST_DIR),-host)
|
2019-11-05 09:46:42 -07:00
|
|
|
$(call fixup-libtool-files,$(NAME),$(STAGING_DIR))
|
2014-05-05 07:04:20 -06:00
|
|
|
$(foreach hook,$($(PKG)_PRE_CONFIGURE_HOOKS),$(call $(hook))$(sep))
|
2009-11-08 10:37:36 -07:00
|
|
|
$($(PKG)_CONFIGURE_CMDS)
|
|
|
|
|
$(foreach hook,$($(PKG)_POST_CONFIGURE_HOOKS),$(call $(hook))$(sep))
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_end,configure)
|
2015-11-03 12:06:03 -07:00
|
|
|
$(Q)touch $@
|
2009-11-08 10:37:36 -07:00
|
|
|
|
|
|
|
|
# Build
|
|
|
|
|
$(BUILD_DIR)/%/.stamp_built::
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_start,build)
|
2009-11-08 10:37:36 -07:00
|
|
|
@$(call MESSAGE,"Building")
|
2014-05-05 07:04:20 -06:00
|
|
|
$(foreach hook,$($(PKG)_PRE_BUILD_HOOKS),$(call $(hook))$(sep))
|
2014-02-14 02:55:07 -07:00
|
|
|
+$($(PKG)_BUILD_CMDS)
|
2009-11-08 10:37:36 -07:00
|
|
|
$(foreach hook,$($(PKG)_POST_BUILD_HOOKS),$(call $(hook))$(sep))
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_end,build)
|
2015-11-03 12:06:03 -07:00
|
|
|
$(Q)touch $@
|
2009-11-08 10:37:36 -07:00
|
|
|
|
|
|
|
|
# Install to host dir
|
|
|
|
|
$(BUILD_DIR)/%/.stamp_host_installed:
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_start,install-host)
|
2011-07-18 04:49:25 -06:00
|
|
|
@$(call MESSAGE,"Installing to host directory")
|
2014-05-05 07:04:20 -06:00
|
|
|
$(foreach hook,$($(PKG)_PRE_INSTALL_HOOKS),$(call $(hook))$(sep))
|
2014-02-14 02:55:07 -07:00
|
|
|
+$($(PKG)_INSTALL_CMDS)
|
2009-11-08 10:37:36 -07:00
|
|
|
$(foreach hook,$($(PKG)_POST_INSTALL_HOOKS),$(call $(hook))$(sep))
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_end,install-host)
|
2015-11-03 12:06:03 -07:00
|
|
|
$(Q)touch $@
|
2009-11-08 10:37:36 -07:00
|
|
|
|
|
|
|
|
# Install to staging dir
|
2015-05-03 15:30:39 -06:00
|
|
|
#
|
|
|
|
|
# Some packages install libtool .la files alongside any installed
|
|
|
|
|
# libraries. These .la files sometimes refer to paths relative to the
|
|
|
|
|
# sysroot, which libtool will interpret as absolute paths to host
|
|
|
|
|
# libraries instead of the target libraries. Since this is not what we
|
|
|
|
|
# want, these paths are fixed by prefixing them with $(STAGING_DIR).
|
|
|
|
|
# As we configure with --prefix=/usr, this fix needs to be applied to
|
|
|
|
|
# any path that starts with /usr.
|
|
|
|
|
#
|
|
|
|
|
# To protect against the case that the output or staging directories or
|
|
|
|
|
# the pre-installed external toolchain themselves are under /usr, we first
|
|
|
|
|
# substitute away any occurrences of these directories with @BASE_DIR@,
|
|
|
|
|
# @STAGING_DIR@ and @TOOLCHAIN_EXTERNAL_INSTALL_DIR@ respectively.
|
|
|
|
|
#
|
|
|
|
|
# Note that STAGING_DIR can be outside BASE_DIR when the user sets
|
|
|
|
|
# BR2_HOST_DIR to a custom value. Note that TOOLCHAIN_EXTERNAL_INSTALL_DIR
|
|
|
|
|
# can be under @BASE_DIR@ when it's a downloaded toolchain, and can be
|
|
|
|
|
# empty when we use an internal toolchain.
|
|
|
|
|
#
|
2009-11-08 10:37:36 -07:00
|
|
|
$(BUILD_DIR)/%/.stamp_staging_installed:
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_start,install-staging)
|
2011-07-18 04:49:25 -06:00
|
|
|
@$(call MESSAGE,"Installing to staging directory")
|
2014-05-05 07:04:20 -06:00
|
|
|
$(foreach hook,$($(PKG)_PRE_INSTALL_STAGING_HOOKS),$(call $(hook))$(sep))
|
2014-02-14 02:55:07 -07:00
|
|
|
+$($(PKG)_INSTALL_STAGING_CMDS)
|
2009-11-08 10:37:36 -07:00
|
|
|
$(foreach hook,$($(PKG)_POST_INSTALL_STAGING_HOOKS),$(call $(hook))$(sep))
|
2013-02-07 05:35:03 -07:00
|
|
|
$(Q)if test -n "$($(PKG)_CONFIG_SCRIPTS)" ; then \
|
2013-01-29 19:46:40 -07:00
|
|
|
$(call MESSAGE,"Fixing package configuration files") ;\
|
package/pkg-generic: adjust config scripts tweaks for per-package directories
This commit adjusts the logic in pkg-generic.mk that tweaks the
*-config shell scripts installed by various libraries to make it
compatible with per-package directories.
This requires two fixes:
- replacing $(STAGING_DIR) with a relative path from the config script
to the staging directory, rather than using an absolute path of the
staging directory.
Without this, a *-config script provided by package A, but called
from package B per-package directory will return paths from package A
per-package directory:
$ ./output/per-package/mcrypt/host/usr/<tuple>/sysroot/usr/bin/libmcrypt-config --libs
-L..../output/per-package/libmcrypt/host/usr/<tuple>/sysroot/usr/lib/
The libmcrypt-config script is installed by the libmcrypt package,
and mcrypt is a package that depends on libmcrypt. When we call the
libmcrypt-config script from the mcrypt per-package directory, it
returns a -L flag that points to the libmcrypt per-package
directory.
One might say: but this is OK, since the sysroot of the libmcrypt
per-package directory also contains the libmcrypt library. This is
true, but we encounter a more subtle issue: because -L paths are
considered before standard paths, ld ends up finding libc.so in the
libmcrypt per-package directory. This libc.so file is a linker
script that looks like this:
GROUP ( /lib/libc.so.6 /usr/lib/libc_nonshared.a AS_NEEDED ( /lib/ld-linux.so.3 ) )
Normally, thanks to ld sysroot awareness, /lib/libc.so.6 in this
script is re-interpreted according to the sysroot. But in this
case, the library is *outside* the compiler sysroot. Remember: we
are using the compiler/linker from the "mcrypt" per-package
directory, but we found "libc.so.6" in the "libmcrypt" per-package
directory.
This causes the linker to really use the /lib/libc.so.6 from the
host machine, obvisouly leading to a build failure such as:
output/per-package/libgcrypt/host/opt/ext-toolchain/bin/../lib/gcc/nios2-linux-gnu/7.3.1/../../../../nios2-linux-gnu/bin/ld: cannot find /lib/libc.so.6
output/per-package/libgcrypt/host/opt/ext-toolchain/bin/../lib/gcc/nios2-linux-gnu/7.3.1/../../../../nios2-linux-gnu/bin/ld: cannot find /usr/lib/libc_nonshared.a
output/per-package/libgcrypt/host/opt/ext-toolchain/bin/../lib/gcc/nios2-linux-gnu/7.3.1/../../../../nios2-linux-gnu/bin/ld: cannot find /lib/ld-linux-nios2.so.1
- Some *-config scripts, such as the apr-1-config script, contain
references to host tools:
CC=".../output/per-package/apr/hosr/bin/arm-linux-gcc"
CCP=".../output/per-package/apr/hosr/bin/arm-linux-cpp"
We also want to replace those with proper relative paths. To
achieve this, we need to also replace $(HOST_DIR) with a relative
path. Since $(STAGING_DIR) is inside $(HOST_DIR), the first
replacement of $(STAGING_DIR) by @STAGING_DIR@ is no longer needed:
replacing $(HOST_DIR) by @HOST_DIR@ is sufficient. We still need to
replace @STAGING_DIR@ by the proper path though, as we introduce
@STAGING_DIR@ references in exec_prefix and prefix variables, as
well as -I and -L flags.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-23 07:58:11 -07:00
|
|
|
$(SED) "s,$(HOST_DIR),@HOST_DIR@,g" \
|
|
|
|
|
-e "s,$(BASE_DIR),@BASE_DIR@,g" \
|
2014-06-21 09:01:47 -06:00
|
|
|
-e "s,^\(exec_\)\?prefix=.*,\1prefix=@STAGING_DIR@/usr,g" \
|
|
|
|
|
-e "s,-I/usr/,-I@STAGING_DIR@/usr/,g" \
|
|
|
|
|
-e "s,-L/usr/,-L@STAGING_DIR@/usr/,g" \
|
2018-12-01 14:51:55 -07:00
|
|
|
-e 's,@STAGING_DIR@,$$(dirname $$(readlink -e $$0))/../..,g' \
|
|
|
|
|
-e 's,@HOST_DIR@,$$(dirname $$(readlink -e $$0))/../../../..,g' \
|
2014-06-21 09:01:47 -06:00
|
|
|
-e "s,@BASE_DIR@,$(BASE_DIR),g" \
|
2013-02-07 05:35:03 -07:00
|
|
|
$(addprefix $(STAGING_DIR)/usr/bin/,$($(PKG)_CONFIG_SCRIPTS)) ;\
|
2013-01-29 19:46:40 -07:00
|
|
|
fi
|
2015-05-03 15:30:39 -06:00
|
|
|
@$(call MESSAGE,"Fixing libtool files")
|
2019-02-10 14:38:18 -07:00
|
|
|
for la in $$(find $(STAGING_DIR)/usr/lib* -name "*.la"); do \
|
|
|
|
|
cp -a "$${la}" "$${la}.fixed" && \
|
2015-05-03 15:30:39 -06:00
|
|
|
$(SED) "s:$(BASE_DIR):@BASE_DIR@:g" \
|
|
|
|
|
-e "s:$(STAGING_DIR):@STAGING_DIR@:g" \
|
|
|
|
|
$(if $(TOOLCHAIN_EXTERNAL_INSTALL_DIR),\
|
|
|
|
|
-e "s:$(TOOLCHAIN_EXTERNAL_INSTALL_DIR):@TOOLCHAIN_EXTERNAL_INSTALL_DIR@:g") \
|
|
|
|
|
-e "s:\(['= ]\)/usr:\\1@STAGING_DIR@/usr:g" \
|
package/pkg-generic.mk: also replace /lib by STAGING_DIR/lib in .la files
After the staging installation, we replace a number of paths in libtool
.la files so that those paths point to STAGING_DIR instead of a location
in the build machine.
However, we replace only paths that start with /usr. And it turns out
that the linux-pam package is configured with --libdir=/lib (linux-pam
seems to always be installed in /lib rather than /usr/lib).
Due to this, libpam.la contains the following line:
libdir='/lib'
When building a configuration that has:
- BR2_ROOTFS_MERGED_USR=y
- BR2_PACKAGE_LINUX_PAM=y
- BR2_PACKAGE_POLKIT=y
on a system that has its system-wide PAM library installed in /lib,
the build fails with:
/lib/libpam.so: file not recognized: File format not recognized
For some reason, libtool searches only in STAGING_DIR/usr/lib, but
when BR2_ROOTFS_MERGED_USR=y, STAGING_DIR/lib points to
STAGING_DIR/usr/lib, so libtool finds libpam.la. And this libpam.la
contains a bogus libdir='/lib' path. libtool then goes on, finds
/lib/libpam.so, and links with it, causing the build failure.
By doing the proper replacement of libdir='/lib', we have a correct
libpam.la, and solve the build issue.
There is no autobuilder failure associated to this issue, as it
requires /lib/libpam.so to exist. This is the case on ArchLinux, on
which Xogium reported the issue, which can also be reproduced in an
ArchLinux container.
Reported-by: Xogium <contact@xogium.me>
Cc: Xogium <contact@xogium.me>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Yann E. MORIN <yann.morin.1998@free.fr>
[yann.morin.1998@free.fr:
- tested by manually creating a symlink to libpam.so in /lib
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-05 08:21:42 -06:00
|
|
|
-e "s:\(['= ]\)/lib:\\1@STAGING_DIR@/lib:g" \
|
2015-05-03 15:30:39 -06:00
|
|
|
$(if $(TOOLCHAIN_EXTERNAL_INSTALL_DIR),\
|
|
|
|
|
-e "s:@TOOLCHAIN_EXTERNAL_INSTALL_DIR@:$(TOOLCHAIN_EXTERNAL_INSTALL_DIR):g") \
|
|
|
|
|
-e "s:@STAGING_DIR@:$(STAGING_DIR):g" \
|
2019-02-10 14:38:18 -07:00
|
|
|
-e "s:@BASE_DIR@:$(BASE_DIR):g" \
|
|
|
|
|
"$${la}.fixed" && \
|
|
|
|
|
if cmp -s "$${la}" "$${la}.fixed"; then \
|
|
|
|
|
rm -f "$${la}.fixed"; \
|
|
|
|
|
else \
|
|
|
|
|
mv "$${la}.fixed" "$${la}"; \
|
|
|
|
|
fi || exit 1; \
|
|
|
|
|
done
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_end,install-staging)
|
2015-11-03 12:06:03 -07:00
|
|
|
$(Q)touch $@
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2011-07-05 13:53:56 -06:00
|
|
|
# Install to images dir
|
|
|
|
|
$(BUILD_DIR)/%/.stamp_images_installed:
|
package/pkg-generic.mk: create directories before calling hooks
In commit 0e2be4db8ab01d479177a3a187c22525752195ae
("package/pkg-generic: make file list logic parallel build
compatible"), the logic to create the list of files installed by a
particular package was significantly reworked to be compatible with
top-level parallel build.
Before this commit, there was only a after-install step of listing the
files in HOST_DIR/TARGET_DIR/STAGING_DIR. But after this commit, we
now have a before-install logic and an after-install logic.
It turns out that when the before-install logic is called for the very
first host package, $(HOST_DIR) doesn't exist yet, and therefore the
cd $(2) fails, with an error message:
/bin/sh: line 0: cd: /home/thomas/buildroot/output/host: No such file or directory
In fact, $(HOST_DIR), $(STAGING_DIR), $(TARGET_DIR) and
$(BINARIES_DIR) are created by the make rules for host installation,
staging installation, target installation and images installation, but
*after* calling the step_start hooks.
So, we simply fix this problem by creating the directories *before*
calling the step_start hooks.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-03-12 03:15:29 -06:00
|
|
|
@$(call step_start,install-image)
|
2011-07-18 04:49:25 -06:00
|
|
|
@$(call MESSAGE,"Installing to images directory")
|
2020-02-28 08:04:20 -07:00
|
|
|
$(foreach hook,$($(PKG)_PRE_INSTALL_IMAGES_HOOKS),$(call $(hook))$(sep))
|
2014-02-14 02:55:07 -07:00
|
|
|
+$($(PKG)_INSTALL_IMAGES_CMDS)
|
2011-07-05 13:53:56 -06:00
|
|
|
$(foreach hook,$($(PKG)_POST_INSTALL_IMAGES_HOOKS),$(call $(hook))$(sep))
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_end,install-image)
|
2015-11-03 12:06:03 -07:00
|
|
|
$(Q)touch $@
|
2011-07-05 13:53:56 -06:00
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
# Install to target dir
|
|
|
|
|
$(BUILD_DIR)/%/.stamp_target_installed:
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_start,install-target)
|
2009-11-08 10:37:36 -07:00
|
|
|
@$(call MESSAGE,"Installing to target")
|
2014-05-05 07:04:20 -06:00
|
|
|
$(foreach hook,$($(PKG)_PRE_INSTALL_TARGET_HOOKS),$(call $(hook))$(sep))
|
2014-07-06 07:45:47 -06:00
|
|
|
+$($(PKG)_INSTALL_TARGET_CMDS)
|
2012-07-28 01:21:20 -06:00
|
|
|
$(if $(BR2_INIT_SYSTEMD),\
|
|
|
|
|
$($(PKG)_INSTALL_INIT_SYSTEMD))
|
|
|
|
|
$(if $(BR2_INIT_SYSV)$(BR2_INIT_BUSYBOX),\
|
|
|
|
|
$($(PKG)_INSTALL_INIT_SYSV))
|
2019-05-12 13:55:42 -06:00
|
|
|
$(if $(BR2_INIT_OPENRC), \
|
2019-08-04 06:14:15 -06:00
|
|
|
$(or $($(PKG)_INSTALL_INIT_OPENRC), \
|
|
|
|
|
$($(PKG)_INSTALL_INIT_SYSV)))
|
2009-11-08 10:37:36 -07:00
|
|
|
$(foreach hook,$($(PKG)_POST_INSTALL_TARGET_HOOKS),$(call $(hook))$(sep))
|
2013-02-07 05:35:04 -07:00
|
|
|
$(Q)if test -n "$($(PKG)_CONFIG_SCRIPTS)" ; then \
|
|
|
|
|
$(RM) -f $(addprefix $(TARGET_DIR)/usr/bin/,$($(PKG)_CONFIG_SCRIPTS)) ; \
|
|
|
|
|
fi
|
2013-11-11 08:03:27 -07:00
|
|
|
@$(call step_end,install-target)
|
2015-11-03 12:06:03 -07:00
|
|
|
$(Q)touch $@
|
2009-11-08 10:37:36 -07:00
|
|
|
|
package/pkg-generic.mk: introduce final 'install' step
We currently have four different install steps: target installation,
staging installation, images installation and host installation. These
steps are directly triggered from the $(1)-install make target, so
there is no place where we can run some logic once all installation
steps have completed.
However, as part of improving the reliability of the logic done in
step_pkg_size_before and step_pkg_size_after to detect the files
installed by packages, we would in fact need to run some logic after
all installation steps have completed. This will allow us to make sure
that all files are detected, even if a host package installs something
in the target directory, or if a target package installs something in
the host directory.
To achieve this, this commit implements a new stamp file,
.stamp_installed, which is a step that depends on all four install
steps. Currently, this step does nothing except creating the stamp
file.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: remove stampfile on foo-reinstall]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-30 03:52:40 -06:00
|
|
|
# Final installation step, completed when all installation steps
|
|
|
|
|
# (host, images, staging, target) have completed
|
|
|
|
|
$(BUILD_DIR)/%/.stamp_installed:
|
package/pkg-generic.mk: rework pkg_size logic with the "installed" step
This commits reworks the pkg_size logic to no longer use the
GLOBAL_INSTRUMENTATION_HOOKS mechanism, but instead be directly
implemented within the configure step and install step.
The problem with the current implementation in the
GLOBAL_INSTRUMENTATION_HOOKS is that we only capture what is installed
in $(HOST_DIR) during the "host installation step", what is installed
in $(TARGET_DIR) during the "target installation step" and what is
installed in "$(STAGING_DIR)" during the staging installation step.
While this seems reasonable in principle, it is in fact not completely
true. For example, "toolchain", which is a target package, installs
tons of files in $(HOST_DIR). "qt5base", which is also a target
package, also installs things in $(HOST_DIR). Same with the "syslinux"
package.
The idea behind this patch is pretty simple:
- At the beginning of the configure step, right after the per-package
directories have been populated (if BR2_PER_PACKAGE_DIRECTORIES=y),
we capture the state of the HOST_DIR, TARGET_DIR and STAGING_DIR.
- At the end of all install steps (which is possible thanks to the
newly introduced "install" step), we capture again the state of
HOST_DIR, TARGET_DIR and STAGING_DIR, and compare it to what we
have saved at the configure step.
So regardless of whether a file has been installed in $(HOST_DIR)
during the target or staging installation steps of a target package,
or if a host package has installed a file in $(TARGET_DIR), we will
detect it.
The pkg_size_before and pkg_size_after macros are intentionally left
where they are (even if they now fall in the middle of the
GLOBAL_INSTRUMENTATION_HOOKS implementations) to minimize the diffstat
and facilitate review.
Note that we also have to change check_bin_arch to be explicitly
called from the install step rather than through a
GLOBAL_INSTRUMENTATION_HOOKS as it depends on the .files-list.txt file
produced by the pkg_size_after function.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-30 03:52:42 -06:00
|
|
|
@$(call pkg_size_after,$(TARGET_DIR))
|
|
|
|
|
@$(call pkg_size_after,$(STAGING_DIR),-staging)
|
|
|
|
|
@$(call pkg_size_after,$(HOST_DIR),-host)
|
|
|
|
|
@$(call check_bin_arch)
|
package/pkg-generic.mk: introduce final 'install' step
We currently have four different install steps: target installation,
staging installation, images installation and host installation. These
steps are directly triggered from the $(1)-install make target, so
there is no place where we can run some logic once all installation
steps have completed.
However, as part of improving the reliability of the logic done in
step_pkg_size_before and step_pkg_size_after to detect the files
installed by packages, we would in fact need to run some logic after
all installation steps have completed. This will allow us to make sure
that all files are detected, even if a host package installs something
in the target directory, or if a target package installs something in
the host directory.
To achieve this, this commit implements a new stamp file,
.stamp_installed, which is a step that depends on all four install
steps. Currently, this step does nothing except creating the stamp
file.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: remove stampfile on foo-reinstall]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-30 03:52:40 -06:00
|
|
|
$(Q)touch $@
|
|
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
# Remove package sources
|
|
|
|
|
$(BUILD_DIR)/%/.stamp_dircleaned:
|
core: implement per-package SDK and target
This commit implements the core of the move to per-package SDK and
target directories. The main idea is that instead of having a global
output/host and output/target in which all packages install files, we
switch to per-package host and target directories, that only contain
their explicit dependencies.
There are two main benefits:
- Packages will now see only the dependencies they explicitly list in
their <pkg>_DEPENDENCIES variable, and the recursive dependencies
thereof.
- We can support top-level parallel build properly, because a package
only "sees" its own host directory and target directory, isolated
from the build of other packages that can happen in parallel.
It works as follows:
- A new output/per-package/ directory is created, which will contain
one sub-directory per package, and inside it, a "host" directory
and a "target" directory:
output/per-package/busybox/target
output/per-package/busybox/host
output/per-package/host-fakeroot/target
output/per-package/host-fakeroot/host
This output/per-package/ directory is PER_PACKAGE_DIR.
- The global TARGET_DIR and HOST_DIR variable now automatically point
to the per-package directory when PKG is defined. So whenever a
package references $(HOST_DIR) or $(TARGET_DIR) in its build
process, it effectively references the per-package host/target
directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it
is handled as well.
- Of course, packages have dependencies, so those dependencies must
be installed in the per-package host and target directories. To do
so, we simply rsync (using hard links to save space and time) the
host and target directories of the direct dependencies of the
package to the current package host and target directories.
We only need to take care of direct dependencies (and not
recursively all dependencies), because we accumulate into those
per-package host and target directories the files installed by the
dependencies. Note that this only works because we make the
assumption that one package does *not* overwrite files installed by
another package.
This is done for "extract dependencies" at the beginning of the
extract step, and for "normal dependencies" at the beginning of the
configure step.
This is basically enough to make per-package SDK and target work. The
only gotcha is that at the end of the build, output/target and
output/host are empty, which means that:
- The filesystem image creation code cannot work.
- We don't have a SDK to build code outside of Buildroot.
In order to fix this, this commit extends the target-finalize step so
that it starts by populating output/target and output/host by
rsync-ing into them the target and host directories of all packages
listed in the $(PACKAGES) variable. It is necessary to do this
sequentially in the target-finalize step and not in each
package. Doing it in package installation means that it can be done in
parallel. In that case, there is a chance that two rsyncs are creating
the same hardlink or directory at the same time, which makes one of
them fail.
This change to per-package directories has an impact on the RPATH
built into the host binaries, as those RPATH now point to various
per-package host directories, and no longer to the global host
directory. We do not try to rewrite such RPATHs during the build as
having such RPATHs is perfectly fine, but we still need to handle two
fallouts from this change:
- The check-host-rpath script, which verifies at the end of each
package installation that it has the appropriate RPATH, is modified
to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is
a correct RPAT.
- The fix-rpath script, which mungles the RPATH mainly for the SDK
preparation, is modified to rewrite the RPATH to not point to
per-package directories. Indeed the patchelf --make-rpath-relative
call only works if the RPATH points to the ROOTDIR passed as
argument, and this ROOTDIR is the global host directory. Rewriting
the RPATH to not point to per-package host directories prior to
this is an easy solution to this issue.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 09:46:40 -07:00
|
|
|
$(if $(BR2_PER_PACKAGE_DIRECTORIES),rm -Rf $(PER_PACKAGE_DIR)/$(NAME))
|
2009-11-08 10:37:36 -07:00
|
|
|
rm -Rf $(@D)
|
|
|
|
|
|
2014-05-15 11:37:03 -06:00
|
|
|
################################################################################
|
|
|
|
|
# virt-provides-single -- check that provider-pkg is the declared provider for
|
|
|
|
|
# the virtual package virt-pkg
|
|
|
|
|
#
|
|
|
|
|
# argument 1 is the lower-case name of the virtual package
|
|
|
|
|
# argument 2 is the upper-case name of the virtual package
|
|
|
|
|
# argument 3 is the lower-case name of the provider
|
|
|
|
|
#
|
|
|
|
|
# example:
|
|
|
|
|
# $(call virt-provides-single,libegl,LIBEGL,rpi-userland)
|
|
|
|
|
################################################################################
|
|
|
|
|
define virt-provides-single
|
|
|
|
|
ifneq ($$(call qstrip,$$(BR2_PACKAGE_PROVIDES_$(2))),$(3))
|
|
|
|
|
$$(error Configuration error: both "$(3)" and $$(BR2_PACKAGE_PROVIDES_$(2))\
|
|
|
|
|
are selected as providers for virtual package "$(1)". Only one provider can\
|
|
|
|
|
be selected at a time. Please fix your configuration)
|
|
|
|
|
endif
|
|
|
|
|
endef
|
|
|
|
|
|
2016-10-23 11:19:44 -06:00
|
|
|
define pkg-graph-depends
|
|
|
|
|
@$$(INSTALL) -d $$(GRAPHS_DIR)
|
|
|
|
|
@cd "$$(CONFIG_DIR)"; \
|
|
|
|
|
$$(TOPDIR)/support/scripts/graph-depends $$(BR2_GRAPH_DEPS_OPTS) \
|
|
|
|
|
-p $(1) $(2) -o $$(GRAPHS_DIR)/$$(@).dot
|
|
|
|
|
dot $$(BR2_GRAPH_DOT_OPTS) -T$$(BR_GRAPH_OUT) \
|
|
|
|
|
-o $$(GRAPHS_DIR)/$$(@).$$(BR_GRAPH_OUT) \
|
|
|
|
|
$$(GRAPHS_DIR)/$$(@).dot
|
|
|
|
|
endef
|
|
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
################################################################################
|
2012-07-02 16:07:08 -06:00
|
|
|
# inner-generic-package -- generates the make targets needed to build a
|
2009-11-08 10:37:36 -07:00
|
|
|
# generic package
|
|
|
|
|
#
|
|
|
|
|
# argument 1 is the lowercase package name
|
2014-07-24 12:57:41 -06:00
|
|
|
# argument 2 is the uppercase package name, including a HOST_ prefix
|
2009-11-08 10:37:36 -07:00
|
|
|
# for host packages
|
|
|
|
|
# argument 3 is the uppercase package name, without the HOST_ prefix
|
|
|
|
|
# for host packages
|
2014-02-05 02:44:03 -07:00
|
|
|
# argument 4 is the type (target or host)
|
2014-06-11 13:12:25 -06:00
|
|
|
#
|
|
|
|
|
# Note about variable and function references: inside all blocks that are
|
|
|
|
|
# evaluated with $(eval), which includes all 'inner-xxx-package' blocks,
|
|
|
|
|
# specific rules apply with respect to variable and function references.
|
|
|
|
|
# - Numbered variables (parameters to the block) can be referenced with a single
|
|
|
|
|
# dollar sign: $(1), $(2), $(3), etc.
|
|
|
|
|
# - pkgdir and pkgname should be referenced with a single dollar sign too. These
|
|
|
|
|
# functions rely on 'the most recently parsed makefile' which is supposed to
|
|
|
|
|
# be the package .mk file. If we defer the evaluation of these functions using
|
|
|
|
|
# double dollar signs, then they may be evaluated too late, when other
|
|
|
|
|
# makefiles have already been parsed. One specific case is when $$(pkgdir) is
|
|
|
|
|
# assigned to a variable using deferred evaluation with '=' and this variable
|
|
|
|
|
# is used in a target rule outside the eval'ed inner block. In this case, the
|
|
|
|
|
# pkgdir will be that of the last makefile parsed by buildroot, which is not
|
|
|
|
|
# the expected value. This mechanism is for example used for the TARGET_PATCH
|
|
|
|
|
# rule.
|
|
|
|
|
# - All other variables should be referenced with a double dollar sign:
|
|
|
|
|
# $$(TARGET_DIR), $$($(2)_VERSION), etc. Also all make functions should be
|
|
|
|
|
# referenced with a double dollar sign: $$(subst), $$(call), $$(filter-out),
|
|
|
|
|
# etc. This rule ensures that these variables and functions are only expanded
|
|
|
|
|
# during the $(eval) step, and not earlier. Otherwise, unintuitive and
|
|
|
|
|
# undesired behavior occurs with respect to these variables and functions.
|
|
|
|
|
#
|
2009-11-08 10:37:36 -07:00
|
|
|
################################################################################
|
|
|
|
|
|
2012-07-02 16:07:08 -06:00
|
|
|
define inner-generic-package
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2018-03-31 03:05:51 -06:00
|
|
|
# When doing a package, we're definitely not doing a rootfs, but we
|
|
|
|
|
# may inherit it via the dependency chain, so we reset it.
|
|
|
|
|
$(1): ROOTFS=
|
|
|
|
|
|
2015-10-22 14:33:56 -06:00
|
|
|
# Ensure the package is only declared once, i.e. do not accept that a
|
|
|
|
|
# package be re-defined by a br2-external tree
|
|
|
|
|
ifneq ($(call strip,$(filter $(1),$(PACKAGES_ALL))),)
|
|
|
|
|
$$(error Package '$(1)' defined a second time in '$(pkgdir)'; \
|
|
|
|
|
previous definition was in '$$($(2)_PKGDIR)')
|
|
|
|
|
endif
|
|
|
|
|
PACKAGES_ALL += $(1)
|
|
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
# Define default values for various package-related variables, if not
|
|
|
|
|
# already defined. For some variables (version, source, site and
|
|
|
|
|
# subdir), if they are undefined, we try to see if a variable without
|
|
|
|
|
# the HOST_ prefix is defined. If so, we use such a variable, so that
|
2014-07-24 12:07:02 -06:00
|
|
|
# this information has only to be specified once, for both the
|
2009-11-08 10:37:36 -07:00
|
|
|
# target and host packages of a given .mk file.
|
|
|
|
|
|
2014-02-05 02:44:03 -07:00
|
|
|
$(2)_TYPE = $(4)
|
2009-11-08 10:37:36 -07:00
|
|
|
$(2)_NAME = $(1)
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 13:12:24 -06:00
|
|
|
$(2)_RAWNAME = $$(patsubst host-%,%,$(1))
|
2015-10-22 14:33:56 -06:00
|
|
|
$(2)_PKGDIR = $(pkgdir)
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2010-09-18 17:21:15 -06:00
|
|
|
# Keep the package version that may contain forward slashes in the _DL_VERSION
|
|
|
|
|
# variable, then replace all forward slashes ('/') by underscores ('_') to
|
|
|
|
|
# sanitize the package version that is used in paths, directory and file names.
|
|
|
|
|
# Forward slashes may appear in the package's version when pointing to a
|
|
|
|
|
# version control system branch or tag, for example remotes/origin/1_10_stable.
|
2015-04-22 17:08:08 -06:00
|
|
|
# Similar for spaces and colons (:) that may appear in date-based revisions for
|
|
|
|
|
# CVS.
|
2009-11-08 10:37:36 -07:00
|
|
|
ifndef $(2)_VERSION
|
2015-07-11 09:40:14 -06:00
|
|
|
ifdef $(3)_DL_VERSION
|
|
|
|
|
$(2)_DL_VERSION := $$($(3)_DL_VERSION)
|
|
|
|
|
else ifdef $(3)_VERSION
|
|
|
|
|
$(2)_DL_VERSION := $$($(3)_VERSION)
|
2009-11-08 10:37:36 -07:00
|
|
|
endif
|
2010-09-18 17:21:15 -06:00
|
|
|
else
|
2015-07-11 09:40:14 -06:00
|
|
|
$(2)_DL_VERSION := $$(strip $$($(2)_VERSION))
|
2009-11-08 10:37:36 -07:00
|
|
|
endif
|
2015-07-11 09:40:14 -06:00
|
|
|
$(2)_VERSION := $$(call sanitize,$$($(2)_DL_VERSION))
|
2009-11-08 10:37:36 -07:00
|
|
|
|
core: add a variable that points to the package's hash file
When a package has a version selection (e.g. Qt5), the licensing terms
may be different across versions, but lie in similarly named files (e.g.
'LICENSE').
However, when we check a file, all the hashes for it must match. So, we
can't have the hashes for two different content of the same file. We
overcame that limitation in the legal-license-file macro, which checks
whether a package has a .hash file in a versioned subdir.
For consistency, we would like to also store the source hashes in that
per-version subdir.
Rather than reconstruct the path to the hash file everywhere we need it,
add a variable that points to it.
Existing users will be converted over in followup patches.
Note: the check for a missing hash file is done in the check-hash helper
script, so this variable must always yield a filename, even of a missing
file, thus we do not use $(wildcard...) to resolve the hash file path;
we use $(wildcard...) only to check if the versioned .hash file exists.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Baruch Siach <baruch@tkos.co.il>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-10-14 06:25:40 -06:00
|
|
|
$(2)_HASH_FILE = \
|
|
|
|
|
$$(strip \
|
|
|
|
|
$$(if $$(wildcard $$($(2)_PKGDIR)/$$($(2)_VERSION)/$$($(2)_RAWNAME).hash),\
|
|
|
|
|
$$($(2)_PKGDIR)/$$($(2)_VERSION)/$$($(2)_RAWNAME).hash,\
|
|
|
|
|
$$($(2)_PKGDIR)/$$($(2)_RAWNAME).hash))
|
|
|
|
|
|
2015-07-19 07:15:28 -06:00
|
|
|
ifdef $(3)_OVERRIDE_SRCDIR
|
|
|
|
|
$(2)_OVERRIDE_SRCDIR ?= $$($(3)_OVERRIDE_SRCDIR)
|
|
|
|
|
endif
|
|
|
|
|
|
core: rename FOO_BASE_NAME to FOO_BASENAME to avoid clashes
In current Buildroot, clashes occur between the variables _NAME and
_BASE_NAME for two packages called foo and foo-base, i.e.
Package foo:
FOO_NAME = foo
FOO_BASE_NAME = foo-1.2.3
Package foo-base:
FOO_BASE_NAME = foo-base
FOO_BASE_BASE_NAME = foo-base-4.5.6
where variable FOO_BASE_NAME is clashing between these two packages.
Specific cases where this clash is already existing are:
- alljoyn-base
- alljoyn-tcl-base
- perl-xml-sax-base
The problem is generic and can occur for a number of variables in Buildroot.
A non-exhaustive list:
<pkg>_BASE and <pkg>_BASE_NAME
<pkg>_BASE_NAME and <pkg>_RAW_BASE_NAME
<pkg>_DIR and <pkg>_DL_DIR
<pkg>_VERSION and <pkg>_DL_VERSION
<pkg>_SOURCE and <pkg>_TARGET_SOURCE
<pkg>_INSTALL_IMAGES and <pkg>_TARGET_INSTALL_IMAGES (same for _STAGING and _TARGET)
<pkg>_LICENSE_FILES and <pkg>_MANIFEST_LICENSE_FILES
<pkg>_DEPENDENCIES and <pkg>_FINAL_DEPENDENCIES
One solution is to use another separator than '_' to separate the
package name from the rest of the variable name. For example, a double
underscore:
FOO__NAME
FOO__BASE_NAME
FOO_BASE__NAME
FOO_BASE__BASE_NAME
However, making that change for only this case means that the variable
naming is no longer consistent. And making the change for all variables has
a large impact, also on certain user scripts.
For now, keep it simple, and rename FOO_BASE_NAME into FOO_BASENAME, so that
the variables become:
FOO_NAME
FOO_BASENAME
FOO_BASE_NAME
FOO_BASE_BASENAME
For consistency, also adapt FOO_RAW_BASE_NAME. Since FOO_RAW_BASENAME would
still pose a conflict with a package called 'foo-raw', take the opportunity
to rename it into FOO_BASENAME_RAW instead, which does not pose a conflict
as we have no variable called FOO_RAW.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Sam Voss <sam.voss@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-06 05:59:23 -07:00
|
|
|
$(2)_BASENAME = $$(if $$($(2)_VERSION),$(1)-$$($(2)_VERSION),$(1))
|
|
|
|
|
$(2)_BASENAME_RAW = $$(if $$($(2)_VERSION),$$($(2)_RAWNAME)-$$($(2)_VERSION),$$($(2)_RAWNAME))
|
2018-04-02 08:57:56 -06:00
|
|
|
$(2)_DL_SUBDIR ?= $$($(2)_RAWNAME)
|
|
|
|
|
$(2)_DL_DIR = $$(DL_DIR)/$$($(2)_DL_SUBDIR)
|
core: rename FOO_BASE_NAME to FOO_BASENAME to avoid clashes
In current Buildroot, clashes occur between the variables _NAME and
_BASE_NAME for two packages called foo and foo-base, i.e.
Package foo:
FOO_NAME = foo
FOO_BASE_NAME = foo-1.2.3
Package foo-base:
FOO_BASE_NAME = foo-base
FOO_BASE_BASE_NAME = foo-base-4.5.6
where variable FOO_BASE_NAME is clashing between these two packages.
Specific cases where this clash is already existing are:
- alljoyn-base
- alljoyn-tcl-base
- perl-xml-sax-base
The problem is generic and can occur for a number of variables in Buildroot.
A non-exhaustive list:
<pkg>_BASE and <pkg>_BASE_NAME
<pkg>_BASE_NAME and <pkg>_RAW_BASE_NAME
<pkg>_DIR and <pkg>_DL_DIR
<pkg>_VERSION and <pkg>_DL_VERSION
<pkg>_SOURCE and <pkg>_TARGET_SOURCE
<pkg>_INSTALL_IMAGES and <pkg>_TARGET_INSTALL_IMAGES (same for _STAGING and _TARGET)
<pkg>_LICENSE_FILES and <pkg>_MANIFEST_LICENSE_FILES
<pkg>_DEPENDENCIES and <pkg>_FINAL_DEPENDENCIES
One solution is to use another separator than '_' to separate the
package name from the rest of the variable name. For example, a double
underscore:
FOO__NAME
FOO__BASE_NAME
FOO_BASE__NAME
FOO_BASE__BASE_NAME
However, making that change for only this case means that the variable
naming is no longer consistent. And making the change for all variables has
a large impact, also on certain user scripts.
For now, keep it simple, and rename FOO_BASE_NAME into FOO_BASENAME, so that
the variables become:
FOO_NAME
FOO_BASENAME
FOO_BASE_NAME
FOO_BASE_BASENAME
For consistency, also adapt FOO_RAW_BASE_NAME. Since FOO_RAW_BASENAME would
still pose a conflict with a package called 'foo-raw', take the opportunity
to rename it into FOO_BASENAME_RAW instead, which does not pose a conflict
as we have no variable called FOO_RAW.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Sam Voss <sam.voss@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-06 05:59:23 -07:00
|
|
|
$(2)_DIR = $$(BUILD_DIR)/$$($(2)_BASENAME)
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2012-07-24 01:17:56 -06:00
|
|
|
ifndef $(2)_SUBDIR
|
|
|
|
|
ifdef $(3)_SUBDIR
|
|
|
|
|
$(2)_SUBDIR = $$($(3)_SUBDIR)
|
2012-07-22 07:28:34 -06:00
|
|
|
else
|
2012-07-24 01:17:56 -06:00
|
|
|
$(2)_SUBDIR ?=
|
2012-07-22 07:28:34 -06:00
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
2015-07-11 08:15:04 -06:00
|
|
|
ifndef $(2)_STRIP_COMPONENTS
|
|
|
|
|
ifdef $(3)_STRIP_COMPONENTS
|
|
|
|
|
$(2)_STRIP_COMPONENTS = $$($(3)_STRIP_COMPONENTS)
|
|
|
|
|
else
|
|
|
|
|
$(2)_STRIP_COMPONENTS ?= 1
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
2012-07-22 07:28:34 -06:00
|
|
|
$(2)_SRCDIR = $$($(2)_DIR)/$$($(2)_SUBDIR)
|
|
|
|
|
$(2)_BUILDDIR ?= $$($(2)_SRCDIR)
|
|
|
|
|
|
2011-09-29 13:57:37 -06:00
|
|
|
ifneq ($$($(2)_OVERRIDE_SRCDIR),)
|
|
|
|
|
$(2)_VERSION = custom
|
|
|
|
|
endif
|
|
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
ifndef $(2)_SOURCE
|
|
|
|
|
ifdef $(3)_SOURCE
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 13:12:24 -06:00
|
|
|
$(2)_SOURCE = $$($(3)_SOURCE)
|
2016-07-03 03:49:49 -06:00
|
|
|
else ifdef $(2)_VERSION
|
2019-03-25 13:59:20 -06:00
|
|
|
$(2)_SOURCE ?= $$($(2)_BASENAME_RAW)$$(call pkg_source_ext,$(2))
|
2009-11-08 10:37:36 -07:00
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
2016-05-07 10:14:37 -06:00
|
|
|
# If FOO_ACTUAL_SOURCE_TARBALL is explicitly defined, it means FOO_SOURCE is
|
|
|
|
|
# indeed a binary (e.g. external toolchain) and FOO_ACTUAL_SOURCE_TARBALL/_SITE
|
|
|
|
|
# point to the actual sources tarball. Use the actual sources for legal-info.
|
|
|
|
|
# For most packages the FOO_SITE/FOO_SOURCE pair points to real source code,
|
|
|
|
|
# so these are the defaults for FOO_ACTUAL_*.
|
|
|
|
|
$(2)_ACTUAL_SOURCE_TARBALL ?= $$($(2)_SOURCE)
|
|
|
|
|
$(2)_ACTUAL_SOURCE_SITE ?= $$(call qstrip,$$($(2)_SITE))
|
|
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
ifndef $(2)_PATCH
|
|
|
|
|
ifdef $(3)_PATCH
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 13:12:24 -06:00
|
|
|
$(2)_PATCH = $$($(3)_PATCH)
|
2009-11-08 10:37:36 -07:00
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
2015-04-26 03:51:11 -06:00
|
|
|
$(2)_ALL_DOWNLOADS = \
|
2018-05-03 14:17:57 -06:00
|
|
|
$$(if $$($(2)_SOURCE),$$($(2)_SITE_METHOD)+$$($(2)_SITE)/$$($(2)_SOURCE)) \
|
core/pkg-infra: don't enforce site-method for extra downloads
The site method only ever applies to the main download, while extra
downloads are always to be fetched with wget.
However, the site method is prepended to the URL from within the
DOWNLOAD macro (well, a variable evaluated in the DOWNLOAD macro),
which is called for each download of a package, thus effectively
prepending the site method to all downloads, even the extra ones (and
the patches).
We fix that by prepending the site method from within the
generic-package infra, so that it only applies to the main download.
For that, we move the main _SOURCE out of the foreach loop, so that
we can prepend the site-method to it, without impacting the other
downloads.
Reported-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Maxime Hadjinlian <maxime.hadjinlian@gmail.com>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-04-25 13:41:53 -06:00
|
|
|
$$(foreach p,$$($(2)_PATCH) $$($(2)_EXTRA_DOWNLOADS),\
|
2015-04-26 03:51:11 -06:00
|
|
|
$$(if $$(findstring ://,$$(p)),$$(p),\
|
package/pkg-generic.mk: use site method for same-site extra downloads
When a package specifies extra downloads, it has the option to only name
the basename of the extra download, in which case that extra download
will be retrieved from the same location the main download is retrieved
from.
In that case, if the extra download contains a '+', it would confuse the
dl-wrapper, which believes the LHS of the '+' is the site method, and
the RHS the actual URI, and so the dl-wrapper mangles and damages the
URI when fetching such extra downloads, like that happens with android
tools, where the proper URI and mangled URIs of the extra download are,
respectively:
https://launchpad.net/ubuntu/+archive/primary/+files/android-tools_4.2.2+git20130218-3ubuntu41.debian.tar.gz
http://archive/primary/+files/android-tools_4.2.2+git20130218-3ubuntu41.debian.tar.gz
We fix that by always propagating the site method to extra downloads,
but only when they are specified as relative to the main download URI.
For the extra downloads that specify a full URI, it is not systematic
that it is the same site method. For example, a main download could be a
git clone, but an extra download a pure http download; in that case we
can't replicate the site method for extra downloads, so they'll have to
take appropriate care to specify the required method and encoding if
needed.
Reported-by: Jemy Zhang <jemy.zhang@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Jemy Zhang <jemy.zhang@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-11-08 10:26:45 -07:00
|
|
|
$$($(2)_SITE_METHOD)+$$($(2)_SITE)/$$(p)))
|
2015-04-26 03:51:11 -06:00
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
ifndef $(2)_SITE
|
|
|
|
|
ifdef $(3)_SITE
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 13:12:24 -06:00
|
|
|
$(2)_SITE = $$($(3)_SITE)
|
2009-11-08 10:37:36 -07:00
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
Implement basic non-wget download methods
Packages can now be sourced from Git and Subversion repositories. The
download method will be autodetected from the URI (git://, svn://, etc).
If the repository is accessed through http(s), you can force the
download method by setting a _SITE_METHOD variable to either 'git' or
'svn', respectively and without the quotes.
The package's _VERSION variable defines which commit, revision, tag or
branch should be checked out. For Git, it can be HEAD, a commit ID, a
tag name or branch name (anything that can be checked out with `git
checkout`). For Subversion, it must be a revision number, or HEAD.
Signed-off-by: Maxime Petazzoni <maxime.petazzoni@bulix.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2010-09-02 04:09:47 -06:00
|
|
|
ifndef $(2)_SITE_METHOD
|
|
|
|
|
ifdef $(3)_SITE_METHOD
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 13:12:24 -06:00
|
|
|
$(2)_SITE_METHOD = $$($(3)_SITE_METHOD)
|
Implement basic non-wget download methods
Packages can now be sourced from Git and Subversion repositories. The
download method will be autodetected from the URI (git://, svn://, etc).
If the repository is accessed through http(s), you can force the
download method by setting a _SITE_METHOD variable to either 'git' or
'svn', respectively and without the quotes.
The package's _VERSION variable defines which commit, revision, tag or
branch should be checked out. For Git, it can be HEAD, a commit ID, a
tag name or branch name (anything that can be checked out with `git
checkout`). For Subversion, it must be a revision number, or HEAD.
Signed-off-by: Maxime Petazzoni <maxime.petazzoni@bulix.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2010-09-02 04:09:47 -06:00
|
|
|
else
|
|
|
|
|
# Try automatic detection using the scheme part of the URI
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 13:12:24 -06:00
|
|
|
$(2)_SITE_METHOD = $$(call geturischeme,$$($(2)_SITE))
|
Implement basic non-wget download methods
Packages can now be sourced from Git and Subversion repositories. The
download method will be autodetected from the URI (git://, svn://, etc).
If the repository is accessed through http(s), you can force the
download method by setting a _SITE_METHOD variable to either 'git' or
'svn', respectively and without the quotes.
The package's _VERSION variable defines which commit, revision, tag or
branch should be checked out. For Git, it can be HEAD, a commit ID, a
tag name or branch name (anything that can be checked out with `git
checkout`). For Subversion, it must be a revision number, or HEAD.
Signed-off-by: Maxime Petazzoni <maxime.petazzoni@bulix.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2010-09-02 04:09:47 -06:00
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
2019-11-28 11:55:52 -07:00
|
|
|
ifndef $(2)_DL_OPTS
|
|
|
|
|
ifdef $(3)_DL_OPTS
|
|
|
|
|
$(2)_DL_OPTS = $$($(3)_DL_OPTS)
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
2020-05-24 15:28:29 -06:00
|
|
|
ifneq ($$(filter bzr cvs hg,$$($(2)_SITE_METHOD)),)
|
2018-06-28 19:42:46 -06:00
|
|
|
BR_NO_CHECK_HASH_FOR += $$($(2)_SOURCE)
|
|
|
|
|
endif
|
|
|
|
|
|
2021-01-13 15:10:14 -07:00
|
|
|
ifndef $(2)_GIT_SUBMODULES
|
|
|
|
|
ifdef $(3)_GIT_SUBMODULES
|
|
|
|
|
$(2)_GIT_SUBMODULES = $$($(3)_GIT_SUBMODULES)
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
2016-07-01 03:01:19 -06:00
|
|
|
# Do not accept to download git submodule if not using the git method
|
|
|
|
|
ifneq ($$($(2)_GIT_SUBMODULES),)
|
|
|
|
|
ifneq ($$($(2)_SITE_METHOD),git)
|
|
|
|
|
$$(error $(2) declares having git sub-modules, but does not use the \
|
|
|
|
|
'git' method (uses '$$($(2)_SITE_METHOD)' instead))
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
2011-09-29 13:57:40 -06:00
|
|
|
ifeq ($$($(2)_SITE_METHOD),local)
|
|
|
|
|
ifeq ($$($(2)_OVERRIDE_SRCDIR),)
|
2013-10-12 04:15:08 -06:00
|
|
|
$(2)_OVERRIDE_SRCDIR = $$($(2)_SITE)
|
2011-09-29 13:57:40 -06:00
|
|
|
endif
|
2018-05-23 03:01:08 -06:00
|
|
|
ifeq ($$($(2)_OVERRIDE_SRCDIR),)
|
|
|
|
|
$$(error $(1) has local site method, but `$(2)_SITE` is not defined)
|
|
|
|
|
endif
|
2011-09-29 13:57:40 -06:00
|
|
|
endif
|
|
|
|
|
|
2012-05-17 11:33:00 -06:00
|
|
|
ifndef $(2)_LICENSE
|
|
|
|
|
ifdef $(3)_LICENSE
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 13:12:24 -06:00
|
|
|
$(2)_LICENSE = $$($(3)_LICENSE)
|
2012-05-17 11:33:00 -06:00
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
2012-11-01 17:25:40 -06:00
|
|
|
$(2)_LICENSE ?= unknown
|
|
|
|
|
|
2012-05-17 11:33:00 -06:00
|
|
|
ifndef $(2)_LICENSE_FILES
|
|
|
|
|
ifdef $(3)_LICENSE_FILES
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 13:12:24 -06:00
|
|
|
$(2)_LICENSE_FILES = $$($(3)_LICENSE_FILES)
|
2012-05-17 11:33:00 -06:00
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
2012-11-02 03:25:41 -06:00
|
|
|
ifndef $(2)_REDISTRIBUTE
|
|
|
|
|
ifdef $(3)_REDISTRIBUTE
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 13:12:24 -06:00
|
|
|
$(2)_REDISTRIBUTE = $$($(3)_REDISTRIBUTE)
|
2012-11-02 03:25:41 -06:00
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
|
|
|
|
$(2)_REDISTRIBUTE ?= YES
|
|
|
|
|
|
core: rename FOO_BASE_NAME to FOO_BASENAME to avoid clashes
In current Buildroot, clashes occur between the variables _NAME and
_BASE_NAME for two packages called foo and foo-base, i.e.
Package foo:
FOO_NAME = foo
FOO_BASE_NAME = foo-1.2.3
Package foo-base:
FOO_BASE_NAME = foo-base
FOO_BASE_BASE_NAME = foo-base-4.5.6
where variable FOO_BASE_NAME is clashing between these two packages.
Specific cases where this clash is already existing are:
- alljoyn-base
- alljoyn-tcl-base
- perl-xml-sax-base
The problem is generic and can occur for a number of variables in Buildroot.
A non-exhaustive list:
<pkg>_BASE and <pkg>_BASE_NAME
<pkg>_BASE_NAME and <pkg>_RAW_BASE_NAME
<pkg>_DIR and <pkg>_DL_DIR
<pkg>_VERSION and <pkg>_DL_VERSION
<pkg>_SOURCE and <pkg>_TARGET_SOURCE
<pkg>_INSTALL_IMAGES and <pkg>_TARGET_INSTALL_IMAGES (same for _STAGING and _TARGET)
<pkg>_LICENSE_FILES and <pkg>_MANIFEST_LICENSE_FILES
<pkg>_DEPENDENCIES and <pkg>_FINAL_DEPENDENCIES
One solution is to use another separator than '_' to separate the
package name from the rest of the variable name. For example, a double
underscore:
FOO__NAME
FOO__BASE_NAME
FOO_BASE__NAME
FOO_BASE__BASE_NAME
However, making that change for only this case means that the variable
naming is no longer consistent. And making the change for all variables has
a large impact, also on certain user scripts.
For now, keep it simple, and rename FOO_BASE_NAME into FOO_BASENAME, so that
the variables become:
FOO_NAME
FOO_BASENAME
FOO_BASE_NAME
FOO_BASE_BASENAME
For consistency, also adapt FOO_RAW_BASE_NAME. Since FOO_RAW_BASENAME would
still pose a conflict with a package called 'foo-raw', take the opportunity
to rename it into FOO_BASENAME_RAW instead, which does not pose a conflict
as we have no variable called FOO_RAW.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Sam Voss <sam.voss@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-06 05:59:23 -07:00
|
|
|
$(2)_REDIST_SOURCES_DIR = $$(REDIST_SOURCES_DIR_$$(call UPPERCASE,$(4)))/$$($(2)_BASENAME_RAW)
|
2016-05-07 10:14:29 -06:00
|
|
|
|
package/pkg-generic.mk: add CPE ID related package variables
Currently, the match between Buildroot packages and CVEs is solely
based on the package names. Unfortunately, as one can imagine, there
isn't necessarily a strict mapping between Buildroot package names,
and how software projects are referenced in the National Vulnerability
Database (NVD) which we use.
The NVD has defined the concept of CPE (Common Platform Enumeration)
identifiers, which uniquely identifies software components based on
string looking like this:
cpe:2.3:a:netsurf-browser:libnsbmp:0.1.2:*:*:*:*:*:*:*
In particular, this CPE identifier contains a vendor name (here
"netsurf-browser"), a product name (here "libnsbmp") and a version
(here "0.1.2").
This patch series introduces the concept of CPE ID in Buildroot, where
each package can be associated to a CPE ID. A package can define one
or several of:
- <pkg>_CPE_ID_VENDOR
- <pkg>_CPE_ID_PRODUCT
- <pkg>_CPE_ID_VERSION
- <pkg>_CPE_ID_VERSION_MINOR
- <pkg>_CPE_ID_PREFIX
If one or several of those variables are defined, then the
<pkg>_CPE_ID will be defined by the generic package infrastructure as
follows:
$(2)_CPE_ID = $$($(2)_CPE_ID_PREFIX):$$($(2)_CPE_ID_VENDOR):$$($(2)_CPE_ID_NAME):$$($(2)_CPE_ID_VERSION):$$($(2)_CPE_ID_VERSION_MINOR):*:*:*:*:*:*
<pkg>_CPE_ID_* variables that are not explicitly specified by the
package will carry a default value defined by the generic package
infrastructure.
If a package is happy with the default <pkg>_CPE_ID, and therefore
does not need to define any of <pkg>_CPE_ID_{VENDOR,PRODUCT,...}, it
can set <pkg>_CPE_ID_VALID = YES.
If any of the <pkg>_CPE_ID_{VENDOR,PRODUCT,...} variables are defined
by the package, then <pkg>_CPE_ID_VALID = YES will be set by the
generic package infrastructure.
Then, it's only if <pkg>_CPE_ID_VALID = YES that a <pkg>_CPE_ID will
be defined. Indeed, we want to be able to distinguish packages for
which the CPE ID information has been checked and is considered valid,
from packages for which the CPE ID information has never been
verified. For this reason, we cannot simply define a default value
for <pkg>_CPE_ID.
The <pkg>_CPE_ID_* values for the host package are inherited from the
same variables of the corresponding target package, as we normally do
for most package variables.
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Reviewed-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-11-04 07:51:37 -07:00
|
|
|
# If any of the <pkg>_CPE_ID_* variables are set, we assume the CPE ID
|
|
|
|
|
# information is valid for this package.
|
|
|
|
|
ifneq ($$($(2)_CPE_ID_VENDOR)$$($(2)_CPE_ID_NAME)$$($(2)_CPE_ID_VERSION)$$($(2)_CPE_ID_VERSION_MINOR)$$($(2)_CPE_ID_PREFIX),)
|
|
|
|
|
$(2)_CPE_ID_VALID = YES
|
|
|
|
|
endif
|
|
|
|
|
|
|
|
|
|
# When we're a host package, make sure to use the variables of the
|
|
|
|
|
# corresponding target package, if any.
|
|
|
|
|
ifneq ($$($(3)_CPE_ID_VENDOR)$$($(3)_CPE_ID_NAME)$$($(3)_CPE_ID_VERSION)$$($(3)_CPE_ID_VERSION_MINOR)$$($(3)_CPE_ID_PREFIX),)
|
|
|
|
|
$(2)_CPE_ID_VALID = YES
|
|
|
|
|
endif
|
|
|
|
|
|
|
|
|
|
# If the CPE ID is valid for the target package so it is for the host
|
|
|
|
|
# package
|
|
|
|
|
ifndef $(2)_CPE_ID_VALID
|
|
|
|
|
ifdef $(3)_CPE_ID_VALID
|
|
|
|
|
$(2)_CPE_ID_VALID = $$($(3)_CPE_ID_VALID)
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
|
|
|
|
ifeq ($$($(2)_CPE_ID_VALID),YES)
|
|
|
|
|
# CPE_ID_VENDOR
|
|
|
|
|
ifndef $(2)_CPE_ID_VENDOR
|
|
|
|
|
ifdef $(3)_CPE_ID_VENDOR
|
|
|
|
|
$(2)_CPE_ID_VENDOR = $$($(3)_CPE_ID_VENDOR)
|
|
|
|
|
else
|
|
|
|
|
$(2)_CPE_ID_VENDOR = $$($(2)_RAWNAME)_project
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
|
|
|
|
# CPE_ID_NAME
|
|
|
|
|
ifndef $(2)_CPE_ID_NAME
|
|
|
|
|
ifdef $(3)_CPE_ID_NAME
|
|
|
|
|
$(2)_CPE_ID_NAME = $$($(3)_CPE_ID_NAME)
|
|
|
|
|
else
|
|
|
|
|
$(2)_CPE_ID_NAME = $$($(2)_RAWNAME)
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
|
|
|
|
# CPE_ID_VERSION
|
|
|
|
|
ifndef $(2)_CPE_ID_VERSION
|
|
|
|
|
ifdef $(3)_CPE_ID_VERSION
|
|
|
|
|
$(2)_CPE_ID_VERSION = $$($(3)_CPE_ID_VERSION)
|
|
|
|
|
else
|
|
|
|
|
$(2)_CPE_ID_VERSION = $$($(2)_VERSION)
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
|
|
|
|
# CPE_ID_VERSION_MINOR
|
|
|
|
|
ifndef $(2)_CPE_ID_VERSION_MINOR
|
|
|
|
|
ifdef $(3)_CPE_ID_VERSION_MINOR
|
|
|
|
|
$(2)_CPE_ID_VERSION_MINOR = $$($(3)_CPE_ID_VERSION_MINOR)
|
|
|
|
|
else
|
|
|
|
|
$(2)_CPE_ID_VERSION_MINOR = *
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
|
|
|
|
# CPE_ID_PREFIX
|
|
|
|
|
ifndef $(2)_CPE_ID_PREFIX
|
|
|
|
|
ifdef $(3)_CPE_ID_PREFIX
|
|
|
|
|
$(2)_CPE_ID_PREFIX = $$($(3)_CPE_ID_PREFIX)
|
|
|
|
|
else
|
|
|
|
|
$(2)_CPE_ID_PREFIX = cpe:2.3:a
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
|
|
|
|
# Calculate complete CPE ID
|
|
|
|
|
$(2)_CPE_ID = $$($(2)_CPE_ID_PREFIX):$$($(2)_CPE_ID_VENDOR):$$($(2)_CPE_ID_NAME):$$($(2)_CPE_ID_VERSION):$$($(2)_CPE_ID_VERSION_MINOR):*:*:*:*:*:*
|
|
|
|
|
endif # ifeq ($$($(2)_CPE_ID_VALID),YES)
|
|
|
|
|
|
2014-02-14 02:55:04 -07:00
|
|
|
# When a target package is a toolchain dependency set this variable to
|
|
|
|
|
# 'NO' so the 'toolchain' dependency is not added to prevent a circular
|
2017-07-18 11:25:28 -06:00
|
|
|
# dependency.
|
|
|
|
|
# Similarly for the skeleton.
|
2014-02-14 02:55:04 -07:00
|
|
|
$(2)_ADD_TOOLCHAIN_DEPENDENCY ?= YES
|
2017-07-18 11:25:28 -06:00
|
|
|
$(2)_ADD_SKELETON_DEPENDENCY ?= YES
|
|
|
|
|
|
2012-11-02 03:25:41 -06:00
|
|
|
|
2014-02-14 02:55:04 -07:00
|
|
|
ifeq ($(4),target)
|
2017-07-18 11:25:28 -06:00
|
|
|
ifeq ($$($(2)_ADD_SKELETON_DEPENDENCY),YES)
|
2015-07-14 05:36:25 -06:00
|
|
|
$(2)_DEPENDENCIES += skeleton
|
|
|
|
|
endif
|
2014-02-14 02:55:04 -07:00
|
|
|
ifeq ($$($(2)_ADD_TOOLCHAIN_DEPENDENCY),YES)
|
|
|
|
|
$(2)_DEPENDENCIES += toolchain
|
|
|
|
|
endif
|
|
|
|
|
endif
|
2012-01-16 06:58:35 -07:00
|
|
|
|
2018-03-24 08:20:00 -06:00
|
|
|
ifneq ($(1),host-skeleton)
|
|
|
|
|
$(2)_DEPENDENCIES += host-skeleton
|
|
|
|
|
endif
|
|
|
|
|
|
2018-05-08 14:40:19 -06:00
|
|
|
ifneq ($$(filter cvs git svn,$$($(2)_SITE_METHOD)),)
|
support/dependencies: add a check for a suitable gzip
Recently, some hash mismatch have been reported, both by users as well
as autobuilder failures, about tarballs generated from git repositories.
This turned out to be caused by users having the 'gzip' command somehow
aliased to 'pigz' (which stand for: parallel implementation of gzip,
which takes advantage of multi-processor system to parallelise the
compression).
Unfortunately, the output of pigz-compressed archives differ from that
of gzip (even though they *are* valid gzip-compressed streams).
Add a dependency check that ensures that gzip is not pigz. If that is
the case, define a conditional dependency to host-gzip, that is used as
a download dependency for packages that will generate compressed files,
i.e. cvs, git, and svn.
Fixes:
http://autobuild.buildroot.org/results/330/3308271fc641cadb59dbf1b5ee529a84f79e6d5c/
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Marcin Niestrój <m.niestroj@grinn-global.com>
Cc: Erico Nunes <nunes.erico@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-11-17 10:15:51 -07:00
|
|
|
$(2)_DOWNLOAD_DEPENDENCIES += \
|
|
|
|
|
$(BR2_GZIP_HOST_DEPENDENCY) \
|
|
|
|
|
$(BR2_TAR_HOST_DEPENDENCY)
|
2018-05-08 14:40:19 -06:00
|
|
|
endif
|
|
|
|
|
|
package/pkg-generic: postpone evaluation of dependency conditions
In the pkg-inner macros, all variables, but the positional arguments,
must be $$-prefixed, so that they are expanded only when the macro is
evaluated in each package, not when the macro is parsed.
It is to be noted, though, that the current code, even though
incorrect by the above rules, seemed to work. However, the upcoming
addition of download dependencies, mimicking that code, would not work
unless it was $$-prefixed.
So, for consistency sake, and for correctness sake, let's always use
the $$-prefix in the inner macro.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-08 14:40:17 -06:00
|
|
|
ifeq ($$(filter host-tar host-skeleton host-fakedate,$(1)),)
|
|
|
|
|
$(2)_EXTRACT_DEPENDENCIES += $$(BR2_TAR_HOST_DEPENDENCY)
|
2018-03-24 08:20:03 -06:00
|
|
|
endif
|
|
|
|
|
|
package/pkg-generic: postpone evaluation of dependency conditions
In the pkg-inner macros, all variables, but the positional arguments,
must be $$-prefixed, so that they are expanded only when the macro is
evaluated in each package, not when the macro is parsed.
It is to be noted, though, that the current code, even though
incorrect by the above rules, seemed to work. However, the upcoming
addition of download dependencies, mimicking that code, would not work
unless it was $$-prefixed.
So, for consistency sake, and for correctness sake, let's always use
the $$-prefix in the inner macro.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-08 14:40:17 -06:00
|
|
|
ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate,$(1)),)
|
package/pkg-generic.mk: also apply extractor-pkg-dependency to <pkg>_EXTRA_DOWNLOADS
For now, the extractor dependencies were only calculated for
<pkg>_SOURCE, so if the package manually downloads another file using
<pkg>_EXTRA_DOWNLOADS and then extracts it with $(call
suitable-extractor), we are missing the corresponding dependency on
the appropriate extracting tool.
Since the vast majority of <pkg>_EXTRA_DOWNLOADS are compressed files
that will be uncompressed at build time, it makes sense to derive the
corresponding extractor dependencies directly in the common package
infrastructure, rather than having each and every package using
<pkg>_EXTRA_DOWNLOADS making this effort.
On a system without xzcat, before this patch:
$ make printvars VARS=HOST_GETTEXT_TINY_EXTRACT_DEPENDENCIES
HOST_GETTEXT_TINY_EXTRACT_DEPENDENCIES=host-tar
After this patch:
$ make printvars VARS=HOST_GETTEXT_TINY_EXTRACT_DEPENDENCIES
HOST_GETTEXT_TINY_EXTRACT_DEPENDENCIES=host-tar host-xz
This commit most notably fixes the build of host-gettext-tiny on
systems without xzcat, and with per-package support enabled. Indeed,
the main _SOURCE for gettext-tiny is a .gz file, but it has a .xz file
in its _EXTRA_DOWNLOADS, which is then extracted. Except that xzcat
being missing from the dependencies, it is not built.
Fixes:
http://autobuild.buildroot.net/results/83c6d47c06334bef27791a59bdd491b1de124c49/
Suggested-by: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2019-12-16 08:31:17 -07:00
|
|
|
$(2)_EXTRACT_DEPENDENCIES += \
|
|
|
|
|
$$(foreach dl,$$($(2)_ALL_DOWNLOADS),\
|
|
|
|
|
$$(call extractor-pkg-dependency,$$(notdir $$(dl))))
|
2019-03-17 15:20:13 -06:00
|
|
|
endif
|
package/pkg-generic: handle host-lzip as an extract dependency
This moves the host-lzip dependency handling from
DEPENDENCY_HOST_PREREQ to an extract dependency.
To achieve that, check-host-lzip.mk fills in the
BR2_LZIP_HOST_DEPENDENCY with host-lzip if building a host-lzip is
needed. The name BR2_LZIP_HOST_DEPENDENCY has been chosen because it
matches the name BR2_CMAKE_HOST_DEPENDENCY already used in
check-host-cmake.mk.
The BR2_LZIP_HOST_DEPENDENCY is added to all packages, except:
- host-lzip, because we would otherwise depend on ourself.
- host-tar, because lzip itself is delivered as a tarball, so we need
to have host-lzip depend on host-tar, and not host-tar depend on
host-lzip
- host-skeleton, because we need to have host-lzip depend on
host-skeleton, and not the opposite.
We also mutually exclude host-lzip and host-xz from dependending on
each other, to avoid a circular dependency.
In addition, we modify lzip.mk to explicitly build host-lzip without
ccache. We generally took the approach of building host-ccache *after*
all the extractors have been built.
[Peter: fix s/host-tar/host-lzip/ typo, fix s/xz/lzip/ typo]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-24 08:20:05 -06:00
|
|
|
|
package/pkg-generic: postpone evaluation of dependency conditions
In the pkg-inner macros, all variables, but the positional arguments,
must be $$-prefixed, so that they are expanded only when the macro is
evaluated in each package, not when the macro is parsed.
It is to be noted, though, that the current code, even though
incorrect by the above rules, seemed to work. However, the upcoming
addition of download dependencies, mimicking that code, would not work
unless it was $$-prefixed.
So, for consistency sake, and for correctness sake, let's always use
the $$-prefix in the inner macro.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-08 14:40:17 -06:00
|
|
|
ifeq ($$(BR2_CCACHE),y)
|
|
|
|
|
ifeq ($$(filter host-tar host-skeleton host-xz host-lzip host-fakedate host-ccache,$(1)),)
|
2018-03-24 08:20:06 -06:00
|
|
|
$(2)_DEPENDENCIES += host-ccache
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
package/pkg-generic: postpone evaluation of dependency conditions
In the pkg-inner macros, all variables, but the positional arguments,
must be $$-prefixed, so that they are expanded only when the macro is
evaluated in each package, not when the macro is parsed.
It is to be noted, though, that the current code, even though
incorrect by the above rules, seemed to work. However, the upcoming
addition of download dependencies, mimicking that code, would not work
unless it was $$-prefixed.
So, for consistency sake, and for correctness sake, let's always use
the $$-prefix in the inner macro.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2018-05-08 14:40:17 -06:00
|
|
|
ifeq ($$(BR2_REPRODUCIBLE),y)
|
|
|
|
|
ifeq ($$(filter host-skeleton host-fakedate,$(1)),)
|
2018-03-24 08:20:07 -06:00
|
|
|
$(2)_DEPENDENCIES += host-fakedate
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
2014-06-01 04:28:54 -06:00
|
|
|
# Eliminate duplicates in dependencies
|
|
|
|
|
$(2)_FINAL_DEPENDENCIES = $$(sort $$($(2)_DEPENDENCIES))
|
2018-05-08 14:40:18 -06:00
|
|
|
$(2)_FINAL_DOWNLOAD_DEPENDENCIES = $$(sort $$($(2)_DOWNLOAD_DEPENDENCIES))
|
package/pkg-generic: add the concept of extract dependency
Extract dependencies are dependencies that must be ready before the
extract step of a package, i.e for tools that are needed to extract
packages themselves. Current examples of such tools are host-tar,
host-lzip and host-xz.
They are currently handled through DEPENDENCIES_HOST_PREREQ. However,
this mechanism has a number of drawbacks:
- First and foremost, because host-tar/host-lzip/host-xz are not
listed in the dependencies of packages, the package infrastructure
does not know it should rsync them in the context of per-package
SDK.
- Second, there is no dependency handling *between* them. I.e, we
have no mechanism that says host-tar should be built before
host-lzip, while it is in fact the case: if you need to build
host-lzip, you need to extract a tarball, so you may need host-tar
if your system tarball is not capable enough.
For those reasons, it makes sense to add explicit support for "extract
dependencies" in the package infrastructure, through the
<pkg>_EXTRACT_DEPENDENCIES variable. It is unlikely this variable will
ever be used by a package .mk file, but it will be used internally by
the package infrastructure.
[Peter: fix typo in manual]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-24 08:20:02 -06:00
|
|
|
$(2)_FINAL_EXTRACT_DEPENDENCIES = $$(sort $$($(2)_EXTRACT_DEPENDENCIES))
|
2015-03-14 08:25:17 -06:00
|
|
|
$(2)_FINAL_PATCH_DEPENDENCIES = $$(sort $$($(2)_PATCH_DEPENDENCIES))
|
package/pkg-generic: add the concept of extract dependency
Extract dependencies are dependencies that must be ready before the
extract step of a package, i.e for tools that are needed to extract
packages themselves. Current examples of such tools are host-tar,
host-lzip and host-xz.
They are currently handled through DEPENDENCIES_HOST_PREREQ. However,
this mechanism has a number of drawbacks:
- First and foremost, because host-tar/host-lzip/host-xz are not
listed in the dependencies of packages, the package infrastructure
does not know it should rsync them in the context of per-package
SDK.
- Second, there is no dependency handling *between* them. I.e, we
have no mechanism that says host-tar should be built before
host-lzip, while it is in fact the case: if you need to build
host-lzip, you need to extract a tarball, so you may need host-tar
if your system tarball is not capable enough.
For those reasons, it makes sense to add explicit support for "extract
dependencies" in the package infrastructure, through the
<pkg>_EXTRACT_DEPENDENCIES variable. It is unlikely this variable will
ever be used by a package .mk file, but it will be used internally by
the package infrastructure.
[Peter: fix typo in manual]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-24 08:20:02 -06:00
|
|
|
$(2)_FINAL_ALL_DEPENDENCIES = \
|
|
|
|
|
$$(sort \
|
|
|
|
|
$$($(2)_FINAL_DEPENDENCIES) \
|
2018-05-08 14:40:18 -06:00
|
|
|
$$($(2)_FINAL_DOWNLOAD_DEPENDENCIES) \
|
package/pkg-generic: add the concept of extract dependency
Extract dependencies are dependencies that must be ready before the
extract step of a package, i.e for tools that are needed to extract
packages themselves. Current examples of such tools are host-tar,
host-lzip and host-xz.
They are currently handled through DEPENDENCIES_HOST_PREREQ. However,
this mechanism has a number of drawbacks:
- First and foremost, because host-tar/host-lzip/host-xz are not
listed in the dependencies of packages, the package infrastructure
does not know it should rsync them in the context of per-package
SDK.
- Second, there is no dependency handling *between* them. I.e, we
have no mechanism that says host-tar should be built before
host-lzip, while it is in fact the case: if you need to build
host-lzip, you need to extract a tarball, so you may need host-tar
if your system tarball is not capable enough.
For those reasons, it makes sense to add explicit support for "extract
dependencies" in the package infrastructure, through the
<pkg>_EXTRACT_DEPENDENCIES variable. It is unlikely this variable will
ever be used by a package .mk file, but it will be used internally by
the package infrastructure.
[Peter: fix typo in manual]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-24 08:20:02 -06:00
|
|
|
$$($(2)_FINAL_EXTRACT_DEPENDENCIES) \
|
|
|
|
|
$$($(2)_FINAL_PATCH_DEPENDENCIES))
|
package/pkg-generic: speed up RECURSIVE_FINAL_DEPENDENCIES
Evaluating all the <PKG>_RECURSIVE_FINAL_DEPENDENCIES variables
(abbreviated RFD hereafter) ends up being quite slow. Enough, on a
reasonable modern workstation, to increase the time it takes to run
"make printvars" from 13 seconds in 2018.02 to 371 seconds in 2019.02.
This patch improves this by using dynamic programming to speed the
evaluation of RFD, reducing the before mentioned printvars time to about
14.6 seconds.
The evaluation of PKG1_RFD requires recursively evaluating each of
PKG1's dependencies' RFDs, then their dependencies' RFDs, and so on.
The same is done for PKG2_RFD. But it's likely that many of the
dependencies of PKG2 are the same as PKG1. And when we consider all
packages, the dependencies are re-computed many thousands of times.
To avoid this re-computation we memoize, or save, the computed value of
each RFD variable when it found the first time. Subsequent evaluations
re-use the memoized value.
Surprisingly, this ends up being not all the hard to implement in make.
The basic construct is this:
VAR = $(if !defined(VAR__X),$(eval VAR__X := value))$(VAR__X)
The first time VAR is evaluated VAR__X will not be defined, and code to
set VAR__X to the computed value is eval'd. Then the now defined value
of VAR__X is returned. Subsequent evaluations can just return VAR__X.
It is important to note that VAR is defined with '=', as not enough
information (namely, all packages' dependencies) is know when it is
parsed to find the correct value. VAR will be evaluated each time it is
used. But VAR__X is defined with ":=", so that it is evaluated once
when defined, and not each time it is used.
Signed-off-by: Trent Piepho <tpiepho@impinj.com>
Tested-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Acked-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-02-25 16:55:14 -07:00
|
|
|
$(2)_FINAL_RECURSIVE_DEPENDENCIES = $$(sort \
|
|
|
|
|
$$(if $$(filter undefined,$$(origin $(2)_FINAL_RECURSIVE_DEPENDENCIES__X)), \
|
|
|
|
|
$$(eval $(2)_FINAL_RECURSIVE_DEPENDENCIES__X := \
|
|
|
|
|
$$(foreach p, \
|
|
|
|
|
$$($(2)_FINAL_ALL_DEPENDENCIES), \
|
|
|
|
|
$$(p) \
|
|
|
|
|
$$($$(call UPPERCASE,$$(p))_FINAL_RECURSIVE_DEPENDENCIES) \
|
|
|
|
|
) \
|
|
|
|
|
) \
|
|
|
|
|
) \
|
|
|
|
|
$$($(2)_FINAL_RECURSIVE_DEPENDENCIES__X))
|
2014-06-01 04:28:54 -06:00
|
|
|
|
infra/pkg-generic: use pure Makefile-based recursive dependencies
Calling to the graph-depends script is very costly, as it calls back to
'make' a lot of time.
It turns out that we already have the list of recursive dependencies, so
we can just print that.
As for the recursive reverse dependencies, we use the same memoisation
technique to cut-down on the expansion cost, which would otherwise be on
the order of 𝑶(𝑛²) (with 𝑛 enabled packages).
>From a defconfig, modified to use glibc, C++, wchar, locales, ssp, and
allyespackageconfig (tweaked to avoid multi providers, etc...), the
timings for X-show-recursive-rdepends are:
before after speedup #rdeps
libnss 0m22.932s 0m5.775s 3.97x 3
qt5base 0m41.176s 0m5.781s 7.12x 67
libjpeg 0m56.185s 0m5.749s 9.71x 228
libxml2 0m54.964s 0m5.795s 9.48x 271
freetype 0m46.754s 0m5.819s 8.07x 287
libpng 0m53.577s 0m5.760s 9.30x 303
sqlite 1m15.222s 0m5.807s 12.95x 801
libopenssl 1m25.471s 0m5.844s 14.63x 931
readline 1m13.805s 0m5.775s 12.78x 958
libzlib 1m11.807s 0m5.820s 12.34x 1039
toolchain 1m23.712s 0m6.080s 13.77x 2107
skeleton 1m27.839s 0m6.293s 13.96x 2111 (+1)
host-skeleton 1m27.405s 0m6.350s 13.76x 2172 (+2)
- speedup: ratio before/after
- #rdeps: number of recursive reverse dependencies, with the extra
dependencies returned with this patch, see below for the
reason.
So, for a low-level package with a lot of reverse dependencies, like
libzlibz, libopenssl or readline are, the timings are already very much
in favour of the change. This is less impressive with packages that
have few dependencies (libnss), but still much faster.
Also, remember that the config tested has as much packages enabled as
possible, so is in itself a degenerate case. With simpler and more
realistic configurations, the gains would probably be a bit lower than
reported above, but various tests still report good improvements
overall (note: coming up with a 'realistic' configuration is pretty
hard, as everyone and their dog have their notion of what is realistic
in their context, so nothing displayed here; timings are left as an
exercise for the interested parties to report aggravation in their
cases should they notice some regression).
Note that, more recursive reverse dependencies may be displayed now,
since we do not apply the exceptions applied in graph-depends. For
example, host-skeleton gains two new recursive reverse dependencies:
skeleton and toolchain, which are both exceptions in graph-depends.
As for direct (not reverse) dependencies: the gain is not as fantastic
as for reverse ones, but it is still noticeable, especially thanks to
a21212fb7cf (package/pkg-generic: speed up RECURSIVE_FINAL_DEPENDENCIES);
just a few examples for %-show-recursive-depends:
before after speedup #deps
libzlib 0m46.864s 0m5.902s 7.94x 17
qt5base 0m57.590s 0m5.848s 9.85x 190
sqlite 0m46.601s 0m5.816s 8.01x 24
Basically, displaying recursive dependencies, direct or reverse, is
almost a constant now: it only slightly varies by about 10% depending
on the complexity of the dependency chain, with the parsing of the
Makefiles still accounting for the large majority of the time.
(PS. Thanks to Joseph for suggesting a list of interesting packages
to test, and thanks to Trent for his example of memoisation!)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Joseph Kogut <joseph.kogut@gmail.com>
Cc: Trent Piepho <tpiepho@impinj.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-03 03:16:34 -07:00
|
|
|
$(2)_FINAL_RECURSIVE_RDEPENDENCIES = $$(sort \
|
|
|
|
|
$$(if $$(filter undefined,$$(origin $(2)_FINAL_RECURSIVE_RDEPENDENCIES__X)), \
|
|
|
|
|
$$(eval $(2)_FINAL_RECURSIVE_RDEPENDENCIES__X := \
|
|
|
|
|
$$(foreach p, \
|
|
|
|
|
$$($(2)_RDEPENDENCIES), \
|
|
|
|
|
$$(p) \
|
|
|
|
|
$$($$(call UPPERCASE,$$(p))_FINAL_RECURSIVE_RDEPENDENCIES) \
|
|
|
|
|
) \
|
|
|
|
|
) \
|
|
|
|
|
) \
|
|
|
|
|
$$($(2)_FINAL_RECURSIVE_RDEPENDENCIES__X))
|
|
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
# define sub-target stamps
|
package/pkg-generic.mk: introduce final 'install' step
We currently have four different install steps: target installation,
staging installation, images installation and host installation. These
steps are directly triggered from the $(1)-install make target, so
there is no place where we can run some logic once all installation
steps have completed.
However, as part of improving the reliability of the logic done in
step_pkg_size_before and step_pkg_size_after to detect the files
installed by packages, we would in fact need to run some logic after
all installation steps have completed. This will allow us to make sure
that all files are detected, even if a host package installs something
in the target directory, or if a target package installs something in
the host directory.
To achieve this, this commit implements a new stamp file,
.stamp_installed, which is a step that depends on all four install
steps. Currently, this step does nothing except creating the stamp
file.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: remove stampfile on foo-reinstall]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-30 03:52:40 -06:00
|
|
|
$(2)_TARGET_INSTALL = $$($(2)_DIR)/.stamp_installed
|
2009-11-08 10:37:36 -07:00
|
|
|
$(2)_TARGET_INSTALL_TARGET = $$($(2)_DIR)/.stamp_target_installed
|
|
|
|
|
$(2)_TARGET_INSTALL_STAGING = $$($(2)_DIR)/.stamp_staging_installed
|
2011-07-05 13:53:56 -06:00
|
|
|
$(2)_TARGET_INSTALL_IMAGES = $$($(2)_DIR)/.stamp_images_installed
|
2019-11-24 16:09:18 -07:00
|
|
|
$(2)_TARGET_INSTALL_HOST = $$($(2)_DIR)/.stamp_host_installed
|
2009-11-08 10:37:36 -07:00
|
|
|
$(2)_TARGET_BUILD = $$($(2)_DIR)/.stamp_built
|
|
|
|
|
$(2)_TARGET_CONFIGURE = $$($(2)_DIR)/.stamp_configured
|
2019-11-24 16:09:18 -07:00
|
|
|
$(2)_TARGET_RSYNC = $$($(2)_DIR)/.stamp_rsynced
|
2009-11-08 10:37:36 -07:00
|
|
|
$(2)_TARGET_PATCH = $$($(2)_DIR)/.stamp_patched
|
|
|
|
|
$(2)_TARGET_EXTRACT = $$($(2)_DIR)/.stamp_extracted
|
2010-09-02 04:59:26 -06:00
|
|
|
$(2)_TARGET_SOURCE = $$($(2)_DIR)/.stamp_downloaded
|
core/legal-info: ensure legal-info works in off-line mode
Almost all packages which are saved for legal-info have their source
archives downloaded as part of 'make source', which makes an off-line
build completely possible [0].
However, for the pre-configured external toolchains, the source tarball
is different, as the main tarball is a binary package. And that source
tarball is only downloaded during the legal-info phase, which makes it
inconvenient for full off-line builds.
We fix that by adding a new rule, $(1)-legal-source which only
$(1)-all-source depends on, so that we only download it for a top-level
'make source', not as part of the standard download mechanism (i.e. only
what is really needed to build).
This new rule depends, like the normal download mechanism, on a stamp
file, so that we do not emit a spurious hash-check message on successive
runs of 'make source'.
This way, we can do a complete [0] off-line build and are still able to
generate legal-info, while at the same time we do not incur any download
overhead during a simple build.
Also, we previously downloaded the _ACTUAL_SOURCE_TARBALL when it was
not empty. However, since _ACTUAL_SOURCE_TARBALL defaults to the value
of _SOURCE, it can not be empty when _SOURCE is not. Thus, we'd get a
spurious report of a missing hash for the tarball, since it was not in
a standard package rule (configure, build, install..) and thus would
miss the PKG and PKGDIR variables to find the .hash file.
We fix that in this commit as well, by:
- setting PKG and PKGDIR just for the -legal-source rule;
- only downloading _ACTUAL_SOURCE_TARBALL if it is not empty *and* not
the same as _SOURCE (to avoid a second report about the hash).
[0] Save for nodejs which invarriably wants to download stuff at build
time. Sigh... :-( Fixing that is work for another time...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-07 10:14:38 -06:00
|
|
|
$(2)_TARGET_ACTUAL_SOURCE = $$($(2)_DIR)/.stamp_actual_downloaded
|
2009-11-08 10:37:36 -07:00
|
|
|
$(2)_TARGET_DIRCLEAN = $$($(2)_DIR)/.stamp_dircleaned
|
|
|
|
|
|
2011-07-05 13:53:52 -06:00
|
|
|
# default extract command
|
2011-07-05 13:53:53 -06:00
|
|
|
$(2)_EXTRACT_CMDS ?= \
|
2018-04-02 07:09:26 -06:00
|
|
|
$$(if $$($(2)_SOURCE),$$(INFLATE$$(suffix $$($(2)_SOURCE))) $$($(2)_DL_DIR)/$$($(2)_SOURCE) | \
|
2015-10-24 06:48:53 -06:00
|
|
|
$$(TAR) --strip-components=$$($(2)_STRIP_COMPONENTS) \
|
|
|
|
|
-C $$($(2)_DIR) \
|
|
|
|
|
$$(foreach x,$$($(2)_EXCLUDES),--exclude='$$(x)' ) \
|
|
|
|
|
$$(TAR_OPTIONS) -)
|
2011-07-05 13:53:52 -06:00
|
|
|
|
2014-05-05 07:04:20 -06:00
|
|
|
# pre/post-steps hooks
|
|
|
|
|
$(2)_PRE_DOWNLOAD_HOOKS ?=
|
2011-07-05 13:54:11 -06:00
|
|
|
$(2)_POST_DOWNLOAD_HOOKS ?=
|
2014-05-05 07:04:20 -06:00
|
|
|
$(2)_PRE_EXTRACT_HOOKS ?=
|
2009-11-08 10:37:36 -07:00
|
|
|
$(2)_POST_EXTRACT_HOOKS ?=
|
2014-05-05 07:04:20 -06:00
|
|
|
$(2)_PRE_RSYNC_HOOKS ?=
|
2013-11-07 03:41:21 -07:00
|
|
|
$(2)_POST_RSYNC_HOOKS ?=
|
2011-09-19 14:10:52 -06:00
|
|
|
$(2)_PRE_PATCH_HOOKS ?=
|
2009-11-08 10:37:36 -07:00
|
|
|
$(2)_POST_PATCH_HOOKS ?=
|
2010-11-03 20:50:24 -06:00
|
|
|
$(2)_PRE_CONFIGURE_HOOKS ?=
|
2009-11-08 10:37:36 -07:00
|
|
|
$(2)_POST_CONFIGURE_HOOKS ?=
|
2014-05-05 07:04:20 -06:00
|
|
|
$(2)_PRE_BUILD_HOOKS ?=
|
2009-11-08 10:37:36 -07:00
|
|
|
$(2)_POST_BUILD_HOOKS ?=
|
2014-05-05 07:04:20 -06:00
|
|
|
$(2)_PRE_INSTALL_HOOKS ?=
|
2009-11-08 10:37:36 -07:00
|
|
|
$(2)_POST_INSTALL_HOOKS ?=
|
2014-05-05 07:04:20 -06:00
|
|
|
$(2)_PRE_INSTALL_STAGING_HOOKS ?=
|
2009-11-08 10:37:36 -07:00
|
|
|
$(2)_POST_INSTALL_STAGING_HOOKS ?=
|
2014-05-05 07:04:20 -06:00
|
|
|
$(2)_PRE_INSTALL_TARGET_HOOKS ?=
|
2009-11-08 10:37:36 -07:00
|
|
|
$(2)_POST_INSTALL_TARGET_HOOKS ?=
|
2014-05-05 07:04:20 -06:00
|
|
|
$(2)_PRE_INSTALL_IMAGES_HOOKS ?=
|
2011-07-05 13:53:56 -06:00
|
|
|
$(2)_POST_INSTALL_IMAGES_HOOKS ?=
|
2014-05-05 07:04:20 -06:00
|
|
|
$(2)_PRE_LEGAL_INFO_HOOKS ?=
|
2012-11-03 00:52:16 -06:00
|
|
|
$(2)_POST_LEGAL_INFO_HOOKS ?=
|
2016-06-22 13:07:36 -06:00
|
|
|
$(2)_TARGET_FINALIZE_HOOKS ?=
|
fs: add pre- and post-command hooks
In some cases, the directory structure we want in the filesystem is not
exactly what we have in target/
For example, when systemd is used on a read-only rootfs, /var must be a
tmpfs. However, we may have packages that install stuff in there, and
set important rights (via the permission-table). So, at build time, we
need /var to be a symlink to the remanent location (/usr/share/factory)
while at runtime we need /var to be a directory.
One option would have been to have /var as a real directory even during
build time, and in a target-finalize hook, move everything out of there
and into the "factory" location. However, that's not possible because
it's too early: some packages may want to set ownership and/or acces
rights on directories or files in /var, and this is only done in the
fakeroot script, which is called only later during the assembling of the
filesystem images.
Also, there would have been no way to undo the tweak (i.e. we need to
restore the /var symlink so that subsequent builds continue to work) if
it were done as a target-finalize hook.
The only solution is to allow packages to register pre- and post-hooks
that are called right before and right after the rootfs commands are
executed, and inside in the fakeroot script.
We can however not re-use the BR2_ROOTFS_POST_FAKEROOT_SCRIPT feature
either because it is done before the filesystem command, but there is
nothing that is done after. Also, we don't want to add to, and modify a
user-supplied variable.
So, we introduce two new variables that packages can set to add the
commands they need to run to tweak the filesystem right at the last
moment.
Those hooks are not documented on-purpose; they are probably going to
only ever be used by systemd.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-08-01 16:52:22 -06:00
|
|
|
$(2)_ROOTFS_PRE_CMD_HOOKS ?=
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2018-04-29 06:15:22 -06:00
|
|
|
ifeq ($$($(2)_TYPE),target)
|
|
|
|
|
ifneq ($$(HOST_$(2)_KCONFIG_VAR),)
|
|
|
|
|
$$(error "Package $(1) defines host variant before target variant!")
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
# human-friendly targets and target sequencing
|
|
|
|
|
$(1): $(1)-install
|
package/pkg-generic.mk: introduce final 'install' step
We currently have four different install steps: target installation,
staging installation, images installation and host installation. These
steps are directly triggered from the $(1)-install make target, so
there is no place where we can run some logic once all installation
steps have completed.
However, as part of improving the reliability of the logic done in
step_pkg_size_before and step_pkg_size_after to detect the files
installed by packages, we would in fact need to run some logic after
all installation steps have completed. This will allow us to make sure
that all files are detected, even if a host package installs something
in the target directory, or if a target package installs something in
the host directory.
To achieve this, this commit implements a new stamp file,
.stamp_installed, which is a step that depends on all four install
steps. Currently, this step does nothing except creating the stamp
file.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: remove stampfile on foo-reinstall]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-30 03:52:40 -06:00
|
|
|
$(1)-install: $$($(2)_TARGET_INSTALL)
|
2009-11-08 10:37:36 -07:00
|
|
|
|
|
|
|
|
ifeq ($$($(2)_TYPE),host)
|
package/pkg-generic.mk: introduce final 'install' step
We currently have four different install steps: target installation,
staging installation, images installation and host installation. These
steps are directly triggered from the $(1)-install make target, so
there is no place where we can run some logic once all installation
steps have completed.
However, as part of improving the reliability of the logic done in
step_pkg_size_before and step_pkg_size_after to detect the files
installed by packages, we would in fact need to run some logic after
all installation steps have completed. This will allow us to make sure
that all files are detected, even if a host package installs something
in the target directory, or if a target package installs something in
the host directory.
To achieve this, this commit implements a new stamp file,
.stamp_installed, which is a step that depends on all four install
steps. Currently, this step does nothing except creating the stamp
file.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: remove stampfile on foo-reinstall]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-30 03:52:40 -06:00
|
|
|
$$($(2)_TARGET_INSTALL): $$($(2)_TARGET_INSTALL_HOST)
|
2009-11-08 10:37:36 -07:00
|
|
|
else
|
package/pkg-generic.mk: don't set INSTALL_{TARGET, STAGING, IMAGES} for host packages
By their very nature, host packages have no target, staging, or images
install steps; they have a single install step, that is always
performed.
As such, setting the corresponding _INSTALL_{TARGET,STAGING,IMAGES}
variables does not make sense for host packages.
However, people (and scripts) may get confused when they process the
output of printvars, e.g.:
$ make printvars VARS=HOST_LIBTOOL_INSTALL_TARGET
HOST_LIBTOOL_INSTALL_TARGET=YES
Only set those variables for target packages. There is no
corresponding variable for host packages, as they are always installed
(and only once).
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vadim Kochan <vadim4j@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: eeppeliteloop@gmail.com
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-11 02:12:27 -06:00
|
|
|
$(2)_INSTALL_STAGING ?= NO
|
|
|
|
|
$(2)_INSTALL_IMAGES ?= NO
|
|
|
|
|
$(2)_INSTALL_TARGET ?= YES
|
package/pkg-generic.mk: introduce final 'install' step
We currently have four different install steps: target installation,
staging installation, images installation and host installation. These
steps are directly triggered from the $(1)-install make target, so
there is no place where we can run some logic once all installation
steps have completed.
However, as part of improving the reliability of the logic done in
step_pkg_size_before and step_pkg_size_after to detect the files
installed by packages, we would in fact need to run some logic after
all installation steps have completed. This will allow us to make sure
that all files are detected, even if a host package installs something
in the target directory, or if a target package installs something in
the host directory.
To achieve this, this commit implements a new stamp file,
.stamp_installed, which is a step that depends on all four install
steps. Currently, this step does nothing except creating the stamp
file.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: remove stampfile on foo-reinstall]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-30 03:52:40 -06:00
|
|
|
ifeq ($$($(2)_INSTALL_TARGET),YES)
|
|
|
|
|
$$($(2)_TARGET_INSTALL): $$($(2)_TARGET_INSTALL_TARGET)
|
|
|
|
|
endif
|
|
|
|
|
ifeq ($$($(2)_INSTALL_STAGING),YES)
|
|
|
|
|
$$($(2)_TARGET_INSTALL): $$($(2)_TARGET_INSTALL_STAGING)
|
|
|
|
|
endif
|
|
|
|
|
ifeq ($$($(2)_INSTALL_IMAGES),YES)
|
|
|
|
|
$$($(2)_TARGET_INSTALL): $$($(2)_TARGET_INSTALL_IMAGES)
|
|
|
|
|
endif
|
2009-11-08 10:37:36 -07:00
|
|
|
endif
|
|
|
|
|
|
|
|
|
|
ifeq ($$($(2)_INSTALL_TARGET),YES)
|
2014-02-14 02:55:05 -07:00
|
|
|
$(1)-install-target: $$($(2)_TARGET_INSTALL_TARGET)
|
|
|
|
|
$$($(2)_TARGET_INSTALL_TARGET): $$($(2)_TARGET_BUILD)
|
2009-11-08 10:37:36 -07:00
|
|
|
else
|
|
|
|
|
$(1)-install-target:
|
|
|
|
|
endif
|
|
|
|
|
|
|
|
|
|
ifeq ($$($(2)_INSTALL_STAGING),YES)
|
2014-02-14 02:55:05 -07:00
|
|
|
$(1)-install-staging: $$($(2)_TARGET_INSTALL_STAGING)
|
|
|
|
|
$$($(2)_TARGET_INSTALL_STAGING): $$($(2)_TARGET_BUILD)
|
|
|
|
|
# Some packages use install-staging stuff for install-target
|
|
|
|
|
$$($(2)_TARGET_INSTALL_TARGET): $$($(2)_TARGET_INSTALL_STAGING)
|
2009-11-08 10:37:36 -07:00
|
|
|
else
|
|
|
|
|
$(1)-install-staging:
|
|
|
|
|
endif
|
|
|
|
|
|
2011-07-05 13:53:56 -06:00
|
|
|
ifeq ($$($(2)_INSTALL_IMAGES),YES)
|
2014-02-14 02:55:05 -07:00
|
|
|
$(1)-install-images: $$($(2)_TARGET_INSTALL_IMAGES)
|
|
|
|
|
$$($(2)_TARGET_INSTALL_IMAGES): $$($(2)_TARGET_BUILD)
|
2011-07-05 13:53:56 -06:00
|
|
|
else
|
|
|
|
|
$(1)-install-images:
|
|
|
|
|
endif
|
|
|
|
|
|
2017-03-26 09:05:43 -06:00
|
|
|
$(1)-install-host: $$($(2)_TARGET_INSTALL_HOST)
|
2014-02-14 02:55:05 -07:00
|
|
|
$$($(2)_TARGET_INSTALL_HOST): $$($(2)_TARGET_BUILD)
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2014-02-14 02:55:05 -07:00
|
|
|
$(1)-build: $$($(2)_TARGET_BUILD)
|
|
|
|
|
$$($(2)_TARGET_BUILD): $$($(2)_TARGET_CONFIGURE)
|
|
|
|
|
|
2014-06-01 04:28:54 -06:00
|
|
|
# Since $(2)_FINAL_DEPENDENCIES are phony targets, they are always "newer"
|
2014-02-14 02:55:05 -07:00
|
|
|
# than $(2)_TARGET_CONFIGURE. This would force the configure step (and
|
|
|
|
|
# therefore the other steps as well) to be re-executed with every
|
2014-06-01 04:28:54 -06:00
|
|
|
# invocation of make. Therefore, make $(2)_FINAL_DEPENDENCIES an order-only
|
2014-02-14 02:55:05 -07:00
|
|
|
# dependency by using |.
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2014-02-14 02:55:05 -07:00
|
|
|
$(1)-configure: $$($(2)_TARGET_CONFIGURE)
|
2014-06-01 04:28:54 -06:00
|
|
|
$$($(2)_TARGET_CONFIGURE): | $$($(2)_FINAL_DEPENDENCIES)
|
2009-11-08 10:37:36 -07:00
|
|
|
|
Makefile: rework main directory creation logic
In the current code, the creation of the main output directories
(BUILD_DIR, STAGING_DIR, HOST_DIR, TARGET_DIR, etc.) is done by a
global "dirs" target. While this works fine in the current situation,
it doesn't work well in a context where per-package host and target
directories are used.
For example, with the current code and per-package host directories,
the output/staging symbolic link ends up being created as a link to
the per-package package sysroot directory of the first package being
built, instead of the global sysroot.
This commit reworks the creation of those directories by having the
package/pkg-generic.mk code ensure that the build directory, target
directory, host directory, staging directory and binaries directory
exist before they are needed.
Two new targets, host-finalize and staging-finalize are added in the
main Makefile to create the compatibility symlinks for host and
staging directories. They will be extended later with additional logic
for per-package directories.
Thanks to those changes, the global "dirs" target is entirely removed.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-23 07:58:08 -07:00
|
|
|
$$($(2)_TARGET_SOURCE) $$($(2)_TARGET_RSYNC): | prepare
|
2014-02-14 02:55:03 -07:00
|
|
|
$$($(2)_TARGET_SOURCE) $$($(2)_TARGET_RSYNC): | dependencies
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2011-09-29 13:57:37 -06:00
|
|
|
ifeq ($$($(2)_OVERRIDE_SRCDIR),)
|
|
|
|
|
# In the normal case (no package override), the sequence of steps is
|
|
|
|
|
# source, by downloading
|
|
|
|
|
# depends
|
|
|
|
|
# extract
|
|
|
|
|
# patch
|
|
|
|
|
# configure
|
2014-02-14 02:55:05 -07:00
|
|
|
$$($(2)_TARGET_CONFIGURE): $$($(2)_TARGET_PATCH)
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2014-02-14 02:55:05 -07:00
|
|
|
$(1)-patch: $$($(2)_TARGET_PATCH)
|
|
|
|
|
$$($(2)_TARGET_PATCH): $$($(2)_TARGET_EXTRACT)
|
2015-03-14 08:25:17 -06:00
|
|
|
# Order-only dependency
|
|
|
|
|
$$($(2)_TARGET_PATCH): | $$(patsubst %,%-patch,$$($(2)_FINAL_PATCH_DEPENDENCIES))
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2014-02-14 02:55:05 -07:00
|
|
|
$(1)-extract: $$($(2)_TARGET_EXTRACT)
|
|
|
|
|
$$($(2)_TARGET_EXTRACT): $$($(2)_TARGET_SOURCE)
|
package/pkg-generic: add the concept of extract dependency
Extract dependencies are dependencies that must be ready before the
extract step of a package, i.e for tools that are needed to extract
packages themselves. Current examples of such tools are host-tar,
host-lzip and host-xz.
They are currently handled through DEPENDENCIES_HOST_PREREQ. However,
this mechanism has a number of drawbacks:
- First and foremost, because host-tar/host-lzip/host-xz are not
listed in the dependencies of packages, the package infrastructure
does not know it should rsync them in the context of per-package
SDK.
- Second, there is no dependency handling *between* them. I.e, we
have no mechanism that says host-tar should be built before
host-lzip, while it is in fact the case: if you need to build
host-lzip, you need to extract a tarball, so you may need host-tar
if your system tarball is not capable enough.
For those reasons, it makes sense to add explicit support for "extract
dependencies" in the package infrastructure, through the
<pkg>_EXTRACT_DEPENDENCIES variable. It is unlikely this variable will
ever be used by a package .mk file, but it will be used internally by
the package infrastructure.
[Peter: fix typo in manual]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-03-24 08:20:02 -06:00
|
|
|
$$($(2)_TARGET_EXTRACT): | $$($(2)_FINAL_EXTRACT_DEPENDENCIES)
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2019-07-02 14:12:50 -06:00
|
|
|
$(1)-depends: $$($(2)_FINAL_ALL_DEPENDENCIES)
|
2010-05-06 02:05:43 -06:00
|
|
|
|
2011-09-29 13:57:37 -06:00
|
|
|
$(1)-source: $$($(2)_TARGET_SOURCE)
|
2018-05-08 14:40:18 -06:00
|
|
|
$$($(2)_TARGET_SOURCE): | $$($(2)_FINAL_DOWNLOAD_DEPENDENCIES)
|
2015-03-29 11:33:24 -06:00
|
|
|
|
core/legal-info: ensure legal-info works in off-line mode
Almost all packages which are saved for legal-info have their source
archives downloaded as part of 'make source', which makes an off-line
build completely possible [0].
However, for the pre-configured external toolchains, the source tarball
is different, as the main tarball is a binary package. And that source
tarball is only downloaded during the legal-info phase, which makes it
inconvenient for full off-line builds.
We fix that by adding a new rule, $(1)-legal-source which only
$(1)-all-source depends on, so that we only download it for a top-level
'make source', not as part of the standard download mechanism (i.e. only
what is really needed to build).
This new rule depends, like the normal download mechanism, on a stamp
file, so that we do not emit a spurious hash-check message on successive
runs of 'make source'.
This way, we can do a complete [0] off-line build and are still able to
generate legal-info, while at the same time we do not incur any download
overhead during a simple build.
Also, we previously downloaded the _ACTUAL_SOURCE_TARBALL when it was
not empty. However, since _ACTUAL_SOURCE_TARBALL defaults to the value
of _SOURCE, it can not be empty when _SOURCE is not. Thus, we'd get a
spurious report of a missing hash for the tarball, since it was not in
a standard package rule (configure, build, install..) and thus would
miss the PKG and PKGDIR variables to find the .hash file.
We fix that in this commit as well, by:
- setting PKG and PKGDIR just for the -legal-source rule;
- only downloading _ACTUAL_SOURCE_TARBALL if it is not empty *and* not
the same as _SOURCE (to avoid a second report about the hash).
[0] Save for nodejs which invarriably wants to download stuff at build
time. Sigh... :-( Fixing that is work for another time...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-07 10:14:38 -06:00
|
|
|
$(1)-all-source: $(1)-legal-source
|
|
|
|
|
$(1)-legal-info: $(1)-legal-source
|
|
|
|
|
$(1)-legal-source: $(1)-source
|
|
|
|
|
|
|
|
|
|
# Only download the actual source if it differs from the 'main' archive
|
|
|
|
|
ifneq ($$($(2)_ACTUAL_SOURCE_TARBALL),)
|
|
|
|
|
ifneq ($$($(2)_ACTUAL_SOURCE_TARBALL),$$($(2)_SOURCE))
|
|
|
|
|
$(1)-legal-source: $$($(2)_TARGET_ACTUAL_SOURCE)
|
|
|
|
|
endif # actual sources != sources
|
|
|
|
|
endif # actual sources != ""
|
|
|
|
|
|
2015-03-29 11:33:24 -06:00
|
|
|
$(1)-external-deps:
|
|
|
|
|
@for p in $$($(2)_SOURCE) $$($(2)_PATCH) $$($(2)_EXTRA_DOWNLOADS) ; do \
|
|
|
|
|
echo `basename $$$$p` ; \
|
|
|
|
|
done
|
2011-09-29 13:57:37 -06:00
|
|
|
else
|
|
|
|
|
# In the package override case, the sequence of steps
|
|
|
|
|
# source, by rsyncing
|
|
|
|
|
# depends
|
|
|
|
|
# configure
|
2014-06-30 03:51:28 -06:00
|
|
|
|
|
|
|
|
# Use an order-only dependency so the "<pkg>-clean-for-rebuild" rule
|
|
|
|
|
# can remove the stamp file without triggering the configure step.
|
|
|
|
|
$$($(2)_TARGET_CONFIGURE): | $$($(2)_TARGET_RSYNC)
|
2011-09-29 13:57:37 -06:00
|
|
|
|
2014-06-01 04:28:54 -06:00
|
|
|
$(1)-depends: $$($(2)_FINAL_DEPENDENCIES)
|
2011-09-29 13:57:37 -06:00
|
|
|
|
2012-12-06 06:16:00 -07:00
|
|
|
$(1)-patch: $(1)-rsync
|
|
|
|
|
$(1)-extract: $(1)-rsync
|
|
|
|
|
|
2011-09-29 13:57:37 -06:00
|
|
|
$(1)-rsync: $$($(2)_TARGET_RSYNC)
|
|
|
|
|
|
2015-04-26 03:51:07 -06:00
|
|
|
$(1)-source:
|
core/legal-info: ensure legal-info works in off-line mode
Almost all packages which are saved for legal-info have their source
archives downloaded as part of 'make source', which makes an off-line
build completely possible [0].
However, for the pre-configured external toolchains, the source tarball
is different, as the main tarball is a binary package. And that source
tarball is only downloaded during the legal-info phase, which makes it
inconvenient for full off-line builds.
We fix that by adding a new rule, $(1)-legal-source which only
$(1)-all-source depends on, so that we only download it for a top-level
'make source', not as part of the standard download mechanism (i.e. only
what is really needed to build).
This new rule depends, like the normal download mechanism, on a stamp
file, so that we do not emit a spurious hash-check message on successive
runs of 'make source'.
This way, we can do a complete [0] off-line build and are still able to
generate legal-info, while at the same time we do not incur any download
overhead during a simple build.
Also, we previously downloaded the _ACTUAL_SOURCE_TARBALL when it was
not empty. However, since _ACTUAL_SOURCE_TARBALL defaults to the value
of _SOURCE, it can not be empty when _SOURCE is not. Thus, we'd get a
spurious report of a missing hash for the tarball, since it was not in
a standard package rule (configure, build, install..) and thus would
miss the PKG and PKGDIR variables to find the .hash file.
We fix that in this commit as well, by:
- setting PKG and PKGDIR just for the -legal-source rule;
- only downloading _ACTUAL_SOURCE_TARBALL if it is not empty *and* not
the same as _SOURCE (to avoid a second report about the hash).
[0] Save for nodejs which invarriably wants to download stuff at build
time. Sigh... :-( Fixing that is work for another time...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-07 10:14:38 -06:00
|
|
|
$(1)-legal-source:
|
2015-03-29 11:33:24 -06:00
|
|
|
|
|
|
|
|
$(1)-external-deps:
|
|
|
|
|
@echo "file://$$($(2)_OVERRIDE_SRCDIR)"
|
2011-09-29 13:57:37 -06:00
|
|
|
endif
|
|
|
|
|
|
2015-01-03 07:29:12 -07:00
|
|
|
$(1)-show-version:
|
|
|
|
|
@echo $$($(2)_VERSION)
|
|
|
|
|
|
2010-05-06 02:05:43 -06:00
|
|
|
$(1)-show-depends:
|
2015-04-26 03:51:00 -06:00
|
|
|
@echo $$($(2)_FINAL_ALL_DEPENDENCIES)
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2018-03-31 10:35:43 -06:00
|
|
|
$(1)-show-recursive-depends:
|
infra/pkg-generic: use pure Makefile-based recursive dependencies
Calling to the graph-depends script is very costly, as it calls back to
'make' a lot of time.
It turns out that we already have the list of recursive dependencies, so
we can just print that.
As for the recursive reverse dependencies, we use the same memoisation
technique to cut-down on the expansion cost, which would otherwise be on
the order of 𝑶(𝑛²) (with 𝑛 enabled packages).
>From a defconfig, modified to use glibc, C++, wchar, locales, ssp, and
allyespackageconfig (tweaked to avoid multi providers, etc...), the
timings for X-show-recursive-rdepends are:
before after speedup #rdeps
libnss 0m22.932s 0m5.775s 3.97x 3
qt5base 0m41.176s 0m5.781s 7.12x 67
libjpeg 0m56.185s 0m5.749s 9.71x 228
libxml2 0m54.964s 0m5.795s 9.48x 271
freetype 0m46.754s 0m5.819s 8.07x 287
libpng 0m53.577s 0m5.760s 9.30x 303
sqlite 1m15.222s 0m5.807s 12.95x 801
libopenssl 1m25.471s 0m5.844s 14.63x 931
readline 1m13.805s 0m5.775s 12.78x 958
libzlib 1m11.807s 0m5.820s 12.34x 1039
toolchain 1m23.712s 0m6.080s 13.77x 2107
skeleton 1m27.839s 0m6.293s 13.96x 2111 (+1)
host-skeleton 1m27.405s 0m6.350s 13.76x 2172 (+2)
- speedup: ratio before/after
- #rdeps: number of recursive reverse dependencies, with the extra
dependencies returned with this patch, see below for the
reason.
So, for a low-level package with a lot of reverse dependencies, like
libzlibz, libopenssl or readline are, the timings are already very much
in favour of the change. This is less impressive with packages that
have few dependencies (libnss), but still much faster.
Also, remember that the config tested has as much packages enabled as
possible, so is in itself a degenerate case. With simpler and more
realistic configurations, the gains would probably be a bit lower than
reported above, but various tests still report good improvements
overall (note: coming up with a 'realistic' configuration is pretty
hard, as everyone and their dog have their notion of what is realistic
in their context, so nothing displayed here; timings are left as an
exercise for the interested parties to report aggravation in their
cases should they notice some regression).
Note that, more recursive reverse dependencies may be displayed now,
since we do not apply the exceptions applied in graph-depends. For
example, host-skeleton gains two new recursive reverse dependencies:
skeleton and toolchain, which are both exceptions in graph-depends.
As for direct (not reverse) dependencies: the gain is not as fantastic
as for reverse ones, but it is still noticeable, especially thanks to
a21212fb7cf (package/pkg-generic: speed up RECURSIVE_FINAL_DEPENDENCIES);
just a few examples for %-show-recursive-depends:
before after speedup #deps
libzlib 0m46.864s 0m5.902s 7.94x 17
qt5base 0m57.590s 0m5.848s 9.85x 190
sqlite 0m46.601s 0m5.816s 8.01x 24
Basically, displaying recursive dependencies, direct or reverse, is
almost a constant now: it only slightly varies by about 10% depending
on the complexity of the dependency chain, with the parsing of the
Makefiles still accounting for the large majority of the time.
(PS. Thanks to Joseph for suggesting a list of interesting packages
to test, and thanks to Trent for his example of memoisation!)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Joseph Kogut <joseph.kogut@gmail.com>
Cc: Trent Piepho <tpiepho@impinj.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-03 03:16:34 -07:00
|
|
|
@echo $$($(2)_FINAL_RECURSIVE_DEPENDENCIES)
|
2018-03-31 10:35:43 -06:00
|
|
|
|
2016-10-23 09:28:51 -06:00
|
|
|
$(1)-show-rdepends:
|
|
|
|
|
@echo $$($(2)_RDEPENDENCIES)
|
|
|
|
|
|
2018-03-31 10:35:43 -06:00
|
|
|
$(1)-show-recursive-rdepends:
|
infra/pkg-generic: use pure Makefile-based recursive dependencies
Calling to the graph-depends script is very costly, as it calls back to
'make' a lot of time.
It turns out that we already have the list of recursive dependencies, so
we can just print that.
As for the recursive reverse dependencies, we use the same memoisation
technique to cut-down on the expansion cost, which would otherwise be on
the order of 𝑶(𝑛²) (with 𝑛 enabled packages).
>From a defconfig, modified to use glibc, C++, wchar, locales, ssp, and
allyespackageconfig (tweaked to avoid multi providers, etc...), the
timings for X-show-recursive-rdepends are:
before after speedup #rdeps
libnss 0m22.932s 0m5.775s 3.97x 3
qt5base 0m41.176s 0m5.781s 7.12x 67
libjpeg 0m56.185s 0m5.749s 9.71x 228
libxml2 0m54.964s 0m5.795s 9.48x 271
freetype 0m46.754s 0m5.819s 8.07x 287
libpng 0m53.577s 0m5.760s 9.30x 303
sqlite 1m15.222s 0m5.807s 12.95x 801
libopenssl 1m25.471s 0m5.844s 14.63x 931
readline 1m13.805s 0m5.775s 12.78x 958
libzlib 1m11.807s 0m5.820s 12.34x 1039
toolchain 1m23.712s 0m6.080s 13.77x 2107
skeleton 1m27.839s 0m6.293s 13.96x 2111 (+1)
host-skeleton 1m27.405s 0m6.350s 13.76x 2172 (+2)
- speedup: ratio before/after
- #rdeps: number of recursive reverse dependencies, with the extra
dependencies returned with this patch, see below for the
reason.
So, for a low-level package with a lot of reverse dependencies, like
libzlibz, libopenssl or readline are, the timings are already very much
in favour of the change. This is less impressive with packages that
have few dependencies (libnss), but still much faster.
Also, remember that the config tested has as much packages enabled as
possible, so is in itself a degenerate case. With simpler and more
realistic configurations, the gains would probably be a bit lower than
reported above, but various tests still report good improvements
overall (note: coming up with a 'realistic' configuration is pretty
hard, as everyone and their dog have their notion of what is realistic
in their context, so nothing displayed here; timings are left as an
exercise for the interested parties to report aggravation in their
cases should they notice some regression).
Note that, more recursive reverse dependencies may be displayed now,
since we do not apply the exceptions applied in graph-depends. For
example, host-skeleton gains two new recursive reverse dependencies:
skeleton and toolchain, which are both exceptions in graph-depends.
As for direct (not reverse) dependencies: the gain is not as fantastic
as for reverse ones, but it is still noticeable, especially thanks to
a21212fb7cf (package/pkg-generic: speed up RECURSIVE_FINAL_DEPENDENCIES);
just a few examples for %-show-recursive-depends:
before after speedup #deps
libzlib 0m46.864s 0m5.902s 7.94x 17
qt5base 0m57.590s 0m5.848s 9.85x 190
sqlite 0m46.601s 0m5.816s 8.01x 24
Basically, displaying recursive dependencies, direct or reverse, is
almost a constant now: it only slightly varies by about 10% depending
on the complexity of the dependency chain, with the parsing of the
Makefiles still accounting for the large majority of the time.
(PS. Thanks to Joseph for suggesting a list of interesting packages
to test, and thanks to Trent for his example of memoisation!)
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Joseph Kogut <joseph.kogut@gmail.com>
Cc: Trent Piepho <tpiepho@impinj.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2019-03-03 03:16:34 -07:00
|
|
|
@echo $$($(2)_FINAL_RECURSIVE_RDEPENDENCIES)
|
2018-03-31 10:35:43 -06:00
|
|
|
|
2017-04-02 07:03:38 -06:00
|
|
|
$(1)-show-build-order: $$(patsubst %,%-show-build-order,$$($(2)_FINAL_ALL_DEPENDENCIES))
|
2018-11-15 08:45:42 -07:00
|
|
|
@:
|
2017-04-02 07:03:38 -06:00
|
|
|
$$(info $(1))
|
|
|
|
|
|
core: add per-package and per-filesystem show-info
Sometimes, it is need to quickly get the metadata of a subset of
packages, without resorting to a full-blown JSON query.
Introduce a new per-package (and per-filesystem) foo-show-info rule,
that otputs a per-entity valid JSON blob.
Note that calling it for multiple packages and.or filesystems at once
will not generate a valid JSON blob, as there would be no separator
between the JSON elements:
$ make {foo,bar}-show-info
{ "foo": { foo stuff } }
{ "bar": { bar stuff } }
However, jq is able to absorb this, with its slurping ability, which
generates an array (ellipsed and manualy reformated for readability):
$ make {foo,bar}-show-info |jq -s . -
[
{ "foo": { foo stuff } },
{ "bar": { bar stuff } }
]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-04-15 13:47:32 -06:00
|
|
|
$(1)-show-info:
|
|
|
|
|
@:
|
|
|
|
|
$$(info $$(call clean-json,{ $$(call json-info,$(2)) }))
|
|
|
|
|
|
2014-06-17 03:33:54 -06:00
|
|
|
$(1)-graph-depends: graph-depends-requirements
|
2016-10-23 11:19:44 -06:00
|
|
|
$(call pkg-graph-depends,$(1),--direct)
|
|
|
|
|
|
|
|
|
|
$(1)-graph-rdepends: graph-depends-requirements
|
|
|
|
|
$(call pkg-graph-depends,$(1),--reverse)
|
2013-12-28 10:39:12 -07:00
|
|
|
|
2015-04-26 03:51:00 -06:00
|
|
|
$(1)-all-source: $(1)-source
|
|
|
|
|
$(1)-all-source: $$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-source)
|
2015-03-29 11:33:25 -06:00
|
|
|
|
2015-04-26 03:51:00 -06:00
|
|
|
$(1)-all-external-deps: $(1)-external-deps
|
|
|
|
|
$(1)-all-external-deps: $$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-external-deps)
|
2015-03-29 11:33:25 -06:00
|
|
|
|
2015-04-26 03:51:00 -06:00
|
|
|
$(1)-all-legal-info: $(1)-legal-info
|
|
|
|
|
$(1)-all-legal-info: $$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),$$(p)-all-legal-info)
|
2015-03-29 11:33:25 -06:00
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
$(1)-dirclean: $$($(2)_TARGET_DIRCLEAN)
|
|
|
|
|
|
2014-11-28 08:25:04 -07:00
|
|
|
$(1)-clean-for-reinstall:
|
2011-09-29 13:57:39 -06:00
|
|
|
ifneq ($$($(2)_OVERRIDE_SRCDIR),)
|
|
|
|
|
rm -f $$($(2)_TARGET_RSYNC)
|
|
|
|
|
endif
|
package/pkg-generic.mk: introduce final 'install' step
We currently have four different install steps: target installation,
staging installation, images installation and host installation. These
steps are directly triggered from the $(1)-install make target, so
there is no place where we can run some logic once all installation
steps have completed.
However, as part of improving the reliability of the logic done in
step_pkg_size_before and step_pkg_size_after to detect the files
installed by packages, we would in fact need to run some logic after
all installation steps have completed. This will allow us to make sure
that all files are detected, even if a host package installs something
in the target directory, or if a target package installs something in
the host directory.
To achieve this, this commit implements a new stamp file,
.stamp_installed, which is a step that depends on all four install
steps. Currently, this step does nothing except creating the stamp
file.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: remove stampfile on foo-reinstall]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-30 03:52:40 -06:00
|
|
|
rm -f $$($(2)_TARGET_INSTALL)
|
2011-09-29 13:57:39 -06:00
|
|
|
rm -f $$($(2)_TARGET_INSTALL_STAGING)
|
|
|
|
|
rm -f $$($(2)_TARGET_INSTALL_TARGET)
|
2011-11-15 10:31:17 -07:00
|
|
|
rm -f $$($(2)_TARGET_INSTALL_IMAGES)
|
2011-09-29 13:57:39 -06:00
|
|
|
rm -f $$($(2)_TARGET_INSTALL_HOST)
|
|
|
|
|
|
2014-11-28 08:25:04 -07:00
|
|
|
$(1)-reinstall: $(1)-clean-for-reinstall $(1)
|
|
|
|
|
|
|
|
|
|
$(1)-clean-for-rebuild: $(1)-clean-for-reinstall
|
|
|
|
|
rm -f $$($(2)_TARGET_BUILD)
|
|
|
|
|
|
2013-05-13 09:25:41 -06:00
|
|
|
$(1)-rebuild: $(1)-clean-for-rebuild $(1)
|
2011-09-29 13:57:39 -06:00
|
|
|
|
|
|
|
|
$(1)-clean-for-reconfigure: $(1)-clean-for-rebuild
|
|
|
|
|
rm -f $$($(2)_TARGET_CONFIGURE)
|
|
|
|
|
|
2013-05-13 09:25:41 -06:00
|
|
|
$(1)-reconfigure: $(1)-clean-for-reconfigure $(1)
|
2011-09-29 13:57:39 -06:00
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
# define the PKG variable for all targets, containing the
|
|
|
|
|
# uppercase package variable prefix
|
package/pkg-generic.mk: introduce final 'install' step
We currently have four different install steps: target installation,
staging installation, images installation and host installation. These
steps are directly triggered from the $(1)-install make target, so
there is no place where we can run some logic once all installation
steps have completed.
However, as part of improving the reliability of the logic done in
step_pkg_size_before and step_pkg_size_after to detect the files
installed by packages, we would in fact need to run some logic after
all installation steps have completed. This will allow us to make sure
that all files are detected, even if a host package installs something
in the target directory, or if a target package installs something in
the host directory.
To achieve this, this commit implements a new stamp file,
.stamp_installed, which is a step that depends on all four install
steps. Currently, this step does nothing except creating the stamp
file.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: remove stampfile on foo-reinstall]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-04-30 03:52:40 -06:00
|
|
|
$$($(2)_TARGET_INSTALL): PKG=$(2)
|
2009-11-08 10:37:36 -07:00
|
|
|
$$($(2)_TARGET_INSTALL_TARGET): PKG=$(2)
|
|
|
|
|
$$($(2)_TARGET_INSTALL_STAGING): PKG=$(2)
|
2011-07-05 13:53:56 -06:00
|
|
|
$$($(2)_TARGET_INSTALL_IMAGES): PKG=$(2)
|
2015-09-27 05:34:50 -06:00
|
|
|
$$($(2)_TARGET_INSTALL_HOST): PKG=$(2)
|
2009-11-08 10:37:36 -07:00
|
|
|
$$($(2)_TARGET_BUILD): PKG=$(2)
|
|
|
|
|
$$($(2)_TARGET_CONFIGURE): PKG=$(2)
|
2019-11-05 09:46:42 -07:00
|
|
|
$$($(2)_TARGET_CONFIGURE): NAME=$(1)
|
2015-09-27 05:34:50 -06:00
|
|
|
$$($(2)_TARGET_RSYNC): SRCDIR=$$($(2)_OVERRIDE_SRCDIR)
|
|
|
|
|
$$($(2)_TARGET_RSYNC): PKG=$(2)
|
2009-11-08 10:37:36 -07:00
|
|
|
$$($(2)_TARGET_PATCH): PKG=$(2)
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 13:12:24 -06:00
|
|
|
$$($(2)_TARGET_PATCH): RAWNAME=$$(patsubst host-%,%,$(1))
|
2014-02-05 02:44:01 -07:00
|
|
|
$$($(2)_TARGET_PATCH): PKGDIR=$(pkgdir)
|
2009-11-08 10:37:36 -07:00
|
|
|
$$($(2)_TARGET_EXTRACT): PKG=$(2)
|
|
|
|
|
$$($(2)_TARGET_SOURCE): PKG=$(2)
|
2014-06-30 16:26:20 -06:00
|
|
|
$$($(2)_TARGET_SOURCE): PKGDIR=$(pkgdir)
|
core/legal-info: ensure legal-info works in off-line mode
Almost all packages which are saved for legal-info have their source
archives downloaded as part of 'make source', which makes an off-line
build completely possible [0].
However, for the pre-configured external toolchains, the source tarball
is different, as the main tarball is a binary package. And that source
tarball is only downloaded during the legal-info phase, which makes it
inconvenient for full off-line builds.
We fix that by adding a new rule, $(1)-legal-source which only
$(1)-all-source depends on, so that we only download it for a top-level
'make source', not as part of the standard download mechanism (i.e. only
what is really needed to build).
This new rule depends, like the normal download mechanism, on a stamp
file, so that we do not emit a spurious hash-check message on successive
runs of 'make source'.
This way, we can do a complete [0] off-line build and are still able to
generate legal-info, while at the same time we do not incur any download
overhead during a simple build.
Also, we previously downloaded the _ACTUAL_SOURCE_TARBALL when it was
not empty. However, since _ACTUAL_SOURCE_TARBALL defaults to the value
of _SOURCE, it can not be empty when _SOURCE is not. Thus, we'd get a
spurious report of a missing hash for the tarball, since it was not in
a standard package rule (configure, build, install..) and thus would
miss the PKG and PKGDIR variables to find the .hash file.
We fix that in this commit as well, by:
- setting PKG and PKGDIR just for the -legal-source rule;
- only downloading _ACTUAL_SOURCE_TARBALL if it is not empty *and* not
the same as _SOURCE (to avoid a second report about the hash).
[0] Save for nodejs which invarriably wants to download stuff at build
time. Sigh... :-( Fixing that is work for another time...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-07 10:14:38 -06:00
|
|
|
$$($(2)_TARGET_ACTUAL_SOURCE): PKG=$(2)
|
|
|
|
|
$$($(2)_TARGET_ACTUAL_SOURCE): PKGDIR=$(pkgdir)
|
2009-11-08 10:37:36 -07:00
|
|
|
$$($(2)_TARGET_DIRCLEAN): PKG=$(2)
|
core: implement per-package SDK and target
This commit implements the core of the move to per-package SDK and
target directories. The main idea is that instead of having a global
output/host and output/target in which all packages install files, we
switch to per-package host and target directories, that only contain
their explicit dependencies.
There are two main benefits:
- Packages will now see only the dependencies they explicitly list in
their <pkg>_DEPENDENCIES variable, and the recursive dependencies
thereof.
- We can support top-level parallel build properly, because a package
only "sees" its own host directory and target directory, isolated
from the build of other packages that can happen in parallel.
It works as follows:
- A new output/per-package/ directory is created, which will contain
one sub-directory per package, and inside it, a "host" directory
and a "target" directory:
output/per-package/busybox/target
output/per-package/busybox/host
output/per-package/host-fakeroot/target
output/per-package/host-fakeroot/host
This output/per-package/ directory is PER_PACKAGE_DIR.
- The global TARGET_DIR and HOST_DIR variable now automatically point
to the per-package directory when PKG is defined. So whenever a
package references $(HOST_DIR) or $(TARGET_DIR) in its build
process, it effectively references the per-package host/target
directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it
is handled as well.
- Of course, packages have dependencies, so those dependencies must
be installed in the per-package host and target directories. To do
so, we simply rsync (using hard links to save space and time) the
host and target directories of the direct dependencies of the
package to the current package host and target directories.
We only need to take care of direct dependencies (and not
recursively all dependencies), because we accumulate into those
per-package host and target directories the files installed by the
dependencies. Note that this only works because we make the
assumption that one package does *not* overwrite files installed by
another package.
This is done for "extract dependencies" at the beginning of the
extract step, and for "normal dependencies" at the beginning of the
configure step.
This is basically enough to make per-package SDK and target work. The
only gotcha is that at the end of the build, output/target and
output/host are empty, which means that:
- The filesystem image creation code cannot work.
- We don't have a SDK to build code outside of Buildroot.
In order to fix this, this commit extends the target-finalize step so
that it starts by populating output/target and output/host by
rsync-ing into them the target and host directories of all packages
listed in the $(PACKAGES) variable. It is necessary to do this
sequentially in the target-finalize step and not in each
package. Doing it in package installation means that it can be done in
parallel. In that case, there is a chance that two rsyncs are creating
the same hardlink or directory at the same time, which makes one of
them fail.
This change to per-package directories has an impact on the RPATH
built into the host binaries, as those RPATH now point to various
per-package host directories, and no longer to the global host
directory. We do not try to rewrite such RPATHs during the build as
having such RPATHs is perfectly fine, but we still need to handle two
fallouts from this change:
- The check-host-rpath script, which verifies at the end of each
package installation that it has the appropriate RPATH, is modified
to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is
a correct RPAT.
- The fix-rpath script, which mungles the RPATH mainly for the SDK
preparation, is modified to rewrite the RPATH to not point to
per-package directories. Indeed the patchelf --make-rpath-relative
call only works if the RPATH points to the ROOTDIR passed as
argument, and this ROOTDIR is the global host directory. Rewriting
the RPATH to not point to per-package host directories prior to
this is an easy solution to this issue.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2019-11-05 09:46:40 -07:00
|
|
|
$$($(2)_TARGET_DIRCLEAN): NAME=$(1)
|
2009-11-08 10:37:36 -07:00
|
|
|
|
2011-07-11 14:46:10 -06:00
|
|
|
# Compute the name of the Kconfig option that correspond to the
|
|
|
|
|
# package being enabled. We handle three cases: the special Linux
|
|
|
|
|
# kernel case, the bootloaders case, and the normal packages case.
|
2019-10-30 07:27:57 -06:00
|
|
|
# Virtual packages are handled separately (see below).
|
2011-07-11 14:46:10 -06:00
|
|
|
ifeq ($(1),linux)
|
|
|
|
|
$(2)_KCONFIG_VAR = BR2_LINUX_KERNEL
|
core: add support for multiple br2-external trees
Currently, we only support at most one br2-external tree. Being able
to use more than one br2-external tree can be very useful.
A use-case would be for having a br2-external to contain the basic
packages, basic board defconfigs and board files, provided by one team
responsible for the "board-bringup", while other teams consume that
br2-external as a base, and complements it each with their own set of
packages, defconfigs and extra board files.
Another use-case would be for third-parties to provide their own
Buildroot packaging in a br2-external tree, along-side the archives for
their stuff.
Finally, another use-case is to be able to add FLOSS packages in a
br2-external tree, and proprietary packages in another. This allows
to not touch the Buildroot tree at all, and still be able to get in
compliance by providing only that br2-external tree(s) that contains
FLOSS packages, leaving aside the br2-external tree(s) with the
proprietary bits.
What we do is to treat BR2_EXTERNAL as a colon-separated (space-
separated also work, and we use that internally) list of paths, on which
we iterate to construct:
- the list of all br2-external names, BR2_EXTERNAL_NAMES,
- the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which
point each to the actual location of the corresponding tree,
- the list of paths to all the external.mk files, BR2_EXTERNAL_MKS,
- the space-separated list of absolute paths to the external trees,
BR2_EXTERNAL_DIRS.
Once we have all those variables, we replace references to BR2_EXTERNAL
with either one of those.
This cascades into how we display the list of defconfigs, so that it is
easy to see what br2-external tree provides what defconfigs. As
suggested by Arnout, tweak the comment from "User-provided configs" to
"External configs", on the assumption that some br2-external trees could
be provided by vendors, so not necessarily user-provided. Ditto the menu
in Kconfig, changed from "User-provided options" to "External options".
Now, when more than one br2-external tree is used, each gets its own
sub-menu in the "User-provided options" menu. The sub-menu is labelled
with that br2-external tree's name and the sub-menu's first item is a
comment with the path to that br2-external tree.
If there's only one br2-external tree, then there is no sub-menu; there
is a single comment that contains the name and path to the br2-external
tree.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-14 08:39:20 -06:00
|
|
|
else ifneq ($$(filter boot/% $$(foreach dir,$$(BR2_EXTERNAL_DIRS),$$(dir)/boot/%),$(pkgdir)),)
|
2011-07-11 14:46:10 -06:00
|
|
|
$(2)_KCONFIG_VAR = BR2_TARGET_$(2)
|
core: add support for multiple br2-external trees
Currently, we only support at most one br2-external tree. Being able
to use more than one br2-external tree can be very useful.
A use-case would be for having a br2-external to contain the basic
packages, basic board defconfigs and board files, provided by one team
responsible for the "board-bringup", while other teams consume that
br2-external as a base, and complements it each with their own set of
packages, defconfigs and extra board files.
Another use-case would be for third-parties to provide their own
Buildroot packaging in a br2-external tree, along-side the archives for
their stuff.
Finally, another use-case is to be able to add FLOSS packages in a
br2-external tree, and proprietary packages in another. This allows
to not touch the Buildroot tree at all, and still be able to get in
compliance by providing only that br2-external tree(s) that contains
FLOSS packages, leaving aside the br2-external tree(s) with the
proprietary bits.
What we do is to treat BR2_EXTERNAL as a colon-separated (space-
separated also work, and we use that internally) list of paths, on which
we iterate to construct:
- the list of all br2-external names, BR2_EXTERNAL_NAMES,
- the per-br2-external tree BR2_EXTERNAL_$(NAME) variables, which
point each to the actual location of the corresponding tree,
- the list of paths to all the external.mk files, BR2_EXTERNAL_MKS,
- the space-separated list of absolute paths to the external trees,
BR2_EXTERNAL_DIRS.
Once we have all those variables, we replace references to BR2_EXTERNAL
with either one of those.
This cascades into how we display the list of defconfigs, so that it is
easy to see what br2-external tree provides what defconfigs. As
suggested by Arnout, tweak the comment from "User-provided configs" to
"External configs", on the assumption that some br2-external trees could
be provided by vendors, so not necessarily user-provided. Ditto the menu
in Kconfig, changed from "User-provided options" to "External options".
Now, when more than one br2-external tree is used, each gets its own
sub-menu in the "User-provided options" menu. The sub-menu is labelled
with that br2-external tree's name and the sub-menu's first item is a
comment with the path to that br2-external tree.
If there's only one br2-external tree, then there is no sub-menu; there
is a single comment that contains the name and path to the br2-external
tree.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Romain Naour <romain.naour@openwide.fr>
Cc: Julien CORJON <corjon.j@ecagroup.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2016-10-14 08:39:20 -06:00
|
|
|
else ifneq ($$(filter toolchain/% $$(foreach dir,$$(BR2_EXTERNAL_DIRS),$$(dir)/toolchain/%),$(pkgdir)),)
|
2013-12-16 07:58:36 -07:00
|
|
|
$(2)_KCONFIG_VAR = BR2_$(2)
|
2011-07-11 14:46:10 -06:00
|
|
|
else
|
|
|
|
|
$(2)_KCONFIG_VAR = BR2_PACKAGE_$(2)
|
|
|
|
|
endif
|
|
|
|
|
|
2012-05-17 11:33:00 -06:00
|
|
|
# legal-info: declare dependencies and set values used later for the manifest
|
2012-11-02 03:25:41 -06:00
|
|
|
ifneq ($$($(2)_LICENSE_FILES),)
|
|
|
|
|
$(2)_MANIFEST_LICENSE_FILES = $$($(2)_LICENSE_FILES)
|
|
|
|
|
endif
|
|
|
|
|
|
core/legal-info: also save patches
Currently, the legal-info infra only saves the source archive of a
package. However, that's not enough as we may apply some patches on
packages sources.
We do suggest users to also redistribute the Buildroot sources as part
of their compliance distribution, so the patches bundled in Buildroot
would indeed be included in the compliance distribution.
However, that's still not enough, since we may download some patches, or
the user may use a global patch directory. Patches in there might not
end up in the compliance distribution, and there are risks of
non-conformity.
So, always include patches alongside the source archive.
To ensure reproducibility, we also generate a series file, so patches
can be re-applied in the correct order.
We get the list of patches to include from the list of patches that were
applied by the package infrastructure (via the apply-patches support
script). So, we need to get packages properly extracted and patched
before we can save their legal-info, not just in the case they define
_LICENSE_FILES.
Update the legal-info header accordingly.
Note: this means that, when a package is not patched and defines no
LICENSE_FILES, we will extract and patch it for nothing. There is no
easy way to know whether we have to patch a package or not. We can only
either duplicate the logic to detect patches (bad) or rely on the infra
actually patching the package. Also, a vast majority of packages are
either patched, or define _LICENSE_FILES, so it is best and easiest to
always extract and patch them prior to legal-info.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-07 10:14:33 -06:00
|
|
|
# We need to extract and patch a package to be able to retrieve its
|
|
|
|
|
# license files (if any) and the list of patches applied to it (if
|
|
|
|
|
# any).
|
2014-07-02 06:39:59 -06:00
|
|
|
$(1)-legal-info: $(1)-patch
|
2014-06-22 06:41:12 -06:00
|
|
|
|
2014-06-22 06:41:14 -06:00
|
|
|
# We only save the sources of packages we want to redistribute, that are
|
2016-03-04 16:39:13 -07:00
|
|
|
# non-overriden (local or true override).
|
2012-11-02 03:25:41 -06:00
|
|
|
ifeq ($$($(2)_REDISTRIBUTE),YES)
|
2016-03-04 16:39:12 -07:00
|
|
|
ifeq ($$($(2)_OVERRIDE_SRCDIR),)
|
2014-06-22 06:41:12 -06:00
|
|
|
# Packages that have a tarball need it downloaded beforehand
|
|
|
|
|
$(1)-legal-info: $(1)-source $$(REDIST_SOURCES_DIR_$$(call UPPERCASE,$(4)))
|
2012-05-17 11:33:00 -06:00
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
|
|
|
|
# legal-info: produce legally relevant info.
|
2017-06-25 16:03:38 -06:00
|
|
|
$(1)-legal-info: PKG=$(2)
|
2012-05-17 11:33:00 -06:00
|
|
|
$(1)-legal-info:
|
2017-06-25 16:03:38 -06:00
|
|
|
@$$(call MESSAGE,"Collecting legal info")
|
2012-05-17 11:33:00 -06:00
|
|
|
# Packages without a source are assumed to be part of Buildroot, skip them.
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 13:12:24 -06:00
|
|
|
$$(foreach hook,$$($(2)_PRE_LEGAL_INFO_HOOKS),$$(call $$(hook))$$(sep))
|
|
|
|
|
ifneq ($$(call qstrip,$$($(2)_SOURCE)),)
|
2013-11-12 01:47:56 -07:00
|
|
|
|
2014-06-22 06:41:13 -06:00
|
|
|
# Save license files if defined
|
|
|
|
|
# We save the license files for any kind of package: normal, local,
|
|
|
|
|
# overridden, or non-redistributable alike.
|
|
|
|
|
# The reason to save license files even for no-redistribute packages
|
|
|
|
|
# is that the license still applies to the files distributed as part
|
|
|
|
|
# of the rootfs, even if the sources are not themselves redistributed.
|
|
|
|
|
ifeq ($$(call qstrip,$$($(2)_LICENSE_FILES)),)
|
core: rename FOO_BASE_NAME to FOO_BASENAME to avoid clashes
In current Buildroot, clashes occur between the variables _NAME and
_BASE_NAME for two packages called foo and foo-base, i.e.
Package foo:
FOO_NAME = foo
FOO_BASE_NAME = foo-1.2.3
Package foo-base:
FOO_BASE_NAME = foo-base
FOO_BASE_BASE_NAME = foo-base-4.5.6
where variable FOO_BASE_NAME is clashing between these two packages.
Specific cases where this clash is already existing are:
- alljoyn-base
- alljoyn-tcl-base
- perl-xml-sax-base
The problem is generic and can occur for a number of variables in Buildroot.
A non-exhaustive list:
<pkg>_BASE and <pkg>_BASE_NAME
<pkg>_BASE_NAME and <pkg>_RAW_BASE_NAME
<pkg>_DIR and <pkg>_DL_DIR
<pkg>_VERSION and <pkg>_DL_VERSION
<pkg>_SOURCE and <pkg>_TARGET_SOURCE
<pkg>_INSTALL_IMAGES and <pkg>_TARGET_INSTALL_IMAGES (same for _STAGING and _TARGET)
<pkg>_LICENSE_FILES and <pkg>_MANIFEST_LICENSE_FILES
<pkg>_DEPENDENCIES and <pkg>_FINAL_DEPENDENCIES
One solution is to use another separator than '_' to separate the
package name from the rest of the variable name. For example, a double
underscore:
FOO__NAME
FOO__BASE_NAME
FOO_BASE__NAME
FOO_BASE__BASE_NAME
However, making that change for only this case means that the variable
naming is no longer consistent. And making the change for all variables has
a large impact, also on certain user scripts.
For now, keep it simple, and rename FOO_BASE_NAME into FOO_BASENAME, so that
the variables become:
FOO_NAME
FOO_BASENAME
FOO_BASE_NAME
FOO_BASE_BASENAME
For consistency, also adapt FOO_RAW_BASE_NAME. Since FOO_RAW_BASENAME would
still pose a conflict with a package called 'foo-raw', take the opportunity
to rename it into FOO_BASENAME_RAW instead, which does not pose a conflict
as we have no variable called FOO_RAW.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Sam Voss <sam.voss@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-02-06 05:59:23 -07:00
|
|
|
$(Q)$$(call legal-warning-pkg,$$($(2)_BASENAME_RAW),cannot save license ($(2)_LICENSE_FILES not defined))
|
2014-06-22 06:41:13 -06:00
|
|
|
else
|
2018-10-14 06:25:41 -06:00
|
|
|
$(Q)$$(foreach F,$$($(2)_LICENSE_FILES),$$(call legal-license-file,$$($(2)_RAWNAME),$$($(2)_BASENAME_RAW),$$($(2)_HASH_FILE),$$(F),$$($(2)_DIR)/$$(F),$$(call UPPERCASE,$(4)))$$(sep))
|
2014-06-22 06:41:13 -06:00
|
|
|
endif # license files
|
|
|
|
|
|
2012-11-02 03:25:41 -06:00
|
|
|
ifeq ($$($(2)_SITE_METHOD),local)
|
2012-05-17 11:33:00 -06:00
|
|
|
# Packages without a tarball: don't save and warn
|
2014-06-22 06:41:15 -06:00
|
|
|
@$$(call legal-warning-nosource,$$($(2)_RAWNAME),local)
|
2013-11-12 01:47:56 -07:00
|
|
|
|
2012-12-02 17:05:51 -07:00
|
|
|
else ifneq ($$($(2)_OVERRIDE_SRCDIR),)
|
2014-06-22 06:41:15 -06:00
|
|
|
@$$(call legal-warning-nosource,$$($(2)_RAWNAME),override)
|
2013-11-12 01:47:56 -07:00
|
|
|
|
2012-05-17 11:33:00 -06:00
|
|
|
else
|
|
|
|
|
# Other packages
|
2013-11-12 01:47:56 -07:00
|
|
|
|
2012-11-02 03:25:41 -06:00
|
|
|
ifeq ($$($(2)_REDISTRIBUTE),YES)
|
2016-05-07 10:14:34 -06:00
|
|
|
# Save the source tarball and any extra downloads, but not
|
|
|
|
|
# patches, as they are handled specially afterwards.
|
|
|
|
|
$$(foreach e,$$($(2)_ACTUAL_SOURCE_TARBALL) $$(notdir $$($(2)_EXTRA_DOWNLOADS)),\
|
|
|
|
|
$$(Q)support/scripts/hardlink-or-copy \
|
2018-04-02 07:09:26 -06:00
|
|
|
$$($(2)_DL_DIR)/$$(e) \
|
2016-05-07 10:14:34 -06:00
|
|
|
$$($(2)_REDIST_SOURCES_DIR)$$(sep))
|
core/legal-info: also save patches
Currently, the legal-info infra only saves the source archive of a
package. However, that's not enough as we may apply some patches on
packages sources.
We do suggest users to also redistribute the Buildroot sources as part
of their compliance distribution, so the patches bundled in Buildroot
would indeed be included in the compliance distribution.
However, that's still not enough, since we may download some patches, or
the user may use a global patch directory. Patches in there might not
end up in the compliance distribution, and there are risks of
non-conformity.
So, always include patches alongside the source archive.
To ensure reproducibility, we also generate a series file, so patches
can be re-applied in the correct order.
We get the list of patches to include from the list of patches that were
applied by the package infrastructure (via the apply-patches support
script). So, we need to get packages properly extracted and patched
before we can save their legal-info, not just in the case they define
_LICENSE_FILES.
Update the legal-info header accordingly.
Note: this means that, when a package is not patched and defines no
LICENSE_FILES, we will extract and patch it for nothing. There is no
easy way to know whether we have to patch a package or not. We can only
either duplicate the logic to detect patches (bad) or rely on the infra
actually patching the package. Also, a vast majority of packages are
either patched, or define _LICENSE_FILES, so it is best and easiest to
always extract and patch them prior to legal-info.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Luca Ceresoli <luca@lucaceresoli.net>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-07 10:14:33 -06:00
|
|
|
# Save patches and generate the series file
|
|
|
|
|
$$(Q)while read f; do \
|
|
|
|
|
support/scripts/hardlink-or-copy \
|
|
|
|
|
$$$${f} \
|
|
|
|
|
$$($(2)_REDIST_SOURCES_DIR) || exit 1; \
|
|
|
|
|
printf "%s\n" "$$$${f##*/}" >>$$($(2)_REDIST_SOURCES_DIR)/series || exit 1; \
|
|
|
|
|
done <$$($(2)_DIR)/.applied_patches_list
|
2013-11-12 01:47:56 -07:00
|
|
|
endif # redistribute
|
|
|
|
|
|
|
|
|
|
endif # other packages
|
2019-10-26 02:45:53 -06:00
|
|
|
@$$(call legal-manifest,$$(call UPPERCASE,$(4)),$$($(2)_RAWNAME),$$($(2)_VERSION),$$(subst $$(space)$$(comma),$$(comma),$$($(2)_LICENSE)),$$($(2)_MANIFEST_LICENSE_FILES),$$($(2)_ACTUAL_SOURCE_TARBALL),$$($(2)_ACTUAL_SOURCE_SITE),$$(call legal-deps,$(1)))
|
infra: consistently use double dollar signs inside inner-xxx-targets
The inner-xxx-targets in the buildroot package infrastructures are
evaluated using $(eval) which causes variable references to be a bit
different than in regular make code. As we want most references to be
expanded only at the time of the $(eval) we should not use standard
references $(VAR) but rather use double dollar signs $$(VAR). This includes
function references like $(call), $(subst), etc. The only exception is the
reference to pkgdir/pkgname and numbered variables, which are parameters to
the inner block: $(1), $(2), etc.
This patch introduces consistent usage of double-dollar signs throughout the
different inner-xxx-targets blocks.
In some cases, this would potentially cause circular references, in
particular when the value of HOST_FOO_VAR would be obtained from the
corresponding FOO_VAR if HOST_FOO_VAR is not defined. In these cases, a test
is added to check for a host package (the only case where such constructions
are relevant; these are not circular).
Benefits of these changes are:
- behavior of variables is now again as expected. For example, setting
$(2)_VERSION = virtual in pkg-virtual.mk will effectively work, while
originally it would cause very odd results.
- The output of 'make printvars' is now much more useful. This target shows
the value of all variables, and the expression that led to that value.
However, if the expression was coming from an inner-xxx-targets block, and
was using single dollar signs, it would show in printvars as
VAR = value (value)
while if double dollar signs are used, it would effectively look like
VAR = value (actual expression)
as is intended.
This improvement is for example effective for FOO_DL_VERSION, FOO_RAWNAME,
FOO_SITE_METHOD and FOO_MAKE.
The correctness of this patch has been verified using 'make printvars',
'make manual' and 'make legal-info' before and after applying this patch,
and comparing the output.
Insight-provided-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2014-06-11 13:12:24 -06:00
|
|
|
endif # ifneq ($$(call qstrip,$$($(2)_SOURCE)),)
|
|
|
|
|
$$(foreach hook,$$($(2)_POST_LEGAL_INFO_HOOKS),$$(call $$(hook))$$(sep))
|
2012-05-17 11:33:00 -06:00
|
|
|
|
2009-11-08 10:37:36 -07:00
|
|
|
# add package to the general list of targets if requested by the buildroot
|
|
|
|
|
# configuration
|
2011-07-11 14:46:10 -06:00
|
|
|
ifeq ($$($$($(2)_KCONFIG_VAR)),y)
|
2010-12-24 07:57:29 -07:00
|
|
|
|
2014-05-15 11:37:03 -06:00
|
|
|
# Ensure the calling package is the declared provider for all the virtual
|
|
|
|
|
# packages it claims to be an implementation of.
|
|
|
|
|
ifneq ($$($(2)_PROVIDES),)
|
|
|
|
|
$$(foreach pkg,$$($(2)_PROVIDES),\
|
|
|
|
|
$$(eval $$(call virt-provides-single,$$(pkg),$$(call UPPERCASE,$$(pkg)),$(1))$$(sep)))
|
|
|
|
|
endif
|
|
|
|
|
|
2016-10-23 09:28:51 -06:00
|
|
|
# Register package as a reverse-dependencies of all its dependencies
|
|
|
|
|
$$(eval $$(foreach p,$$($(2)_FINAL_ALL_DEPENDENCIES),\
|
|
|
|
|
$$(call UPPERCASE,$$(p))_RDEPENDENCIES += $(1)$$(sep)))
|
|
|
|
|
|
2014-10-05 03:35:14 -06:00
|
|
|
# Ensure unified variable name conventions between all packages Some
|
|
|
|
|
# of the variables are used by more than one infrastructure; so,
|
|
|
|
|
# rather than duplicating the checks in each infrastructure, we check
|
|
|
|
|
# all variables here in pkg-generic, even though pkg-generic should
|
|
|
|
|
# have no knowledge of infra-specific variables.
|
2014-09-27 13:32:49 -06:00
|
|
|
$(eval $(call check-deprecated-variable,$(2)_MAKE_OPT,$(2)_MAKE_OPTS))
|
|
|
|
|
$(eval $(call check-deprecated-variable,$(2)_INSTALL_OPT,$(2)_INSTALL_OPTS))
|
|
|
|
|
$(eval $(call check-deprecated-variable,$(2)_INSTALL_TARGET_OPT,$(2)_INSTALL_TARGET_OPTS))
|
|
|
|
|
$(eval $(call check-deprecated-variable,$(2)_INSTALL_STAGING_OPT,$(2)_INSTALL_STAGING_OPTS))
|
|
|
|
|
$(eval $(call check-deprecated-variable,$(2)_INSTALL_HOST_OPT,$(2)_INSTALL_HOST_OPTS))
|
|
|
|
|
$(eval $(call check-deprecated-variable,$(2)_AUTORECONF_OPT,$(2)_AUTORECONF_OPTS))
|
|
|
|
|
$(eval $(call check-deprecated-variable,$(2)_CONF_OPT,$(2)_CONF_OPTS))
|
|
|
|
|
$(eval $(call check-deprecated-variable,$(2)_BUILD_OPT,$(2)_BUILD_OPTS))
|
|
|
|
|
$(eval $(call check-deprecated-variable,$(2)_GETTEXTIZE_OPT,$(2)_GETTEXTIZE_OPTS))
|
|
|
|
|
$(eval $(call check-deprecated-variable,$(2)_KCONFIG_OPT,$(2)_KCONFIG_OPTS))
|
|
|
|
|
|
2015-04-12 10:37:48 -06:00
|
|
|
PACKAGES += $(1)
|
2014-05-22 06:35:40 -06:00
|
|
|
|
|
|
|
|
ifneq ($$($(2)_PERMISSIONS),)
|
2012-01-11 10:53:38 -07:00
|
|
|
PACKAGES_PERMISSIONS_TABLE += $$($(2)_PERMISSIONS)$$(sep)
|
2014-05-22 06:35:40 -06:00
|
|
|
endif
|
|
|
|
|
ifneq ($$($(2)_DEVICES),)
|
2012-01-11 10:53:38 -07:00
|
|
|
PACKAGES_DEVICES_TABLE += $$($(2)_DEVICES)$$(sep)
|
2014-05-22 06:35:40 -06:00
|
|
|
endif
|
|
|
|
|
ifneq ($$($(2)_USERS),)
|
2013-04-12 01:14:18 -06:00
|
|
|
PACKAGES_USERS += $$($(2)_USERS)$$(sep)
|
2014-05-22 06:35:40 -06:00
|
|
|
endif
|
linux: allow packages to set kernel config options
Currently, the linux kernel will apply some fixups on its .config file,
based on whether some packages are enabled or not. That list of
conditional fixups is getting bigger and bigger with each new package
that needs such fixups, culminating with the pending firewalld one [0].
Furthermore, these fixups are not accessible to packages in br2-external
trees.
Add a new per-package variable, that packages may set to the commands to
run to fixup the kernel .config file, which is added at the end of the
linux' own fixups.
This opens the possibility to write things like;
define FOO_LINUX_CONFIG_FIXUPS
$(call KCONFIG_ENABLE_OPT,BLA)
endef
Of course, it also opens the way to run arbitrary commands in there, but
any alternative that would be declarative only, such as a list of
options to enable or disable (as an example):
FOO_LINUX_CONFIG_FIXUPS = +BAR -FOO +BUZ="value"
.. is not very nice either, and such lists fall flat when a value would
have a space.
For packages that we have in-tree, we can ensure they won't play foul
with their _LINUX_CONFIG_FIXUPS. For packages in br2-external trees,
there's nothing we can do; users already have the opportunity to hack
into the linux configure process by providing LINUX_PRE_CONFIGURE_HOOKS
or LINUX_POST_CONFIGURE_HOOKS anyway...
.. which brings the question of why we don't use that to implement the
per-package fixups. We don't, because _PRE or _POST_CONFIGURE_HOOKS are
run after we run 'make oldconfig' to sanitise the mangled .config.
[0] http://lists.busybox.net/pipermail/buildroot/2020-March/278683.html
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-04-04 06:10:21 -06:00
|
|
|
ifneq ($$($(2)_LINUX_CONFIG_FIXUPS),)
|
|
|
|
|
PACKAGES_LINUX_CONFIG_FIXUPS += $$($(2)_LINUX_CONFIG_FIXUPS)$$(sep)
|
|
|
|
|
endif
|
2016-06-22 13:07:36 -06:00
|
|
|
TARGET_FINALIZE_HOOKS += $$($(2)_TARGET_FINALIZE_HOOKS)
|
fs: add pre- and post-command hooks
In some cases, the directory structure we want in the filesystem is not
exactly what we have in target/
For example, when systemd is used on a read-only rootfs, /var must be a
tmpfs. However, we may have packages that install stuff in there, and
set important rights (via the permission-table). So, at build time, we
need /var to be a symlink to the remanent location (/usr/share/factory)
while at runtime we need /var to be a directory.
One option would have been to have /var as a real directory even during
build time, and in a target-finalize hook, move everything out of there
and into the "factory" location. However, that's not possible because
it's too early: some packages may want to set ownership and/or acces
rights on directories or files in /var, and this is only done in the
fakeroot script, which is called only later during the assembling of the
filesystem images.
Also, there would have been no way to undo the tweak (i.e. we need to
restore the /var symlink so that subsequent builds continue to work) if
it were done as a target-finalize hook.
The only solution is to allow packages to register pre- and post-hooks
that are called right before and right after the rootfs commands are
executed, and inside in the fakeroot script.
We can however not re-use the BR2_ROOTFS_POST_FAKEROOT_SCRIPT feature
either because it is done before the filesystem command, but there is
nothing that is done after. Also, we don't want to add to, and modify a
user-supplied variable.
So, we introduce two new variables that packages can set to add the
commands they need to run to tweak the filesystem right at the last
moment.
Those hooks are not documented on-purpose; they are probably going to
only ever be used by systemd.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Reviewed-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2017-08-01 16:52:22 -06:00
|
|
|
ROOTFS_PRE_CMD_HOOKS += $$($(2)_ROOTFS_PRE_CMD_HOOKS)
|
2019-12-01 13:55:37 -07:00
|
|
|
KEEP_PYTHON_PY_FILES += $$($(2)_KEEP_PY_FILES)
|
2010-12-24 07:57:29 -07:00
|
|
|
|
2020-07-31 04:10:30 -06:00
|
|
|
ifneq ($$($(2)_SELINUX_MODULES),)
|
|
|
|
|
PACKAGES_SELINUX_MODULES += $$($(2)_SELINUX_MODULES)
|
|
|
|
|
endif
|
2020-07-31 04:10:38 -06:00
|
|
|
PACKAGES_SELINUX_EXTRA_MODULES_DIRS += \
|
|
|
|
|
$$(if $$(wildcard $$($(2)_PKGDIR)/selinux),$$($(2)_PKGDIR)/selinux)
|
2020-07-31 04:10:30 -06:00
|
|
|
|
2010-12-24 07:57:29 -07:00
|
|
|
ifeq ($$($(2)_SITE_METHOD),svn)
|
|
|
|
|
DL_TOOLS_DEPENDENCIES += svn
|
|
|
|
|
else ifeq ($$($(2)_SITE_METHOD),git)
|
|
|
|
|
DL_TOOLS_DEPENDENCIES += git
|
|
|
|
|
else ifeq ($$($(2)_SITE_METHOD),bzr)
|
|
|
|
|
DL_TOOLS_DEPENDENCIES += bzr
|
2011-10-19 01:25:40 -06:00
|
|
|
else ifeq ($$($(2)_SITE_METHOD),scp)
|
|
|
|
|
DL_TOOLS_DEPENDENCIES += scp ssh
|
2011-10-19 01:25:47 -06:00
|
|
|
else ifeq ($$($(2)_SITE_METHOD),hg)
|
|
|
|
|
DL_TOOLS_DEPENDENCIES += hg
|
2013-09-11 06:12:04 -06:00
|
|
|
else ifeq ($$($(2)_SITE_METHOD),cvs)
|
|
|
|
|
DL_TOOLS_DEPENDENCIES += cvs
|
2010-12-24 07:57:29 -07:00
|
|
|
endif # SITE_METHOD
|
|
|
|
|
|
2019-12-16 08:31:13 -07:00
|
|
|
DL_TOOLS_DEPENDENCIES += $$(call extractor-system-dependency,$$($(2)_SOURCE))
|
2011-12-04 12:23:04 -07:00
|
|
|
|
2015-03-29 11:33:28 -06:00
|
|
|
# Ensure all virtual targets are PHONY. Listed alphabetically.
|
|
|
|
|
.PHONY: $(1) \
|
|
|
|
|
$(1)-all-external-deps \
|
|
|
|
|
$(1)-all-legal-info \
|
|
|
|
|
$(1)-all-source \
|
|
|
|
|
$(1)-build \
|
|
|
|
|
$(1)-clean-for-rebuild \
|
|
|
|
|
$(1)-clean-for-reconfigure \
|
|
|
|
|
$(1)-clean-for-reinstall \
|
|
|
|
|
$(1)-configure \
|
|
|
|
|
$(1)-depends \
|
|
|
|
|
$(1)-dirclean \
|
|
|
|
|
$(1)-external-deps \
|
|
|
|
|
$(1)-extract \
|
|
|
|
|
$(1)-graph-depends \
|
2019-03-22 15:07:05 -06:00
|
|
|
$(1)-graph-rdepends \
|
2015-03-29 11:33:28 -06:00
|
|
|
$(1)-install \
|
|
|
|
|
$(1)-install-host \
|
|
|
|
|
$(1)-install-images \
|
|
|
|
|
$(1)-install-staging \
|
|
|
|
|
$(1)-install-target \
|
|
|
|
|
$(1)-legal-info \
|
core/legal-info: ensure legal-info works in off-line mode
Almost all packages which are saved for legal-info have their source
archives downloaded as part of 'make source', which makes an off-line
build completely possible [0].
However, for the pre-configured external toolchains, the source tarball
is different, as the main tarball is a binary package. And that source
tarball is only downloaded during the legal-info phase, which makes it
inconvenient for full off-line builds.
We fix that by adding a new rule, $(1)-legal-source which only
$(1)-all-source depends on, so that we only download it for a top-level
'make source', not as part of the standard download mechanism (i.e. only
what is really needed to build).
This new rule depends, like the normal download mechanism, on a stamp
file, so that we do not emit a spurious hash-check message on successive
runs of 'make source'.
This way, we can do a complete [0] off-line build and are still able to
generate legal-info, while at the same time we do not incur any download
overhead during a simple build.
Also, we previously downloaded the _ACTUAL_SOURCE_TARBALL when it was
not empty. However, since _ACTUAL_SOURCE_TARBALL defaults to the value
of _SOURCE, it can not be empty when _SOURCE is not. Thus, we'd get a
spurious report of a missing hash for the tarball, since it was not in
a standard package rule (configure, build, install..) and thus would
miss the PKG and PKGDIR variables to find the .hash file.
We fix that in this commit as well, by:
- setting PKG and PKGDIR just for the -legal-source rule;
- only downloading _ACTUAL_SOURCE_TARBALL if it is not empty *and* not
the same as _SOURCE (to avoid a second report about the hash).
[0] Save for nodejs which invarriably wants to download stuff at build
time. Sigh... :-( Fixing that is work for another time...
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Luca Ceresoli <luca@lucaceresoli.net>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Peter Korsgaard <jacmet@uclibc.org>
Tested-by: Luca Ceresoli <luca@lucaceresoli.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2016-05-07 10:14:38 -06:00
|
|
|
$(1)-legal-source \
|
2015-03-29 11:33:28 -06:00
|
|
|
$(1)-patch \
|
|
|
|
|
$(1)-rebuild \
|
|
|
|
|
$(1)-reconfigure \
|
|
|
|
|
$(1)-reinstall \
|
|
|
|
|
$(1)-rsync \
|
|
|
|
|
$(1)-show-depends \
|
core: add per-package and per-filesystem show-info
Sometimes, it is need to quickly get the metadata of a subset of
packages, without resorting to a full-blown JSON query.
Introduce a new per-package (and per-filesystem) foo-show-info rule,
that otputs a per-entity valid JSON blob.
Note that calling it for multiple packages and.or filesystems at once
will not generate a valid JSON blob, as there would be no separator
between the JSON elements:
$ make {foo,bar}-show-info
{ "foo": { foo stuff } }
{ "bar": { bar stuff } }
However, jq is able to absorb this, with its slurping ability, which
generates an array (ellipsed and manualy reformated for readability):
$ make {foo,bar}-show-info |jq -s . -
[
{ "foo": { foo stuff } },
{ "bar": { bar stuff } }
]
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Thomas De Schampheleire <patrickdepinguin@gmail.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2019-04-15 13:47:32 -06:00
|
|
|
$(1)-show-info \
|
2015-03-29 11:33:28 -06:00
|
|
|
$(1)-show-version \
|
2017-10-25 14:09:51 -06:00
|
|
|
$(1)-source
|
2015-03-29 11:33:28 -06:00
|
|
|
|
2016-07-03 03:50:01 -06:00
|
|
|
ifneq ($$($(2)_SOURCE),)
|
|
|
|
|
ifeq ($$($(2)_SITE),)
|
|
|
|
|
$$(error $(2)_SITE cannot be empty when $(2)_SOURCE is not)
|
|
|
|
|
endif
|
|
|
|
|
endif
|
|
|
|
|
|
2015-10-03 11:22:17 -06:00
|
|
|
ifeq ($$(patsubst %/,ERROR,$$($(2)_SITE)),ERROR)
|
|
|
|
|
$$(error $(2)_SITE ($$($(2)_SITE)) cannot have a trailing slash)
|
|
|
|
|
endif
|
|
|
|
|
|
2016-06-04 10:30:47 -06:00
|
|
|
ifneq ($$($(2)_HELP_CMDS),)
|
|
|
|
|
HELP_PACKAGES += $(2)
|
|
|
|
|
endif
|
|
|
|
|
|
2019-10-30 07:27:57 -06:00
|
|
|
# Virtual packages are not built but it's useful to allow them to have
|
|
|
|
|
# permission/device/user tables and target-finalize/rootfs-pre-cmd hooks.
|
|
|
|
|
else ifeq ($$(BR2_PACKAGE_HAS_$(2)),y) # $(2)_KCONFIG_VAR
|
|
|
|
|
|
|
|
|
|
ifneq ($$($(2)_PERMISSIONS),)
|
|
|
|
|
PACKAGES_PERMISSIONS_TABLE += $$($(2)_PERMISSIONS)$$(sep)
|
|
|
|
|
endif
|
|
|
|
|
ifneq ($$($(2)_DEVICES),)
|
|
|
|
|
PACKAGES_DEVICES_TABLE += $$($(2)_DEVICES)$$(sep)
|
|
|
|
|
endif
|
|
|
|
|
ifneq ($$($(2)_USERS),)
|
|
|
|
|
PACKAGES_USERS += $$($(2)_USERS)$$(sep)
|
|
|
|
|
endif
|
|
|
|
|
TARGET_FINALIZE_HOOKS += $$($(2)_TARGET_FINALIZE_HOOKS)
|
|
|
|
|
ROOTFS_PRE_CMD_HOOKS += $$($(2)_ROOTFS_PRE_CMD_HOOKS)
|
|
|
|
|
|
2011-07-11 14:46:10 -06:00
|
|
|
endif # $(2)_KCONFIG_VAR
|
2012-07-02 16:07:08 -06:00
|
|
|
endef # inner-generic-package
|
2009-11-08 10:37:36 -07:00
|
|
|
|
|
|
|
|
################################################################################
|
2012-07-02 16:07:08 -06:00
|
|
|
# generic-package -- the target generator macro for generic packages
|
2009-11-08 10:37:36 -07:00
|
|
|
################################################################################
|
|
|
|
|
|
|
|
|
|
# In the case of target packages, keep the package name "pkg"
|
2014-02-05 02:44:03 -07:00
|
|
|
generic-package = $(call inner-generic-package,$(pkgname),$(call UPPERCASE,$(pkgname)),$(call UPPERCASE,$(pkgname)),target)
|
2012-07-02 16:05:46 -06:00
|
|
|
# In the case of host packages, turn the package name "pkg" into "host-pkg"
|
2014-02-05 02:44:03 -07:00
|
|
|
host-generic-package = $(call inner-generic-package,host-$(pkgname),$(call UPPERCASE,host-$(pkgname)),$(call UPPERCASE,$(pkgname)),host)
|
2009-11-08 10:37:36 -07:00
|
|
|
|
|
|
|
|
# :mode=makefile:
|