c4e6d5c8be
This commit implements the core of the move to per-package SDK and target directories. The main idea is that instead of having a global output/host and output/target in which all packages install files, we switch to per-package host and target directories, that only contain their explicit dependencies. There are two main benefits: - Packages will now see only the dependencies they explicitly list in their <pkg>_DEPENDENCIES variable, and the recursive dependencies thereof. - We can support top-level parallel build properly, because a package only "sees" its own host directory and target directory, isolated from the build of other packages that can happen in parallel. It works as follows: - A new output/per-package/ directory is created, which will contain one sub-directory per package, and inside it, a "host" directory and a "target" directory: output/per-package/busybox/target output/per-package/busybox/host output/per-package/host-fakeroot/target output/per-package/host-fakeroot/host This output/per-package/ directory is PER_PACKAGE_DIR. - The global TARGET_DIR and HOST_DIR variable now automatically point to the per-package directory when PKG is defined. So whenever a package references $(HOST_DIR) or $(TARGET_DIR) in its build process, it effectively references the per-package host/target directories. Note that STAGING_DIR is a sub-dir of HOST_DIR, so it is handled as well. - Of course, packages have dependencies, so those dependencies must be installed in the per-package host and target directories. To do so, we simply rsync (using hard links to save space and time) the host and target directories of the direct dependencies of the package to the current package host and target directories. We only need to take care of direct dependencies (and not recursively all dependencies), because we accumulate into those per-package host and target directories the files installed by the dependencies. Note that this only works because we make the assumption that one package does *not* overwrite files installed by another package. This is done for "extract dependencies" at the beginning of the extract step, and for "normal dependencies" at the beginning of the configure step. This is basically enough to make per-package SDK and target work. The only gotcha is that at the end of the build, output/target and output/host are empty, which means that: - The filesystem image creation code cannot work. - We don't have a SDK to build code outside of Buildroot. In order to fix this, this commit extends the target-finalize step so that it starts by populating output/target and output/host by rsync-ing into them the target and host directories of all packages listed in the $(PACKAGES) variable. It is necessary to do this sequentially in the target-finalize step and not in each package. Doing it in package installation means that it can be done in parallel. In that case, there is a chance that two rsyncs are creating the same hardlink or directory at the same time, which makes one of them fail. This change to per-package directories has an impact on the RPATH built into the host binaries, as those RPATH now point to various per-package host directories, and no longer to the global host directory. We do not try to rewrite such RPATHs during the build as having such RPATHs is perfectly fine, but we still need to handle two fallouts from this change: - The check-host-rpath script, which verifies at the end of each package installation that it has the appropriate RPATH, is modified to understand that a RPATH to $(PER_PACKAGE_DIR)/<pkg>/host/lib is a correct RPAT. - The fix-rpath script, which mungles the RPATH mainly for the SDK preparation, is modified to rewrite the RPATH to not point to per-package directories. Indeed the patchelf --make-rpath-relative call only works if the RPATH points to the ROOTDIR passed as argument, and this ROOTDIR is the global host directory. Rewriting the RPATH to not point to per-package host directories prior to this is an easy solution to this issue. Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
112 lines
4.2 KiB
Bash
Executable file
112 lines
4.2 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
|
|
# This script scans $(HOST_DIR)/{bin,sbin} for all ELF files, and checks
|
|
# they have an RPATH to $(HOST_DIR)/lib if they need libraries from
|
|
# there.
|
|
|
|
# Override the user's locale so we are sure we can parse the output of
|
|
# readelf(1) and file(1)
|
|
export LC_ALL=C
|
|
|
|
main() {
|
|
local pkg="${1}"
|
|
local hostdir="${2}"
|
|
local perpackagedir="${3}"
|
|
local file ret
|
|
|
|
# Remove duplicate and trailing '/' for proper match
|
|
hostdir="$( sed -r -e 's:/+:/:g; s:/$::;' <<<"${hostdir}" )"
|
|
|
|
ret=0
|
|
while read file; do
|
|
is_elf "${file}" || continue
|
|
elf_needs_rpath "${file}" "${hostdir}" || continue
|
|
check_elf_has_rpath "${file}" "${hostdir}" "${perpackagedir}" && continue
|
|
if [ ${ret} -eq 0 ]; then
|
|
ret=1
|
|
printf "***\n"
|
|
printf "*** ERROR: package %s installs executables without proper RPATH:\n" "${pkg}"
|
|
fi
|
|
printf "*** %s\n" "${file}"
|
|
done < <( find "${hostdir}"/{bin,sbin} -type f 2>/dev/null )
|
|
|
|
return ${ret}
|
|
}
|
|
|
|
is_elf() {
|
|
local f="${1}"
|
|
|
|
readelf -l "${f}" 2>/dev/null \
|
|
|grep -E 'Requesting program interpreter:' >/dev/null 2>&1
|
|
}
|
|
|
|
# This function tells whether a given ELF executable (first argument)
|
|
# needs a RPATH pointing to the host library directory or not. It
|
|
# needs such an RPATH if at least of the libraries used by the ELF
|
|
# executable is available in the host library directory. This function
|
|
# returns 0 when a RPATH is needed, 1 otherwise.
|
|
#
|
|
# With per-package directory support, ${hostdir} will point to the
|
|
# current package per-package host directory, and this is where this
|
|
# function will check if the libraries needed by the executable are
|
|
# located (or not). In practice, the ELF executable RPATH may point to
|
|
# another package per-package host directory, but that is fine because
|
|
# if such an executable is within the current package per-package host
|
|
# directory, its libraries will also have been copied into the current
|
|
# package per-package host directory.
|
|
elf_needs_rpath() {
|
|
local file="${1}"
|
|
local hostdir="${2}"
|
|
local lib
|
|
|
|
while read lib; do
|
|
[ -e "${hostdir}/lib/${lib}" ] && return 0
|
|
done < <( readelf -d "${file}" \
|
|
|sed -r -e '/^.* \(NEEDED\) .*Shared library: \[(.+)\]$/!d;' \
|
|
-e 's//\1/;' \
|
|
)
|
|
|
|
return 1
|
|
}
|
|
|
|
# This function checks whether at least one of the RPATH of the given
|
|
# ELF executable (first argument) properly points to the host library
|
|
# directory (second argument), either through an absolute RPATH or a
|
|
# relative RPATH. In the context of per-package directory support,
|
|
# ${hostdir} (second argument) points to the current package host
|
|
# directory. However, it is perfectly valid for an ELF binary to have
|
|
# a RPATH pointing to another package per-package host directory,
|
|
# which is why such RPATH is also accepted (the per-package directory
|
|
# gets passed as third argument). Having a RPATH pointing to the host
|
|
# directory will make sure the ELF executable will find at runtime the
|
|
# shared libraries it depends on. This function returns 0 when a
|
|
# proper RPATH was found, or 1 otherwise.
|
|
check_elf_has_rpath() {
|
|
local file="${1}"
|
|
local hostdir="${2}"
|
|
local perpackagedir="${3}"
|
|
local rpath dir
|
|
|
|
while read rpath; do
|
|
for dir in ${rpath//:/ }; do
|
|
# Remove duplicate and trailing '/' for proper match
|
|
dir="$( sed -r -e 's:/+:/:g; s:/$::;' <<<"${dir}" )"
|
|
[ "${dir}" = "${hostdir}/lib" ] && return 0
|
|
[ "${dir}" = "\$ORIGIN/../lib" ] && return 0
|
|
# This check is done even for builds where
|
|
# BR2_PER_PACKAGE_DIRECTORIES is disabled. In this case,
|
|
# PER_PACKAGE_DIR and therefore ${perpackagedir} points to
|
|
# a non-existent directory, and this check will always be
|
|
# false.
|
|
[[ ${dir} =~ ${perpackagedir}/[^/]+/host/lib ]] && return 0
|
|
done
|
|
done < <( readelf -d "${file}" \
|
|
|sed -r -e '/.* \(R(UN)?PATH\) +Library r(un)?path: \[(.+)\]$/!d' \
|
|
-e 's//\3/;' \
|
|
)
|
|
|
|
return 1
|
|
}
|
|
|
|
main "${@}"
|