2013-06-05 17:53:30 -06:00
|
|
|
################################################################################
|
2012-04-17 08:45:20 -06:00
|
|
|
#
|
|
|
|
|
# This file contains the download helpers for the various package
|
|
|
|
|
# infrastructures. It is used to handle downloads from HTTP servers,
|
|
|
|
|
# FTP servers, Git repositories, Subversion repositories, Mercurial
|
|
|
|
|
# repositories, Bazaar repositories, and SCP servers.
|
|
|
|
|
#
|
2013-06-05 17:53:30 -06:00
|
|
|
################################################################################
|
2012-04-17 08:45:20 -06:00
|
|
|
|
2012-04-17 08:45:19 -06:00
|
|
|
# Download method commands
|
2015-01-02 08:53:41 -07:00
|
|
|
export WGET := $(call qstrip,$(BR2_WGET))
|
2014-07-02 15:11:20 -06:00
|
|
|
export SVN := $(call qstrip,$(BR2_SVN))
|
2014-07-02 15:11:21 -06:00
|
|
|
export CVS := $(call qstrip,$(BR2_CVS))
|
2014-07-02 15:11:24 -06:00
|
|
|
export BZR := $(call qstrip,$(BR2_BZR))
|
2014-07-02 15:11:19 -06:00
|
|
|
export GIT := $(call qstrip,$(BR2_GIT))
|
2015-01-02 08:53:41 -07:00
|
|
|
export HG := $(call qstrip,$(BR2_HG))
|
|
|
|
|
export SCP := $(call qstrip,$(BR2_SCP))
|
2020-04-15 10:48:45 -06:00
|
|
|
export SFTP := $(call qstrip,$(BR2_SFTP))
|
2014-07-02 15:11:26 -06:00
|
|
|
export LOCALFILES := $(call qstrip,$(BR2_LOCALFILES))
|
2012-04-17 08:45:19 -06:00
|
|
|
|
support/download: change format of archives generated from git
Switch to using the tarball helper, that can generate reproducible
archives whatever the tar version >= 1.27.
However, those archives are not identical to the previous ones generated
in the (now-broken) gnu format.
To avoid any clashing between old and new archives, and new and old
Buildroot versions, we need to name the new generated archives
differently from the existing ones.
So, we bump the git-specific format-version to -br1.
The %ci date has been supported by git back to 1.6.0, released August
2008); it is not strictly ISO8601, but is still accepted as a PAX date
header. The strict ISO8601 placeholder, %cI, was only introduced with
2.2.0, release in November 2014, so too recent to be widely available.
As the format and the names of the archives changes, we need to update
all the hash files with the new names and hashes.
Of all the bootloaders that have a git download method, vexpress-firmware
is the only one to have a hash. Others have no hash files, or they have
explicitly set BR_NO_CHECK_HASH_FOR.
For the packages, linux-headers is the special snowflake, as the git
download is only for custom git tree, so it is excluded from the hash
verification with BR_NO_CHECK_HASH_FOR.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vincent Fazio <vfazio@xes-inc.com>
---8<------8<------8<------8<---
#!/bin/sh
# Find and download all packages using git as backend.
# Manually fix hashes for affected packages.
# Packages that only have a host variant
HOST_ONLY='imx-mkimage|mxsldr|netsurf-buildsystem|opkg-utils|prelink-cross|qoriq-rcw|vboot-utils'
# Packages that have a non-git main _SOURCE, and/or which
# have BR_NO_CHECK_HASH_FOR for the git _SOURCE
NOT_GIT='aufs|aufs-util|xenomai|linux-headers'
export BR2_DL_DIR=$(pwd)/temp-dl-dir
make defconfig
make $( git grep -l -E 'SITE_METHOD[[:space:]]*:?=[[:space:]]*git\>|_SITE[[:space:]]*:?=[[:space:]]*git:' \
boot/vexpress-firmware/ package/ \
|sed -r -e 's,.*/([^/]+)\.mk,\1,' \
|sed -r -e '/^('"${NOT_GIT}"')$/d;' \
-e 's/^('"${HOST_ONLY}"')/host-\1/;' \
-e 's/$/-legal-info/;'
)
---8<------8<------8<------8<---
2019-03-25 15:48:12 -06:00
|
|
|
# Version of the format of the archives we generate in the corresponding
|
|
|
|
|
# download backend:
|
|
|
|
|
BR_FMT_VERSION_git = -br1
|
package/pkg-generic: don't download svn externals by default
Commit 7dd27cbe5b9e (support/download: add support to exclude svn
externals) departed from the usual opt-in scheme, like is done for
git submodule or large files, in an attempt to keep the previous
behaviour unchanged, that is to download externals by default.
As an afterthought, we've concluded that the chances for svn-hosted
packages with externals that are indeed required to do the build,
are relatively slim. For those cases, it even makes sense to explicitly
requested the use of the externals.
So, we change the default to not download svn externals.
Since the generated archives may change, we bump the version suffix.
This will allow users to more easily catch the situation and decide if
they really need the externals or not.
We have a single in-tree package that uses svn, and it does not use
externals, so the generated archive does not change, and we just need
to update the archive filename in the hash file.
Finally, we add a new section to the manual, in the chapter about
migrating Buildroot to a newer version.
Reported-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2023-08-08 07:17:41 -06:00
|
|
|
BR_FMT_VERSION_svn = -br3
|
support/download: change format of archives generated from git
Switch to using the tarball helper, that can generate reproducible
archives whatever the tar version >= 1.27.
However, those archives are not identical to the previous ones generated
in the (now-broken) gnu format.
To avoid any clashing between old and new archives, and new and old
Buildroot versions, we need to name the new generated archives
differently from the existing ones.
So, we bump the git-specific format-version to -br1.
The %ci date has been supported by git back to 1.6.0, released August
2008); it is not strictly ISO8601, but is still accepted as a PAX date
header. The strict ISO8601 placeholder, %cI, was only introduced with
2.2.0, release in November 2014, so too recent to be widely available.
As the format and the names of the archives changes, we need to update
all the hash files with the new names and hashes.
Of all the bootloaders that have a git download method, vexpress-firmware
is the only one to have a hash. Others have no hash files, or they have
explicitly set BR_NO_CHECK_HASH_FOR.
For the packages, linux-headers is the special snowflake, as the git
download is only for custom git tree, so it is excluded from the hash
verification with BR_NO_CHECK_HASH_FOR.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vincent Fazio <vfazio@xes-inc.com>
---8<------8<------8<------8<---
#!/bin/sh
# Find and download all packages using git as backend.
# Manually fix hashes for affected packages.
# Packages that only have a host variant
HOST_ONLY='imx-mkimage|mxsldr|netsurf-buildsystem|opkg-utils|prelink-cross|qoriq-rcw|vboot-utils'
# Packages that have a non-git main _SOURCE, and/or which
# have BR_NO_CHECK_HASH_FOR for the git _SOURCE
NOT_GIT='aufs|aufs-util|xenomai|linux-headers'
export BR2_DL_DIR=$(pwd)/temp-dl-dir
make defconfig
make $( git grep -l -E 'SITE_METHOD[[:space:]]*:?=[[:space:]]*git\>|_SITE[[:space:]]*:?=[[:space:]]*git:' \
boot/vexpress-firmware/ package/ \
|sed -r -e 's,.*/([^/]+)\.mk,\1,' \
|sed -r -e '/^('"${NOT_GIT}"')$/d;' \
-e 's/^('"${HOST_ONLY}"')/host-\1/;' \
-e 's/$/-legal-info/;'
)
---8<------8<------8<------8<---
2019-03-25 15:48:12 -06:00
|
|
|
|
2014-12-11 15:52:05 -07:00
|
|
|
DL_WRAPPER = support/download/dl-wrapper
|
|
|
|
|
|
2014-02-04 08:18:51 -07:00
|
|
|
# DL_DIR may have been set already from the environment
|
2014-02-10 14:48:55 -07:00
|
|
|
ifeq ($(origin DL_DIR),undefined)
|
2014-02-04 08:18:51 -07:00
|
|
|
DL_DIR ?= $(call qstrip,$(BR2_DL_DIR))
|
2012-04-17 08:45:19 -06:00
|
|
|
ifeq ($(DL_DIR),)
|
2013-07-20 00:52:43 -06:00
|
|
|
DL_DIR := $(TOPDIR)/dl
|
2012-04-17 08:45:19 -06:00
|
|
|
endif
|
2014-02-10 14:48:55 -07:00
|
|
|
else
|
|
|
|
|
# Restore the BR2_DL_DIR that was overridden by the .config file
|
|
|
|
|
BR2_DL_DIR = $(DL_DIR)
|
|
|
|
|
endif
|
2012-04-17 08:45:19 -06:00
|
|
|
|
core: ensure we use the realpath(3) of DL_DIR
When $(TOPDIR)/dl is a symlink, checking out git submodules can fail,
as reported by Michael in #11086.
To reproduce a similarly-related mis-behaviour:
$ mkdir -p foo/bar foo/buz
$ cd foo/bar
$ ln -s ../buz meh
$ cd meh
$ cd ../../foo
The last command should not succeed, because, relative to meh, there is
no ../../foo directory; we would expect it to be ../../../foo, instead.
But since meh is a symlink to a directory, then a relative path from that
symlink is interpreted as relative to the derefrenced directory, i.e.
from buz in this case.
But where this gets even weirder, is that, if the last command is
replaced by:
$ cd ../../../foo
then it still works, too.
And that is the root of Michael's issue: the dl directory in Buildroot's
TOPDIR is a symlink to a similarly-named directory one directory higher,
which then confuses relative paths, which gets especially and noticeably
bad for git submodules.
Avoid this strangeness, and just use so-called "physical" path, i.e. a
path where all symlinks to directories have been dereferenced.
Fixes: #11086
Reported-by: Michael Nosthoff <posted@heine.so>
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Michael Nosthoff <posted@heine.so>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2018-11-15 10:03:48 -07:00
|
|
|
# ensure it exists and a absolute path, derefrecing symlinks
|
|
|
|
|
DL_DIR := $(shell mkdir -p $(DL_DIR) && cd $(DL_DIR) >/dev/null && pwd -P)
|
2012-09-10 04:38:29 -06:00
|
|
|
|
2012-04-17 08:45:19 -06:00
|
|
|
#
|
|
|
|
|
# URI scheme helper functions
|
|
|
|
|
# Example URIs:
|
|
|
|
|
# * http://www.example.com/dir/file
|
|
|
|
|
# * scp://www.example.com:dir/file (with domainseparator :)
|
|
|
|
|
#
|
|
|
|
|
# geturischeme: http
|
.mk files: bulk aligment and whitespace cleanup of assignments
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-07 01:06:03 -06:00
|
|
|
geturischeme = $(firstword $(subst ://, ,$(call qstrip,$(1))))
|
2018-04-02 02:14:23 -06:00
|
|
|
# getschemeplusuri: git|parameter+http://example.com
|
|
|
|
|
getschemeplusuri = $(call geturischeme,$(1))$(if $(2),\|$(2))+$(1)
|
2012-04-17 08:45:19 -06:00
|
|
|
# stripurischeme: www.example.com/dir/file
|
.mk files: bulk aligment and whitespace cleanup of assignments
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-07 01:06:03 -06:00
|
|
|
stripurischeme = $(lastword $(subst ://, ,$(call qstrip,$(1))))
|
2012-04-17 08:45:19 -06:00
|
|
|
# domain: www.example.com
|
.mk files: bulk aligment and whitespace cleanup of assignments
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-07 01:06:03 -06:00
|
|
|
domain = $(firstword $(subst $(call domainseparator,$(2)), ,$(call stripurischeme,$(1))))
|
2012-04-17 08:45:19 -06:00
|
|
|
# notdomain: dir/file
|
.mk files: bulk aligment and whitespace cleanup of assignments
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-07 01:06:03 -06:00
|
|
|
notdomain = $(patsubst $(call domain,$(1),$(2))$(call domainseparator,$(2))%,%,$(call stripurischeme,$(1)))
|
2012-04-17 08:45:19 -06:00
|
|
|
#
|
|
|
|
|
# default domainseparator is /, specify alternative value as first argument
|
.mk files: bulk aligment and whitespace cleanup of assignments
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2014-10-07 01:06:03 -06:00
|
|
|
domainseparator = $(if $(1),$(1),/)
|
2012-04-17 08:45:19 -06:00
|
|
|
|
Revert "pkg-download: remove explicit PKG_VERSION from github helper"
This reverts commit 1e5a8916b2ab4c9c99548fa6fbd4855eee323881.
The idea was that the version string can be derived because we know the
package name.
However, this patch does not account for the fact that $(pkgname) always
points to the latest pacakge scanned, which in all other situation we're
using it, is the current package, because it is used inside one ot he
xxx-inner macros that are $(eval)ualed. So $(pkgname) is evaluated
"early" and gets the expected value.
However, the github value is not in one of those macros, so it gets
evaluated "late", when doing the actual download. So, by that time,
$(pkgname) will expand to the last package scanned, which is actuall the
manual (without a br2-external tree).
That would require that the _SITE variable be assigned with the :=
assignment operator. This is weird, because that would make it the only
variable to require that, but only when using the github helper, which
is even less obvious and would cause a lot of trouble...
The obvious fixup would seem to be to use $(PKG) instead, because that
already contains the upper-case package name that vcan be used as a
prefix to variables.
However, that does not work either, because we have a check that forbids
a trsailing slash in _SITE, check that is done in pacakge/pkg-generic,
inside the xxx-inner macro, during the $(eval) call.
And at that time, PKG is not yet defined, because it is only defined for
an actual recipe.
So we can't seem to have a workable solution. So, just revert the patch.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Marcin Nowakowski <marcin.nowakowski@imgtec.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2017-11-13 15:00:09 -07:00
|
|
|
# github(user,package,version): returns site of GitHub repository
|
|
|
|
|
github = https://github.com/$(1)/$(2)/archive/$(3)
|
2013-12-05 10:20:44 -07:00
|
|
|
|
2021-03-28 13:13:49 -06:00
|
|
|
# gitlab(user,package,version): returns site of Gitlab-generated tarball
|
|
|
|
|
gitlab = https://gitlab.com/$(1)/$(2)/-/archive/$(3)
|
|
|
|
|
|
support/download: add possibility to not fail on missing hash
In very constrained cases, it might be needed to not fail if a hash is
missing. This is notably the case for custom external toolchains to be
downloaded, because we do have a .hash file for external toolchains,
but we obviously can not have hashes for all existing custom toolchains
(he, "custom"!).
So, add a way to avoid failing in that case.
>From the Makefile, we export the list of files for which not to check
the hash. Then, from the check-hash script, if no check was done, and
the file we were trying to match in in this exclusion list, we just exit
without error.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
changes v6 -> v7:
- /beautify/ the pattern in the case clause
Changed v5 -> v6: (Arnout)
- fix the pattern in the case clause
Changes v4 -> v5:
- micro-optimisation, use case-esac instead of a for-loop (Arnout)
- typoes (Arnout)
Changes v3 -> v4:
- drop the magic value, use a list of excluded files (Arnout)
Changes v1 -> v2:
- fix typoes in commit log
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-22 16:08:38 -06:00
|
|
|
# Expressly do not check hashes for those files
|
2015-05-02 03:05:00 -06:00
|
|
|
# Exported variables default to immediately expanded in some versions of
|
|
|
|
|
# make, but we need it to be recursively-epxanded, so explicitly assign it.
|
|
|
|
|
export BR_NO_CHECK_HASH_FOR =
|
support/download: add possibility to not fail on missing hash
In very constrained cases, it might be needed to not fail if a hash is
missing. This is notably the case for custom external toolchains to be
downloaded, because we do have a .hash file for external toolchains,
but we obviously can not have hashes for all existing custom toolchains
(he, "custom"!).
So, add a way to avoid failing in that case.
>From the Makefile, we export the list of files for which not to check
the hash. Then, from the check-hash script, if no check was done, and
the file we were trying to match in in this exclusion list, we just exit
without error.
Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Gustavo Zacarias <gustavo@zacarias.com.ar>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
changes v6 -> v7:
- /beautify/ the pattern in the case clause
Changed v5 -> v6: (Arnout)
- fix the pattern in the case clause
Changes v4 -> v5:
- micro-optimisation, use case-esac instead of a for-loop (Arnout)
- typoes (Arnout)
Changes v3 -> v4:
- drop the magic value, use a list of excluded files (Arnout)
Changes v1 -> v2:
- fix typoes in commit log
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
2015-04-22 16:08:38 -06:00
|
|
|
|
2012-04-17 08:45:19 -06:00
|
|
|
################################################################################
|
2019-04-15 13:47:24 -06:00
|
|
|
# DOWNLOAD_URIS - List the candidates URIs where to get the package from:
|
2018-04-02 02:14:23 -06:00
|
|
|
# 1) BR2_PRIMARY_SITE if enabled
|
|
|
|
|
# 2) Download site, unless BR2_PRIMARY_SITE_ONLY is set
|
|
|
|
|
# 3) BR2_BACKUP_SITE if enabled, unless BR2_PRIMARY_SITE_ONLY is set
|
|
|
|
|
#
|
|
|
|
|
# Argument 1 is the source location
|
2019-04-15 13:47:24 -06:00
|
|
|
# Argument 2 is the upper-case package name
|
2012-04-17 08:45:19 -06:00
|
|
|
#
|
|
|
|
|
################################################################################
|
|
|
|
|
|
2018-04-02 02:14:23 -06:00
|
|
|
ifneq ($(call qstrip,$(BR2_PRIMARY_SITE)),)
|
|
|
|
|
DOWNLOAD_URIS += \
|
2019-04-15 13:47:24 -06:00
|
|
|
$(call getschemeplusuri,$(call qstrip,$(BR2_PRIMARY_SITE)/$($(2)_DL_SUBDIR)),urlencode) \
|
2019-04-15 13:47:23 -06:00
|
|
|
$(call getschemeplusuri,$(call qstrip,$(BR2_PRIMARY_SITE)),urlencode)
|
2018-04-02 02:14:23 -06:00
|
|
|
endif
|
2012-04-17 08:45:19 -06:00
|
|
|
|
2018-04-02 02:14:23 -06:00
|
|
|
ifeq ($(BR2_PRIMARY_SITE_ONLY),)
|
|
|
|
|
DOWNLOAD_URIS += \
|
2019-04-15 13:47:23 -06:00
|
|
|
$(patsubst %/,%,$(dir $(call qstrip,$(1))))
|
2018-04-02 02:14:23 -06:00
|
|
|
ifneq ($(call qstrip,$(BR2_BACKUP_SITE)),)
|
|
|
|
|
DOWNLOAD_URIS += \
|
2019-04-15 13:47:24 -06:00
|
|
|
$(call getschemeplusuri,$(call qstrip,$(BR2_BACKUP_SITE)/$($(2)_DL_SUBDIR)),urlencode) \
|
2019-04-15 13:47:23 -06:00
|
|
|
$(call getschemeplusuri,$(call qstrip,$(BR2_BACKUP_SITE)),urlencode)
|
2018-04-02 02:14:23 -06:00
|
|
|
endif
|
|
|
|
|
endif
|
2013-09-11 06:12:04 -06:00
|
|
|
|
2019-04-15 13:47:24 -06:00
|
|
|
################################################################################
|
|
|
|
|
# DOWNLOAD -- Download helper. Will call DL_WRAPPER which will try to download
|
|
|
|
|
# source from the list returned by DOWNLOAD_URIS.
|
|
|
|
|
#
|
|
|
|
|
# Argument 1 is the source location
|
2019-04-15 13:47:26 -06:00
|
|
|
# Argument 2 is the upper-case package name
|
package/pkg-download: do not try to vendor _EXTRA_DOWNLOADS
For golang- or cargo-based packages, we apply a vendoring pass after the
package's "main" download is done. Whether to vendor or not is based on
the heuristic that a specific directory exists or not; for golang
packages, we look for '/vendor', while for cargo, we look for '/VENDOR'.
This is fine for the "main" (by lack of a better term) download, but
this falls flat on its face for extra downloads. Indeed, some packages
may need to download data sets, or assets, as _EXTRA_DOWNLOADS. Those
are usually just data blobs, and are not actual golang or cargo packages;
as such they do not need to be vendored, but worse, if we try to
actually vendor them, this fails because the required files for
vendoring are missing from the archives in such data sets.
We fix that by decoupling the download for the extra download, from the
download for the main archive. We pass the post-processing option only
to the main download.
This makes the hard assumption that extra downloads will never need to
be post-processed for vendoring, of course; we hope this will always be
correct in practice. If not, we can fix it later.
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Arnout:
- no loop needed for MAIN_DOWNLOAD, it can have only one;
- remove superfluous backslash in the definition of MAIN_DOWNLOAD;
- introduce _ADDITIONAL_DOWNLOADS to avoid filter-out.]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-05 11:19:39 -06:00
|
|
|
# Argument 3 is a space-separated list of optional arguments
|
2019-04-15 13:47:24 -06:00
|
|
|
#
|
|
|
|
|
################################################################################
|
|
|
|
|
|
2018-04-02 02:14:23 -06:00
|
|
|
define DOWNLOAD
|
2019-04-15 13:47:26 -06:00
|
|
|
$(Q)mkdir -p $($(2)_DL_DIR)
|
2020-12-19 08:35:16 -07:00
|
|
|
$(Q)$(EXTRA_ENV) $($(2)_DL_ENV) \
|
|
|
|
|
flock $($(2)_DL_DIR)/.lock $(DL_WRAPPER) \
|
2019-04-15 13:47:26 -06:00
|
|
|
-c '$($(2)_DL_VERSION)' \
|
|
|
|
|
-d '$($(2)_DL_DIR)' \
|
2018-04-02 09:13:58 -06:00
|
|
|
-D '$(DL_DIR)' \
|
2018-04-02 07:33:53 -06:00
|
|
|
-f '$(notdir $(1))' \
|
2019-04-15 13:47:26 -06:00
|
|
|
-H '$($(2)_HASH_FILE)' \
|
|
|
|
|
-n '$($(2)_BASENAME_RAW)' \
|
|
|
|
|
-N '$($(2)_RAWNAME)' \
|
|
|
|
|
-o '$($(2)_DL_DIR)/$(notdir $(1))' \
|
support/download: add support to exclude svn externals
Like git which can have submodules, subversion can have externals. The
default behaviour for subversion is to retrieve all the externals,
unless told otherwise.
For some repositories, the externals may be huge (e.g. a dataset or some
assets) and may not be required for building the package. In such a
case, retrieving the externals is both a waste of network bandwitdh and
time, and a waste of disk storage.
Like for git submodules and git lfs, add an option that packages can set
to specify whether they want externals or not.
Since we've so far been retrieving externals, we keep that the default,
and packages can opt-out (rather than the opt-in for git submodules or
git lfs).
We must only set it when the package is actually hosted on svn, to avoid
passing -r when the package is not hosted by svn; otherwise, -r would
also be passed e.g. to a git-hosted package, triggering the download of
git submodules even when they are not requested. We need to do so,
because we have a default value, which we usually do not have in other
download options.
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2023-08-01 07:11:17 -06:00
|
|
|
$(if $(filter YES,$($(2)_SVN_EXTERNALS)),-r) \
|
2019-04-15 13:47:26 -06:00
|
|
|
$(if $($(2)_GIT_SUBMODULES),-r) \
|
2020-01-21 13:33:37 -07:00
|
|
|
$(if $($(2)_GIT_LFS),-l) \
|
2019-04-15 13:47:26 -06:00
|
|
|
$(foreach uri,$(call DOWNLOAD_URIS,$(1),$(2)),-u $(uri)) \
|
package/pkg-download: do not try to vendor _EXTRA_DOWNLOADS
For golang- or cargo-based packages, we apply a vendoring pass after the
package's "main" download is done. Whether to vendor or not is based on
the heuristic that a specific directory exists or not; for golang
packages, we look for '/vendor', while for cargo, we look for '/VENDOR'.
This is fine for the "main" (by lack of a better term) download, but
this falls flat on its face for extra downloads. Indeed, some packages
may need to download data sets, or assets, as _EXTRA_DOWNLOADS. Those
are usually just data blobs, and are not actual golang or cargo packages;
as such they do not need to be vendored, but worse, if we try to
actually vendor them, this fails because the required files for
vendoring are missing from the archives in such data sets.
We fix that by decoupling the download for the extra download, from the
download for the main archive. We pass the post-processing option only
to the main download.
This makes the hard assumption that extra downloads will never need to
be post-processed for vendoring, of course; we hope this will always be
correct in practice. If not, we can fix it later.
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[Arnout:
- no loop needed for MAIN_DOWNLOAD, it can have only one;
- remove superfluous backslash in the definition of MAIN_DOWNLOAD;
- introduce _ADDITIONAL_DOWNLOADS to avoid filter-out.]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2022-05-05 11:19:39 -06:00
|
|
|
$(3) \
|
2015-01-02 08:53:40 -07:00
|
|
|
$(QUIET) \
|
2014-12-11 15:52:05 -07:00
|
|
|
-- \
|
2019-04-15 13:47:26 -06:00
|
|
|
$($(2)_DL_OPTS)
|
2012-04-17 08:45:19 -06:00
|
|
|
endef
|