We can re-enable some dtc warnings that have been completely or mostly
fixed. There are a few remaining ones in arm64 dts files which crept in
recently.
Signed-off-by: Rob Herring <robh@kernel.org>
If dtc is rebuilt, we should rebuild .dtb files with the new dtc.
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Commit eea199b445 ("kbuild: remove unnecessary LEX_PREFIX and
YACC_PREFIX") removed YACC_PREFIX definition, but left one use of it. There
was not any build error since there is no user of "cmd_bison_h" currently.
Remove the last remaining occurrence of YACC_PREFIX.
Fixes: eea199b445 ("kbuild: remove unnecessary LEX_PREFIX and YACC_PREFIX")
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rob Herring <robh@kernel.org>
With KASAN enabled the kernel has two different memset() functions, one
with KASAN checks (memset) and one without (__memset). KASAN uses some
macro tricks to use the proper version where required. For example
memset() calls in mm/slub.c are without KASAN checks, since they operate
on poisoned slab object metadata.
The issue is that clang emits memset() calls even when there is no
memset() in the source code. They get linked with improper memset()
implementation and the kernel fails to boot due to a huge amount of KASAN
reports during early boot stages.
The solution is to add -fno-builtin flag for files with KASAN_SANITIZE :=
n marker.
Link: http://lkml.kernel.org/r/8ffecfffe04088c52c42b92739c2bd8a0bcb3f5e.1516384594.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Acked-by: Nick Desaulniers <ndesaulniers@google.com>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
A pretty big batch of Kconfig updates. I have to mention the lexer
and parser of Kconfig are now built from real .l and .y sources.
So, flex and bison are the requirement for building the kernel.
Both of them (unlike gperf) have been stable for a long time. This
change has been tested several weeks in linux-next, and I did not
receive any problem report about this.
Summary:
- Add checks for mistakes, like the choice default is not in
choice, help is doubled
- Document data structure and complex code
- Fix various memory leaks
- Change Makefile to build lexer and parser instead of using
pre-generated C files
- Drop 'boolean' keyword, which is equivalent to 'bool'
- Use default 'yy' prefix and remove unneeded Make variables
- Fix gettext() check for xconfig
- Announce that oldnoconfig will be finally removed
- Make 'Selected by:' and 'Implied by' readable in help and
search result
- Hide silentoldconfig from 'make help' to stop confusing people
- Fix misc things and cleanups
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJac0O7AAoJED2LAQed4NsGxRQQAKiDkBmJUKzJykEr3BhEsCW1
1cKwsaCFSKQJZde+Edn14gqBQY3qvQVaWfnQpoBC69IOlMlVQyKqqdtjIUxmGTem
+age94JvgPT3oj7ELigUsL5bFL8CEZYpjwAVkO+Zd+7jPxlM4glNl9F3coTp9ZAl
kaFxnch8qPT3fb6xYSwTpkk28RFNfT8ixwImj0CH6wlwxI635o9wpqAmHCMD5Yid
wBlEB0I0aw9xVx/D2FAh1ZJ+fyOtPLotRTtUQ1kdlDyk98V6EJRfyh1XWF0xE/gH
WR7MAuBNXC6Uu1KqXLTJRWWE4NjBmW8e7OLTaoFQFXdnz/Rgd6gbPXuiiyKR6PUc
mA3h6F8uPDpoRpqOrn6pdu269n1ObuZuC1XZ3MVSXtW3OzLbUk/2GpHPVX8mrep5
SBGsfq4JYtQkgz9/wBrWpdAjl2QbtPJbM/izCbEWk7pod18dVnyHpo3i3YjsaSQf
piDK5JGKsHEisAzp6Onhc9EPeBLAjG236+LvBruaqjJNySXyT0S2kOwV0AMOD1yC
ave8ZQA0QxP7sV/s5DXGkFx5Nt/1LE8JWvcRN0+juavnkUklWfFfnk7epAnFfRPi
HCGfeQSzIQxQ+JV7/vlJ95FWvLZz+KzQ49X5aoYCVcEiFf2MV7RkDHSp5mPE4DnO
5JeHnTBpuvw4mnrZ6UNj
=rsi/
-----END PGP SIGNATURE-----
Merge tag 'kconfig-v4.16' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kconfig updates from Masahiro Yamada:
"A pretty big batch of Kconfig updates.
I have to mention the lexer and parser of Kconfig are now built from
real .l and .y sources. So, flex and bison are the requirement for
building the kernel. Both of them (unlike gperf) have been stable for
a long time. This change has been tested several weeks in linux-next,
and I did not receive any problem report about this.
Summary:
- add checks for mistakes, like the choice default is not in choice,
help is doubled
- document data structure and complex code
- fix various memory leaks
- change Makefile to build lexer and parser instead of using
pre-generated C files
- drop 'boolean' keyword, which is equivalent to 'bool'
- use default 'yy' prefix and remove unneeded Make variables
- fix gettext() check for xconfig
- announce that oldnoconfig will be finally removed
- make 'Selected by:' and 'Implied by' readable in help and search
result
- hide silentoldconfig from 'make help' to stop confusing people
- fix misc things and cleanups"
* tag 'kconfig-v4.16' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (37 commits)
kconfig: Remove silentoldconfig from help and docs; fix kconfig/conf's help
kconfig: make "Selected by:" and "Implied by:" readable
kconfig: announce removal of oldnoconfig if used
kconfig: fix make xconfig when gettext is missing
kconfig: Clarify menu and 'if' dependency propagation
kconfig: Document 'if' flattening logic
kconfig: Clarify choice dependency propagation
kconfig: Document SYMBOL_OPTIONAL logic
kbuild: remove unnecessary LEX_PREFIX and YACC_PREFIX
kconfig: use default 'yy' prefix for lexer and parser
kconfig: make conf_unsaved a local variable of conf_read()
kconfig: make xfgets() really static
kconfig: make input_mode static
kconfig: Warn if there is more than one help text
kconfig: drop 'boolean' keyword
kconfig: use bool instead of boolean for type definition attributes, again
kconfig: Remove menu_end_entry()
kconfig: Document important expression functions
kconfig: Document automatic submenu creation code
kconfig: Fix choice symbol expression leak
...
Kbuild supports 3 levels of extra warnings, and multiple levels can
be combined, like W=12, W=123. It was added by commit a6de553da0
("kbuild: Allow to combine multiple W= levels").
From the log of commit 8654cb8d03 ("dtc: update warning settings
for new bus and node/property name checks"), I assume:
- unit_address_vs_reg, simple_bus_reg, etc. belong to level 1
- node_name_chars_strict, property_name_chars_strict belong to level 2
However, the level 1 warnings are displayed by any argument to W=.
On the other hand, the level 2 warnings are displayed by W=2, but
not by W=12, or W=123.
Use $(findstring ...) like scripts/Makefile.extrawarn.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Kconfig was the only user of these. With Kconfig converted to use
the default 'yy' prefix, we do not need them any more.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Ulf Magnusson <ulfalizer@gmail.com>
In Linux build system convention, pre-generated files are version-
controlled with a "_shipped" suffix. During the kernel building,
they are simply shipped (copied) removing the suffix.
This approach can reduce external tool dependency for the kernel build,
but it is tedious to manually regenerate such artifacts from developers'
point of view. (We need to do "make REGENERATE_PARSERS=1" every time
we touch real source files such as *.l, *.y)
Some months ago, I sent out RFC patches to run flex, bison, and gperf
during the build.
In the review and test, Linus noticed gperf-3.1 had changed the lookup
function prototype. Then, the use of gperf in kernel was entirely
removed by commit bb3290d916 ("Remove gperf usage from toolchain").
This time, I tested several versions of flex and bison, and I was not
hit by any compatibility issue except a flaw in flex-2.6.3; if you
generate lexer for dtc and genksyms with flex-2.6.3, you will see
"yywrap redefined" warning. This was not intentional, but a bug,
fixed by flex-2.6.4. Otherwise, both flex and bison look fairly
stable for a long time.
This commit prepares some build rules to remove the _shipped files.
Also, document minimal requirement for flex and bison.
Rationale for the minimal version:
The -Wmissing-prototypes option of GCC warns "no previous prototype"
for lexers generated by flex-2.5.34 or older, so I chose 2.5.35 as the
required version for flex. Flex-2.5.35 was released in 2008. Bison
looks more stable. I did not see any problem with bison-2.0, released
in 2004. I did not test bison-1.x, but bison-2.0 should be old enough.
Tested flex versions:
2.5.35
2.5.36
2.5.37
2.5.39
2.6.0
2.6.1
2.6.2
2.6.3 (*)
2.6.4
(*) flex-2.6.3 causes "yywrap redefined" warning
Tested bison versions:
2.0
2.1
2.2
2.3
2.4
2.4.1
2.5.1
2.6
2.6.1
2.6.2
2.6.3
2.6.4
2.6.5
2.7
2.7.1
3.0
3.0.1
3.0.2
3.0.3
3.0.4
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Allow users to use their favorite lexer / parser generators.
This is useful for me to test various flex and bison versions.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
$(real-objs-y) in only used in scripts/Makefile.build to form
"targets", but $(extra-y) is added to "targets" in another line.
We do not need to add $(extra-y) twice.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
One of the most remarkable improvements in this cycle is, Kbuild is
now able to cache the result of shell commands. Some variables are
expensive to compute, for example, $(call cc-option,...) invokes the
compiler. It is not efficient to redo this computation every time,
even when we are not actually building anything. Kbuild creates a
hidden file ".cache.mk" that contains invoked shell commands and
their results. The speed-up should be noticeable.
Summary:
- Fix arch build issues (hexagon, sh)
- Clean up various Makefiles and scripts
- Fix wrong usage of {CFLAGS,LDFLAGS}_MODULE in arch Makefiles
- Cache variables that are expensive to compute
- Improve cc-ldopton and ld-option for Clang
- Optimize output directory creation
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJaDxaLAAoJED2LAQed4NsGIHQP/isMxxaIxIAWU56+ZcII74k7
639VgrKi9n5y25d1dBRTQg+vReHE6E2JbkCqpVOu11t7m0LT7yUK8v3WwyLf1qTN
GxnqZ/WMQU5/AYVqIWo8jN4FGHpivHJ6qbeiNJM9qN4RAkzG0sZUq746VaFZYmIR
Lu0Gf4m4qjifkkhXsQdWT5i7yNTidPqaL6GNb+FcFkEHlVre8jma0kJlgfHxru84
WmETpjQXvHAZ/R61vY6ekAWpqFhw3ecJY96A9npnx+SQVQdSNAdpaU0SK29jB0ON
/SAfpHg9oa/gD0LFOKV6zkjnAkd4TEjrJEiHHhz5gjT/SbS3T1llBIGZ1oV4X7Y0
Vlh9KWlm1FJJI4SIzc9qUaQMp6JtLfEfHKJCc45xVaN3fNrDnR8jl80x5+95ELga
dCkZgnq5u82MtTysCbHBESwDYQaVPyIrh7In+mduglaCqhqj9KoDjoLoiGfCg7SA
3tPflYVd629w5l5GrazJ40jWn1+ggMtgMOVooJNJ+dINCP+GxsUpH84Ww2Pdic+/
qLdud6TeqxrZDGzWXqKNLu8alM8NGgSr101l9gIf1oqSyy63duBpMrxGDoIJS3FU
rFDoFFUhlfkAXNbQHtVGNzKtcpCjURh992j9Fa1+NfMwSce5IHkMwTvPmNSRowi8
0llLjXhD/bxK6FpdvlV8
=zIdO
-----END PGP SIGNATURE-----
Merge tag 'kbuild-v4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild updates from Masahiro Yamada:
"One of the most remarkable improvements in this cycle is, Kbuild is
now able to cache the result of shell commands. Some variables are
expensive to compute, for example, $(call cc-option,...) invokes the
compiler. It is not efficient to redo this computation every time,
even when we are not actually building anything. Kbuild creates a
hidden file ".cache.mk" that contains invoked shell commands and their
results. The speed-up should be noticeable.
Summary:
- Fix arch build issues (hexagon, sh)
- Clean up various Makefiles and scripts
- Fix wrong usage of {CFLAGS,LDFLAGS}_MODULE in arch Makefiles
- Cache variables that are expensive to compute
- Improve cc-ldopton and ld-option for Clang
- Optimize output directory creation"
* tag 'kbuild-v4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (30 commits)
kbuild: move coccicheck help from scripts/Makefile.help to top Makefile
sh: decompressor: add shipped files to .gitignore
frv: .gitignore: ignore vmlinux.lds
selinux: remove unnecessary assignment to subdir-
kbuild: specify FORCE in Makefile.headersinst as .PHONY target
kbuild: remove redundant mkdir from ./Kbuild
kbuild: optimize object directory creation for incremental build
kbuild: create object directories simpler and faster
kbuild: filter-out PHONY targets from "targets"
kbuild: remove redundant $(wildcard ...) for cmd_files calculation
kbuild: create directory for make cache only when necessary
sh: select KBUILD_DEFCONFIG depending on ARCH
kbuild: fix linker feature test macros when cross compiling with Clang
kbuild: shrink .cache.mk when it exceeds 1000 lines
kbuild: do not call cc-option before KBUILD_CFLAGS initialization
kbuild: Cache a few more calls to the compiler
kbuild: Add a cache for generated variables
kbuild: add forward declaration of default target to Makefile.asm-generic
kbuild: remove KBUILD_SUBDIR_ASFLAGS and KBUILD_SUBDIR_CCFLAGS
hexagon/kbuild: replace CFLAGS_MODULE with KBUILD_CFLAGS_MODULE
...
For the out-of-tree build, scripts/Makefile.build creates output
directories, but this operation is not efficient.
scripts/Makefile.lib calculates obj-dirs as follows:
obj-dirs := $(dir $(multi-objs) $(obj-y))
Please notice $(sort ...) is not used here. Usually the result is
as many "./" as objects here.
For a lot of duplicated paths, the following command is invoked.
_dummy := $(foreach d,$(obj-dirs), $(shell [ -d $(d) ] || mkdir -p $(d)))
Then, the costly shell command is run over and over again.
I see many points for optimization:
[1] Use $(sort ...) to cut down duplicated paths before passing them
to system call
[2] Use single $(shell ...) instead of repeating it with $(foreach ...)
This will reduce forking.
[3] We can calculate obj-dirs more simply. Most of objects are already
accumulated in $(targets). So, $(dir $(targets)) is fine and more
comprehensive.
I also removed ugly code in arch/x86/entry/vdso/Makefile. This is now
really unnecessary.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Douglas Anderson <dianders@chromium.org>
- kbuild cleanups and improvements for dtbs
- Code clean-up of overlay code and fixing for some long standing memory
leak and race condition in applying overlays
- Improvements to DT memory usage making sysfs/kobjects optional and
skipping unflattening of disabled nodes. This is part of kernel
tinification efforts.
- Final piece of removing storing the full path for every DT node. The
prerequisite conversion of printk's to use device_node format
specifier happened in 4.14.
- Sync with current upstream dtc. This brings additional checks to dtb
compiling.
- Binding doc tree wide removal of leading 0s from examples
- RTC binding documentation adding missing devices and some
consolidation of duplicated bindings
- Vendor prefix documentation for nutsboard, Silicon Storage Technology,
shimafuji, Tecon Microprocessor Technologies, DH electronics GmbH,
Opal Kelly, and Next Thing
-----BEGIN PGP SIGNATURE-----
iQItBAABCAAXBQJaCwaSEBxyb2JoQGtlcm5lbC5vcmcACgkQ+vtdtY28YcNzeA/8
C8uQhSsX2+UQZvFzcEA8KQAMGT3kYdrcf+gidRKwCEUWg1qscUEpTb3n3Rm5NUbU
RPD1s6GSlh6fJCMHDTQ6Tti/T59L7nZa2/AIGmUishGu4x4q1o18AobpFJmYP/EM
SJPwnmm5RV9WcZFao1y+sY3Xtn8DStxHO4cS+dyF5/EvPN9D8nbLJfu7bgTBAZww
HktIMB9kx+GTipRQZBvBwXoy5MJjthIZub4XwzesA4tGananj4cXlc0xaVxpdYy3
5bO6q5F7cbrZ2uyrF+oIChpCENK4VaXh80m0WHc8EzaG++shzEkR4he1vYkwnV+I
OYo4vsUg9dP8rBksUG1eYhS8fJKPvEBRNP7ETT5utVBy5I/tDEbo/crmQZRTIDIC
hZbhcdZlISZj0DzkMK2ZHQV9UYtRWzXrJbZHFIPP12GCyvXVxYJUIWb9iYnUYSon
KugygsFSpZHMWmfAhemw5/ctJZ19qhM5UIl2KZk5tMBHAf466ILmZjg0me6fYkOp
eADfwHJ1dLMdK79CVMHSfp+vArcZXp35B16c3sWpJB36Il97Mc/9siEufCL4GKX7
IBBnQBlbpSBKBejWVyI7Ip/Xp5u4qAQD+ZMJ9oLqBRqfWerHbDuOERlEOgwGqJYr
9v4HvP7V8eVUvAdqXka4EBfCyAgUzXDAxG2Dfmv9vGU=
=jgpN
-----END PGP SIGNATURE-----
Merge tag 'devicetree-for-4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux
Pull DeviceTree updates from Rob Herring:
"A bigger diffstat than usual with the kbuild changes and a tree wide
fix in the binding documentation.
Summary:
- kbuild cleanups and improvements for dtbs
- Code clean-up of overlay code and fixing for some long standing
memory leak and race condition in applying overlays
- Improvements to DT memory usage making sysfs/kobjects optional and
skipping unflattening of disabled nodes. This is part of kernel
tinification efforts.
- Final piece of removing storing the full path for every DT node.
The prerequisite conversion of printk's to use device_node format
specifier happened in 4.14.
- Sync with current upstream dtc. This brings additional checks to
dtb compiling.
- Binding doc tree wide removal of leading 0s from examples
- RTC binding documentation adding missing devices and some
consolidation of duplicated bindings
- Vendor prefix documentation for nutsboard, Silicon Storage
Technology, shimafuji, Tecon Microprocessor Technologies, DH
electronics GmbH, Opal Kelly, and Next Thing"
* tag 'devicetree-for-4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux: (55 commits)
dt-bindings: usb: add #phy-cells to usb-nop-xceiv
dt-bindings: Remove leading zeros from bindings notation
kbuild: handle dtb-y and CONFIG_OF_ALL_DTBS natively in Makefile.lib
MIPS: dts: remove bogus bcm96358nb4ser.dtb from dtb-y entry
kbuild: clean up *.dtb and *.dtb.S patterns from top-level Makefile
.gitignore: move *.dtb and *.dtb.S patterns to the top-level .gitignore
.gitignore: sort normal pattern rules alphabetically
dt-bindings: add vendor prefix for Next Thing Co.
scripts/dtc: Update to upstream version v1.4.5-6-gc1e55a5513e9
of: dynamic: fix memory leak related to properties of __of_node_dup
of: overlay: make pr_err() string unique
of: overlay: pr_err from return NOTIFY_OK to overlay apply/remove
of: overlay: remove unneeded check for NULL kbasename()
of: overlay: remove a dependency on device node full_name
of: overlay: simplify applying symbols from an overlay
of: overlay: avoid race condition between applying multiple overlays
of: overlay: loosen overly strict phandle clash check
of: overlay: expand check of whether overlay changeset can be removed
of: overlay: detect cases where device tree may become corrupt
of: overlay: minor restructuring
...
If CONFIG_OF_ALL_DTBS is enabled, "make ARCH=arm64 dtbs" compiles each
DTB twice; one from arch/arm64/boot/dts/*/Makefile and the other from
the dtb-$(CONFIG_OF_ALL_DTBS) line in arch/arm64/boot/dts/Makefile.
It could be a race problem when building DTBS in parallel.
Another minor issue is CONFIG_OF_ALL_DTBS covers only *.dts in vendor
sub-directories, so this broke when Broadcom added one more hierarchy
in arch/arm64/boot/dts/broadcom/<soc>/.
One idea to fix the issues in a clean way is to move DTB handling
to Kbuild core scripts. Makefile.dtbinst already recognizes dtb-y
natively, so it should not hurt to do so.
Add $(dtb-y) to extra-y, and $(dtb-) as well if CONFIG_OF_ALL_DTBS is
enabled. All clutter things in Makefiles go away.
As a bonus clean-up, I also removed dts-dirs. Just use subdir-y
directly to traverse sub-directories.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
[robh: corrected BUILTIN_DTB to CONFIG_BUILTIN_DTB]
Signed-off-by: Rob Herring <robh@kernel.org>
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Accumulate subdir-{cc,as}flags-y directly to KBUILD_{A,C}FLAGS.
Remove KBUILD_SUBDIR_{AS,CC}FLAGS.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Cao jin <caoj.fnst@cn.fujitsu.com>
It has:
1. Move comments close to what it want to comment.
2. Comments cleanup & improvement.
Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
I removed all the gperf use, but not the Makefile rules. Sam Ravnborg
says I get bonus points for cleaning this up. I'll hold him to it.
Requested-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Since commit d5d332d3f7 ("devicetree: Move include prefixes from
arch to separate directory"), cross-arch DT reference works well,
but only for CPP style #include directives.
It makes as much sense to share DT between different architectures
by using DTC's /include/ directives.
So, scripts/dtc/include-prefixes should be passed to both CPP and DTC.
I refactored Makefile.lib a bit to not repeat the same path.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Having arch/$(SRCARCH)/boot/dts as an include search path is not
very useful these days because some architectures such as ARM64,
MIPS have no DT in this directory. Instead, they have DT in vendor
sub-directories.
With some DT files in ARM and PowerPC fixed, we can now drop this
include search path.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rob Herring <robh@kernel.org>
This search path was added by commit b5190516b2 ("of: Move testcase
FDT data into drivers/of"). At that time, it was needed for platform
DT files to include testcase data.
It became unnecessary when commit ae9304c9d3 ("Adding selftest
testdata dynamically into live tree") introduced dynamic addition of
testcase data, but it missed to delete this search path.
Moreover, the directory drivers/of/testcase-data does not exist since
commit 19fd74879a ("of/unittest: Rename selftest.c to unittest.c").
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rob Herring <robh@kernel.org>
We use a directory under arch/$ARCH/boot/dts as an include path
that has links outside of the subtree to find dt-bindings from under
include/dt-bindings. That's been working well, but new DT architectures
haven't been adding them by default.
Recently there's been a desire to share some of the DT material between
arm and arm64, which originally caused developers to create symlinks or
relative includes between the subtrees. This isn't ideal -- it breaks
if the DT files aren't stored in the exact same hierarchy as the kernel
tree, and generally it's just icky.
As a somewhat cleaner solution we decided to add a $ARCH/ prefix link
once, and allow DTS files to reference dtsi (and dts) files in other
architectures that way.
Original approach was to create these links under each architecture,
but it lead to the problem of recursive symlinks.
As a remedy, move the include link directories out of the architecture
trees into a common location. At the same time, they can now share one
directory and one dt-bindings/ link as well.
Fixes: 4027494ae6 ('ARM: dts: add arm/arm64 include symlinks')
Reported-by: Russell King <linux@armlinux.org.uk>
Reported-by: Omar Sandoval <osandov@osandov.com>
Reviewed-by: Heiko Stuebner <heiko@sntech.de>
Reviewed-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: Heiko Stuebner <heiko@sntech.de>
Acked-by: Rob Herring <robh@kernel.org>
Cc: Heiko Stuebner <heiko@sntech.de>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mikael Starvik <starvik@axis.com>
Cc: Jesper Nilsson <jesper.nilsson@axis.com>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Frank Rowand <frowand.list@gmail.com>
Cc: linux-arch <linux-arch@vger.kernel.org>
Signed-off-by: Olof Johansson <olof@lixom.net>
- Improve Clang support
- Clean up various Makefiles
- Improve build log visibility (objtool, alpha, ia64)
- Improve compiler flag evaluation for better build performance
- Fix GCC version-dependent warning
- Fix genksyms
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJZE6qLAAoJED2LAQed4NsGk8YP/R7ZajSruFRmathN+wO1GEUv
+1cIJVpCv8OpM9fCSIuV3udAUUrH7Sj5IgHdg4P05/qmlgsG/4kUL5r9RyKdwjrA
dWqp8KKh40/JAfYNlMcRGz3cB4csvhMhnzZgV0zMSM1BBPP/xu2bCXrD4f/TGFMg
q04hHkmkesV0RUqOyPsCrKusxIsHhaGOmYUB285omGHO85IRItAW5CPh6sMhoJuQ
x7d5ZaXKCUH2fHXVMaw7gXcg28loUfKVJcU+em/0YZxoNZ31y6brM5YY7buF4FoJ
5RTXlvm12TUygY3fFCb3NURDSILHmL8Wk2wgFgYAC3NnXH1KJDOM4xxKXIVtTLGw
d71/hAurlLLZzjwmdUSSrhD+1OjFRZ4a5TJK/o3xehKUzYmFB49bcSKwdQ4H0jnM
m4iqNHw3rK+LJ0Zp71Ki3k3mcSW9yovpnJ2Uzi5Oz7g+oYAob7SZjejL2KsCtxZH
sQZQE5YeZj39Ot1K3Zw9CWx3JcUXWKtNu8cH7hXBgCyKS7H56xgtmib/S2yjXGu0
YIaMFJWTEB+FLrHRB4fYmeLG/kXIRn+N8Gy+/QUeXEHp4dFTV/il4Q2W+JAG35MS
3IGMr8PEZA7gTSYDQ/2kpv0HXDCwYSDxN3p8dlrnLy1bokVJX7Nk1pNq6gXfOl/E
4zIj3UsK0gPyO+CgprwC
=Nka/
-----END PGP SIGNATURE-----
Merge tag 'kbuild-v4.12' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild
Pull Kbuild updates from Masahiro Yamada:
- improve Clang support
- clean up various Makefiles
- improve build log visibility (objtool, alpha, ia64)
- improve compiler flag evaluation for better build performance
- fix GCC version-dependent warning
- fix genksyms
* tag 'kbuild-v4.12' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (23 commits)
kbuild: dtbinst: remove unnecessary __dtbs_install_prep target
ia64: beatify build log for gate.so and gate-syms.o
alpha: make short build log available for division routines
alpha: merge build rules of division routines
alpha: add $(src)/ rather than $(obj)/ to make source file path
Makefile: evaluate LDFLAGS_BUILD_ID only once
objtool: make it visible in make V=1 output
kbuild: clang: add -no-integrated-as to KBUILD_[AC]FLAGS
kbuild: Add support to generate LLVM assembly files
kbuild: Add better clang cross build support
kbuild: drop -Wno-unknown-warning-option from clang options
kbuild: fix asm-offset generation to work with clang
kbuild: consolidate redundant sed script ASM offset generation
frv: Use OFFSET macro in DEF_*REG()
kbuild: avoid conflict between -ffunction-sections and -pg on gcc-4.7
kbuild: Consolidate header generation from ASM offset information
kbuild: use -Oz instead of -Os when using clang
kbuild, LLVMLinux: Add -Werror to cc-option to support clang
Kbuild: make designated_init attribute fatal
kbuild: drop unneeded patterns '.*.orig' and '.*.rej' from distclean
...
- Fix sparse warnings in drivers/of/.
- Add more overlay unittests.
- Update dtc to v1.4.4-8-g756ffc4f52f6. This adds more checks on dts
files such as unit-address formatting and stricter character sets for
node and property names.
- Add a common DT modalias function.
- Move trivial-devices.txt up and out of i2c dir.
- ARM NVIC interrupt controller binding.
- Vendor prefixes for Sensirion, Dioo, Nordic, ROHM.
- Correct some binding file locations.
-----BEGIN PGP SIGNATURE-----
iQItBAABCAAXBQJZDM+bEBxyb2JoQGtlcm5lbC5vcmcACgkQ+vtdtY28YcM7Yw/+
NPgcfP2iFXWTC/D54neh/QliH7n5jO1YILQOd5ZJulTaKVCv1sNf4JCFVDQO/vuO
592f2jq/blhhh8yKrH0uzHvTADYO7G+XEv3D64Lu/MpmnBHTjut9WeG5h8SZWJD+
jGoz3Kx+Cmgxh/w1Ud/fh2qO3T0Es74TH+Ovi/AfpiywiYD8AqMlw0Tk+UtMRb12
NsihgiYXo8nT/BKe9aOFxOJjV9Pp+pSmyX5Mu2IdOtLnl0MXk5Rvn0mXLU2/oN5n
MONTyielvLB9opxaaSeSBadv3iXcVTagH6MinYjeQRwsLGWDy2YzxLccC/jBkyR5
v8X/IJtpivUzCm3ji+mFEKje5u5c+N3ZLhKmTkeyNBJGwrnD7zj0gVcM5GGdK5aW
Q4exqECklSgmpiCvlL1KWyXi3QsgMuu/wbsv5H5PgDe1wgMAtfKPrIm4kPpbLpW2
Dp+scFfL1v9drfvbcEum6PNw2/EGZY/okaRlcr0zFn8eMsa+yBlPBIXNLzzO7arp
6/lU6O5jDaSFRVKeKZ4qYSsc3GvN81XV+d9go6R3WR964xK4JdEkyt9Hntr1H0Hh
lBwyhSWH4nWnsXunc4GepRPVw+cdnOQdrj6T68bqLM5Gd6XWjh88WIDXqLynH6Kn
OBBBP/lOD7Su3antNkjnlaX9TP6BF8rNQePWne3AzzA=
=ca1M
-----END PGP SIGNATURE-----
Merge tag 'devicetree-for-4.12' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux
Pull DeviceTree updates from Rob Herring:
- fix sparse warnings in drivers/of/
- add more overlay unittests
- update dtc to v1.4.4-8-g756ffc4f52f6. This adds more checks on dts
files such as unit-address formatting and stricter character sets for
node and property names
- add a common DT modalias function
- move trivial-devices.txt up and out of i2c dir
- ARM NVIC interrupt controller binding
- vendor prefixes for Sensirion, Dioo, Nordic, ROHM
- correct some binding file locations
* tag 'devicetree-for-4.12' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux: (24 commits)
of: fix sparse warnings in fdt, irq, reserved mem, and resolver code
of: fix sparse warning in of_pci_range_parser_one
of: fix sparse warnings in of_find_next_cache_node
of/unittest: Missing unlocks on error
of: fix uninitialized variable warning for overlay test
of: fix unittest build without CONFIG_OF_OVERLAY
of: Add unit tests for applying overlays
of: per-file dtc compiler flags
fpga: region: add missing DT documentation for config complete timeout
of: Add vendor prefix for ROHM Semiconductor
of: fix "/cpus" reference leak in of_numa_parse_cpu_nodes()
of: Add vendor prefix for Nordic Semiconductor
dt-bindings: arm,nvic: Binding for ARM NVIC interrupt controller on Cortex-M
dtc: update warning settings for new bus and node/property name checks
scripts/dtc: Update to upstream version v1.4.4-8-g756ffc4f52f6
scripts/dtc: automate getting dtc version and log in update script
of: Add function for generating a DT modalias with a newline
of: fix of_device_get_modalias returned length when truncating buffers
Documentation: devicetree: move trivial-devices out of I2C realm
dt-bindings: add vendor prefix for Dioo
..
The dtc compiler version that adds initial support was available
in 4.11-rc1. Add the ability to set an additional dtc compiler
flag is needed by overlays.
Signed-off-by: Frank Rowand <frank.rowand@sony.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rob Herring <robh@kernel.org>
KBuild abuses the asm statement to write to a file and
clang chokes about these invalid asm statements. Hack it
even more by fooling this is actual valid asm code.
[masahiro:
Import Jeroen's work for U-Boot:
http://patchwork.ozlabs.org/patch/375026/
Tweak sed script a little to avoid garbage '#' for GCC case, like
#define NR_PAGEFLAGS 23 /* __NR_PAGEFLAGS # */ ]
Signed-off-by: Jeroen Hofstee <jeroen@myspectrum.nl>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Matthias Kaehlcke <mka@chromium.org>
Tested-by: Matthias Kaehlcke <mka@chromium.org>
This part ended up in redundant code after touched by multiple
people.
[1] Commit 3234282f33 ("x86, asm: Fix CFI macro invocations to
deal with shortcomings in gas") added parentheses for defined
expressions to support old gas for x86.
[2] Commit a22dcdb003 ("x86, asm: Fix ancient-GAS workaround")
split the pattern into two to avoid parentheses for non-numeric
expressions.
[3] Commit 95a2f6f72d ("Partially revert patch that encloses
asm-offset.h numbers in brackets") removed parentheses from numeric
expressions as well because parentheses in MN10300 assembly have a
special meaning (pointer access).
Apparently, there is a conflict between [1] and [3]. After all,
[3] took precedence, and a long time has passed since then.
Now, merge the two patterns again because the first one is covered
by the other.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Matthias Kaehlcke <mka@chromium.org>
Largely redundant code is used in different places to generate C headers
from offset information extracted from assembly language output.
Consolidate the code in Makefile.lib and use this instead.
Signed-off-by: Matthias Kaehlcke <mka@chromium.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
dtc gained new warnings checking PCI and simple buses, unit address
formatting, and stricter node and property name checking. Disable the
new dtc warnings by default as there are 1000s. As before, warnings are
enabled with W=1 or W=2. The strict node and property name checks are a
bit subjective, so they are only enabled for W=2.
Signed-off-by: Rob Herring <robh@kernel.org>
Commit db547ef190 ("Kbuild: don't add obj tree in additional includes")
causes warnings (-Wmissing-include-dirs) when compiling external modules
with KBUILD_OUTPUT set and W=1. This is because $src can be an absolute
path to the external module source which when prefixed with -I$(srctree)/
generates an incorrect directory path.
Signed-off-by: Bruce Allan <bruce.w.allan@intel.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Merge yet more updates from Andrew Morton:
- the rest of ocfs2
- various hotfixes, mainly MM
- quite a bit of misc stuff - drivers, fork, exec, signals, etc.
- printk updates
- firmware
- checkpatch
- nilfs2
- more kexec stuff than usual
- rapidio updates
- w1 things
* emailed patches from Andrew Morton <akpm@linux-foundation.org>: (111 commits)
ipc: delete "nr_ipc_ns"
kcov: allow more fine-grained coverage instrumentation
init/Kconfig: add clarification for out-of-tree modules
config: add android config fragments
init/Kconfig: ban CONFIG_LOCALVERSION_AUTO with allmodconfig
relay: add global mode support for buffer-only channels
init: allow blacklisting of module_init functions
w1:omap_hdq: fix regression
w1: add helper macro module_w1_family
w1: remove need for ida and use PLATFORM_DEVID_AUTO
rapidio/switches: add driver for IDT gen3 switches
powerpc/fsl_rio: apply changes for RIO spec rev 3
rapidio: modify for rev.3 specification changes
rapidio: change inbound window size type to u64
rapidio/idt_gen2: fix locking warning
rapidio: fix error handling in mbox request/release functions
rapidio/tsi721_dma: advance queue processing from transfer submit call
rapidio/tsi721: add messaging mbox selector parameter
rapidio/tsi721: add PCIe MRRS override parameter
rapidio/tsi721_dma: add channel mask and queue size parameters
...
For more targeted fuzzing, it's better to disable kernel-wide
instrumentation and instead enable it on a per-subsystem basis. This
follows the pattern of UBSAN and allows you to compile in the kcov
driver without instrumenting the whole kernel.
To instrument a part of the kernel, you can use either
# for a single file in the current directory
KCOV_INSTRUMENT_filename.o := y
or
# for all the files in the current directory (excluding subdirectories)
KCOV_INSTRUMENT := y
or
# (same as above)
ccflags-y += $(CFLAGS_KCOV)
or
# for all the files in the current directory (including subdirectories)
subdir-ccflags-y += $(CFLAGS_KCOV)
Link: http://lkml.kernel.org/r/1464008380-11405-1-git-send-email-vegard.nossum@oracle.com
Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
When building with separate object directories and driver specific
Makefiles that add additional header include paths, Kbuild adjusts
the gcc flags so that we include both the directory in the source
tree and in the object tree.
However, due to another bug I fixed earlier, this did not actually
include the correct directory in the object tree, so we know that
we only really need the source tree here. Also, including the
object tree sometimes causes warnings about nonexisting directories
when the include path only exists in the source.
This changes the logic to only emit the -I argument for the srctree,
not for objects. We still need both $(srctree)/$(src) and $(obj)
though, so I'm adding them manually.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Michal Marek <mmarek@suse.com>
Pull kbuild updates from Michal Marek:
- new option CONFIG_TRIM_UNUSED_KSYMS which does a two-pass build and
unexports symbols which are not used in the current config [Nicolas
Pitre]
- several kbuild rule cleanups [Masahiro Yamada]
- warning option adjustments for gcov etc [Arnd Bergmann]
- a few more small fixes
* 'kbuild' of git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild: (31 commits)
kbuild: move -Wunused-const-variable to W=1 warning level
kbuild: fix if_change and friends to consider argument order
kbuild: fix adjust_autoksyms.sh for modules that need only one symbol
kbuild: fix ksym_dep_filter when multiple EXPORT_SYMBOL() on the same line
gcov: disable -Wmaybe-uninitialized warning
gcov: disable tree-loop-im to reduce stack usage
gcov: disable for COMPILE_TEST
Kbuild: disable 'maybe-uninitialized' warning for CONFIG_PROFILE_ALL_BRANCHES
Kbuild: change CC_OPTIMIZE_FOR_SIZE definition
kbuild: forbid kernel directory to contain spaces and colons
kbuild: adjust ksym_dep_filter for some cmd_* renames
kbuild: Fix dependencies for final vmlinux link
kbuild: better abstract vmlinux sequential prerequisites
kbuild: fix call to adjust_autoksyms.sh when output directory specified
kbuild: Get rid of KBUILD_STR
kbuild: rename cmd_as_s_S to cmd_cpp_s_S
kbuild: rename cmd_cc_i_c to cmd_cpp_i_c
kbuild: drop redundant "PHONY += FORCE"
kbuild: delete unnecessary "@:"
kbuild: mark help target as PHONY
...
The compiler can accept -DKBUILD_MODNAME="foo", it's just a matter of
quoting. That way, we reduce the gcc command line a bit.
Signed-off-by: Michal Marek <mmarek@suse.com>
The newly added dtc warning to check DT unit-address without reg
property and vice-versa generates lots of warnings. Turn off the check
unless building with W=1 or W=2.
Signed-off-by: Rob Herring <robh@kernel.org>
Cc: Michal Marek <mmarek@suse.com>
Cc: linux-kbuild@vger.kernel.org
kcov provides code coverage collection for coverage-guided fuzzing
(randomized testing). Coverage-guided fuzzing is a testing technique
that uses coverage feedback to determine new interesting inputs to a
system. A notable user-space example is AFL
(http://lcamtuf.coredump.cx/afl/). However, this technique is not
widely used for kernel testing due to missing compiler and kernel
support.
kcov does not aim to collect as much coverage as possible. It aims to
collect more or less stable coverage that is function of syscall inputs.
To achieve this goal it does not collect coverage in soft/hard
interrupts and instrumentation of some inherently non-deterministic or
non-interesting parts of kernel is disbled (e.g. scheduler, locking).
Currently there is a single coverage collection mode (tracing), but the
API anticipates additional collection modes. Initially I also
implemented a second mode which exposes coverage in a fixed-size hash
table of counters (what Quentin used in his original patch). I've
dropped the second mode for simplicity.
This patch adds the necessary support on kernel side. The complimentary
compiler support was added in gcc revision 231296.
We've used this support to build syzkaller system call fuzzer, which has
found 90 kernel bugs in just 2 months:
https://github.com/google/syzkaller/wiki/Found-Bugs
We've also found 30+ bugs in our internal systems with syzkaller.
Another (yet unexplored) direction where kcov coverage would greatly
help is more traditional "blob mutation". For example, mounting a
random blob as a filesystem, or receiving a random blob over wire.
Why not gcov. Typical fuzzing loop looks as follows: (1) reset
coverage, (2) execute a bit of code, (3) collect coverage, repeat. A
typical coverage can be just a dozen of basic blocks (e.g. an invalid
input). In such context gcov becomes prohibitively expensive as
reset/collect coverage steps depend on total number of basic
blocks/edges in program (in case of kernel it is about 2M). Cost of
kcov depends only on number of executed basic blocks/edges. On top of
that, kernel requires per-thread coverage because there are always
background threads and unrelated processes that also produce coverage.
With inlined gcov instrumentation per-thread coverage is not possible.
kcov exposes kernel PCs and control flow to user-space which is
insecure. But debugfs should not be mapped as user accessible.
Based on a patch by Quentin Casasnovas.
[akpm@linux-foundation.org: make task_struct.kcov_mode have type `enum kcov_mode']
[akpm@linux-foundation.org: unbreak allmodconfig]
[akpm@linux-foundation.org: follow x86 Makefile layout standards]
Signed-off-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Kees Cook <keescook@chromium.org>
Cc: syzkaller <syzkaller@googlegroups.com>
Cc: Vegard Nossum <vegard.nossum@oracle.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Tavis Ormandy <taviso@google.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Cc: Kostya Serebryany <kcc@google.com>
Cc: Eric Dumazet <edumazet@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Kees Cook <keescook@google.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Sasha Levin <sasha.levin@oracle.com>
Cc: David Drysdale <drysdale@google.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Kirill A. Shutemov <kirill@shutemov.name>
Cc: Jiri Slaby <jslaby@suse.cz>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Development of dtc happens in its own upstream repository, but testing
dtc changes against the kernel tree is useful. Change dtc to a variable
that users can override.
Signed-off-by: Rob Herring <robh@kernel.org>
Cc: Michal Marek <mmarek@suse.com>
Cc: linux-kbuild@vger.kernel.org
UBSAN uses compile-time instrumentation to catch undefined behavior
(UB). Compiler inserts code that perform certain kinds of checks before
operations that could cause UB. If check fails (i.e. UB detected)
__ubsan_handle_* function called to print error message.
So the most of the work is done by compiler. This patch just implements
ubsan handlers printing errors.
GCC has this capability since 4.9.x [1] (see -fsanitize=undefined
option and its suboptions).
However GCC 5.x has more checkers implemented [2].
Article [3] has a bit more details about UBSAN in the GCC.
[1] - https://gcc.gnu.org/onlinedocs/gcc-4.9.0/gcc/Debugging-Options.html
[2] - https://gcc.gnu.org/onlinedocs/gcc/Debugging-Options.html
[3] - http://developerblog.redhat.com/2014/10/16/gcc-undefined-behavior-sanitizer-ubsan/
Issues which UBSAN has found thus far are:
Found bugs:
* out-of-bounds access - 97840cb67f ("netfilter: nfnetlink: fix
insufficient validation in nfnetlink_bind")
undefined shifts:
* d48458d4a7 ("jbd2: use a better hash function for the revoke
table")
* 10632008b9 ("clockevents: Prevent shift out of bounds")
* 'x << -1' shift in ext4 -
http://lkml.kernel.org/r/<5444EF21.8020501@samsung.com>
* undefined rol32(0) -
http://lkml.kernel.org/r/<1449198241-20654-1-git-send-email-sasha.levin@oracle.com>
* undefined dirty_ratelimit calculation -
http://lkml.kernel.org/r/<566594E2.3050306@odin.com>
* undefined roundown_pow_of_two(0) -
http://lkml.kernel.org/r/<1449156616-11474-1-git-send-email-sasha.levin@oracle.com>
* [WONTFIX] undefined shift in __bpf_prog_run -
http://lkml.kernel.org/r/<CACT4Y+ZxoR3UjLgcNdUm4fECLMx2VdtfrENMtRRCdgHB2n0bJA@mail.gmail.com>
WONTFIX here because it should be fixed in bpf program, not in kernel.
signed overflows:
* 32a8df4e0b ("sched: Fix odd values in effective_load()
calculations")
* mul overflow in ntp -
http://lkml.kernel.org/r/<1449175608-1146-1-git-send-email-sasha.levin@oracle.com>
* incorrect conversion into rtc_time in rtc_time64_to_tm() -
http://lkml.kernel.org/r/<1449187944-11730-1-git-send-email-sasha.levin@oracle.com>
* unvalidated timespec in io_getevents() -
http://lkml.kernel.org/r/<CACT4Y+bBxVYLQ6LtOKrKtnLthqLHcw-BMp3aqP3mjdAvr9FULQ@mail.gmail.com>
* [NOTABUG] signed overflow in ktime_add_safe() -
http://lkml.kernel.org/r/<CACT4Y+aJ4muRnWxsUe1CMnA6P8nooO33kwG-c8YZg=0Xc8rJqw@mail.gmail.com>
[akpm@linux-foundation.org: fix unused local warning]
[akpm@linux-foundation.org: fix __int128 build woes]
Signed-off-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sasha Levin <sasha.levin@oracle.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Michal Marek <mmarek@suse.cz>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Yury Gribov <y.gribov@samsung.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Konstantin Khlebnikov <koct9i@gmail.com>
Cc: Kostya Serebryany <kcc@google.com>
Cc: Johannes Berg <johannes@sipsolutions.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Pull kbuild updates from Michal Marek:
- Make <modname>-m in makefiles work like <modname>-y and fix the
fallout
- Minor genksyms fix
- Fix race with make -j install modules_install
- Move -Wsign-compare from make W=1 to W=2
- Other minor fixes
* 'kbuild' of git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild:
kbuild: Demote 'sign-compare' warning to W=2
Makefile: revert "Makefile: Document ability to make file.lst and file.S" partially
kbuild: Do not run modules_install and install in paralel
genksyms: Handle string literals with spaces in reference files
fixdep: constify strrcmp arguments
ath10k: Fix build with CONFIG_THERMAL=m
Revert "drm: Hack around CONFIG_AGP=m build failures"
kbuild: Allow to specify composite modules with modname-m
staging/ad7606: Actually build the interface modules
It is already possible to remove CFLAGS with the CFLAGS_REMOVE option
that was introduced with commit 656ee82cc8 ("kbuild: create new
CFLAGS_REMOVE_(basename).o option"). However it is not possible to
remove AFLAGS for assembler files.
So this patch just adds the AFLAGS_REMOVE option which works the same
like CFLAGS_REMOVE.
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Acked-by: Michal Marek <mmarek@suse.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
This allows to write
drm-$(CONFIG_AGP) += drm_agpsupport.o
without having to handle CONFIG_AGP=y vs. CONFIG_AGP=m. Only support
this syntax for modules, since built-in code depending on something
modular cannot work and init/Makefile actually relies on the current
semantics. There are a few drivers which adapted to the current
semantics out of necessity; these are fixed to also work when the
respective subsystem is modular.
Acked-by: Peter Chen <peter.chen@freescale.com> [chipidea]
Signed-off-by: Michal Marek <mmarek@suse.com>
When building specific DTBs out of the kernel tree the vendor subdirs
(boot/dts/<vendor>) are not created, ensure that they are before
building the DTB.
Signed-off-by: Nathan Rossi <nathan.rossi@xilinx.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Olof Johansson <olof@lixom.net>
Kernel Address sanitizer (KASan) is a dynamic memory error detector. It
provides fast and comprehensive solution for finding use-after-free and
out-of-bounds bugs.
KASAN uses compile-time instrumentation for checking every memory access,
therefore GCC > v4.9.2 required. v4.9.2 almost works, but has issues with
putting symbol aliases into the wrong section, which breaks kasan
instrumentation of globals.
This patch only adds infrastructure for kernel address sanitizer. It's
not available for use yet. The idea and some code was borrowed from [1].
Basic idea:
The main idea of KASAN is to use shadow memory to record whether each byte
of memory is safe to access or not, and use compiler's instrumentation to
check the shadow memory on each memory access.
Address sanitizer uses 1/8 of the memory addressable in kernel for shadow
memory and uses direct mapping with a scale and offset to translate a
memory address to its corresponding shadow address.
Here is function to translate address to corresponding shadow address:
unsigned long kasan_mem_to_shadow(unsigned long addr)
{
return (addr >> KASAN_SHADOW_SCALE_SHIFT) + KASAN_SHADOW_OFFSET;
}
where KASAN_SHADOW_SCALE_SHIFT = 3.
So for every 8 bytes there is one corresponding byte of shadow memory.
The following encoding used for each shadow byte: 0 means that all 8 bytes
of the corresponding memory region are valid for access; k (1 <= k <= 7)
means that the first k bytes are valid for access, and other (8 - k) bytes
are not; Any negative value indicates that the entire 8-bytes are
inaccessible. Different negative values used to distinguish between
different kinds of inaccessible memory (redzones, freed memory) (see
mm/kasan/kasan.h).
To be able to detect accesses to bad memory we need a special compiler.
Such compiler inserts a specific function calls (__asan_load*(addr),
__asan_store*(addr)) before each memory access of size 1, 2, 4, 8 or 16.
These functions check whether memory region is valid to access or not by
checking corresponding shadow memory. If access is not valid an error
printed.
Historical background of the address sanitizer from Dmitry Vyukov:
"We've developed the set of tools, AddressSanitizer (Asan),
ThreadSanitizer and MemorySanitizer, for user space. We actively use
them for testing inside of Google (continuous testing, fuzzing,
running prod services). To date the tools have found more than 10'000
scary bugs in Chromium, Google internal codebase and various
open-source projects (Firefox, OpenSSL, gcc, clang, ffmpeg, MySQL and
lots of others): [2] [3] [4].
The tools are part of both gcc and clang compilers.
We have not yet done massive testing under the Kernel AddressSanitizer
(it's kind of chicken and egg problem, you need it to be upstream to
start applying it extensively). To date it has found about 50 bugs.
Bugs that we've found in upstream kernel are listed in [5].
We've also found ~20 bugs in out internal version of the kernel. Also
people from Samsung and Oracle have found some.
[...]
As others noted, the main feature of AddressSanitizer is its
performance due to inline compiler instrumentation and simple linear
shadow memory. User-space Asan has ~2x slowdown on computational
programs and ~2x memory consumption increase. Taking into account that
kernel usually consumes only small fraction of CPU and memory when
running real user-space programs, I would expect that kernel Asan will
have ~10-30% slowdown and similar memory consumption increase (when we
finish all tuning).
I agree that Asan can well replace kmemcheck. We have plans to start
working on Kernel MemorySanitizer that finds uses of unitialized
memory. Asan+Msan will provide feature-parity with kmemcheck. As
others noted, Asan will unlikely replace debug slab and pagealloc that
can be enabled at runtime. Asan uses compiler instrumentation, so even
if it is disabled, it still incurs visible overheads.
Asan technology is easily portable to other architectures. Compiler
instrumentation is fully portable. Runtime has some arch-dependent
parts like shadow mapping and atomic operation interception. They are
relatively easy to port."
Comparison with other debugging features:
========================================
KMEMCHECK:
- KASan can do almost everything that kmemcheck can. KASan uses
compile-time instrumentation, which makes it significantly faster than
kmemcheck. The only advantage of kmemcheck over KASan is detection of
uninitialized memory reads.
Some brief performance testing showed that kasan could be
x500-x600 times faster than kmemcheck:
$ netperf -l 30
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to localhost (127.0.0.1) port 0 AF_INET
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
no debug: 87380 16384 16384 30.00 41624.72
kasan inline: 87380 16384 16384 30.00 12870.54
kasan outline: 87380 16384 16384 30.00 10586.39
kmemcheck: 87380 16384 16384 30.03 20.23
- Also kmemcheck couldn't work on several CPUs. It always sets
number of CPUs to 1. KASan doesn't have such limitation.
DEBUG_PAGEALLOC:
- KASan is slower than DEBUG_PAGEALLOC, but KASan works on sub-page
granularity level, so it able to find more bugs.
SLUB_DEBUG (poisoning, redzones):
- SLUB_DEBUG has lower overhead than KASan.
- SLUB_DEBUG in most cases are not able to detect bad reads,
KASan able to detect both reads and writes.
- In some cases (e.g. redzone overwritten) SLUB_DEBUG detect
bugs only on allocation/freeing of object. KASan catch
bugs right before it will happen, so we always know exact
place of first bad read/write.
[1] https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForKernel
[2] https://code.google.com/p/address-sanitizer/wiki/FoundBugs
[3] https://code.google.com/p/thread-sanitizer/wiki/FoundBugs
[4] https://code.google.com/p/memory-sanitizer/wiki/FoundBugs
[5] https://code.google.com/p/address-sanitizer/wiki/AddressSanitizerForKernel#Trophies
Based on work by Andrey Konovalov.
Signed-off-by: Andrey Ryabinin <a.ryabinin@samsung.com>
Acked-by: Michal Marek <mmarek@suse.cz>
Signed-off-by: Andrey Konovalov <adech.fo@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Konstantin Serebryany <kcc@google.com>
Cc: Dmitry Chernenkov <dmitryc@google.com>
Cc: Yuri Gribov <tetra2005@gmail.com>
Cc: Konstantin Khlebnikov <koct9i@gmail.com>
Cc: Sasha Levin <sasha.levin@oracle.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Move dtbs install rules to Makefile.dtbinst. This change is needed to
implement support for dts vendor subdirs. The change makes Makefiles
easier and smaller as no longer the dtbs_install rule needs to be
defined. Another advantage is that install goals are not encoded in
targets anymore (%.dtb_dtbinst_).
Signed-off-by: Robert Richter <rrichter@cavium.com>
The comment in scripts/Makefile.build says as follows:
We would rather have a list of rules like
foo.o: $(foo-objs)
but that's not so easy, so we rather make all composite objects depend
on the set of all their parts
This commit makes it possible!
For example, assume a Makefile like this
obj-m = foo.o bar.o
foo-objs := foo1.o foo2.o
bar-objs := bar1.o bar2.o
Without this patch, foo.o depends on all of
foo1.o foo2.o bar1.o bar2.o.
It looks funny that foo.o is regenerated when bar1.c is updated.
Now we can handle the dependency of foo.o and bar.o separately.
Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Michal Marek <mmarek@suse.cz>
Pull kbuild changes from Michal Marek:
- cleanups in the main Makefiles and Documentation/DocBook/Makefile
- make O=... directory is automatically created if needed
- mrproper/distclean removes the old include/linux/version.h to make
life easier when bisecting across the commit that moved the version.h
file
* 'kbuild' of git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild:
kbuild: docbook: fix the include error when executing "make help"
kbuild: create a build directory automatically for out-of-tree build
kbuild: remove redundant '.*.cmd' pattern from make distclean
kbuild: move "quote" to Kbuild.include to be consistent
kbuild: docbook: use $(obj) and $(src) rather than specific path
kbuild: unconditionally clobber include/linux/version.h on distclean
kbuild: docbook: specify KERNELDOC dependency correctly
kbuild: docbook: include cmd files more simply
kbuild: specify build_docproc as a phony target
Rob Herring