linux-kselftest-kunit-fixes-5.10-rc5
This Kunit update for Linux 5.10-rc5 consists of several fixes Kunit documentation, tool, compile time fixes not pollute source directory, and fix to remove tools/testing/kunit/.gitattributes file. -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEPZKym/RZuOCGeA/kCwJExA0NQxwFAl+0UhsACgkQCwJExA0N QxxZKw//fHKJC6eTB6c0It5wpWQ62mbzWbKNypoOrNq5LP2euDzOp6FIMwJVvA5H Vcdj0spd+2ZlH+8DqxBrKhDkDFUwU/VeuQG6u0Qvfsm2xF50QKDidaGZ7sxBiqvp 5A1iRh+zxpUmXKivm4ji7PZJ23kR7RT/ZoVrTRqOdqyjguLZSO3TO3mUd12xV4hh QMbw6uaFWIuiulmZrjbwak+v6+Amn2lXZDbtD8Srm6bhZAQ+ejB8bAg8CpOpu5Cp dzY58HbuivuJ8yzosAag7NCtzvak/OWL70uxXn6AMvTyxHA6BmenQS0lRHdEWVR5 YOsLLFDKaCrRFdnMwZthxzsZliV1HNzkTZ+4FdlKl7BHFzAsRjzinA9xgVevn0x3 mvSBwkcRZ2Cy/Ep2RXOOfpRF1uE64XRyBubpl+h8+rxIwbyJs4irFRT2VF9YTjTA NRUxJ0kYJ/KmzMSAZ2YS6QCqXX9Gzq9xH9XQVmkW7BsXUF8dk+p8TkLkQoUaPW6S BO+lH+8mrLoOrTzVQlVHxPpMZpOyhn2jU2JMQmIsLuQkNOmtoNzHtIiLstRMNNVb 6x20tHleUcAkDxXFqwgtuwxUTqTIlcNAhxFtf+YK4shdm8Rnx4uqvy2k19UW1KO3 4uHgDc7+zMFvYPKpSVL8fRenQ7bf5180YUFZPnUXnNhpR6bONWI= =mGWU -----END PGP SIGNATURE----- Merge tag 'linux-kselftest-kunit-fixes-5.10-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest Pull Kunit fixes from Shuah Khan: "Several fixes to Kunit documentation and tools, and to not pollute the source directory. Also remove the incorrect kunit .gitattributes file" * tag 'linux-kselftest-kunit-fixes-5.10-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest: kunit: fix display of failed expectations for strings kunit: tool: fix extra trailing \n in raw + parsed test output kunit: tool: print out stderr from make (like build warnings) KUnit: Docs: usage: wording fixes KUnit: Docs: style: fix some Kconfig example issues KUnit: Docs: fix a wording typo kunit: Do not pollute source directory with generated files (test.log) kunit: Do not pollute source directory with generated files (.kunitconfig) kunit: tool: fix pre-existing python type annotation errors kunit: Fix kunit.py parse subcommand (use null build_dir) kunit: tool: unmark test_data as binary blobszero-sugar-mainline-defconfig
Linus Torvalds
commit
f86fee1845
|
|
@ -90,7 +90,7 @@ things to try.
|
|||
re-run kunit_tool.
|
||||
5. Try to run ``make ARCH=um defconfig`` before running ``kunit.py run``. This
|
||||
may help clean up any residual config items which could be causing problems.
|
||||
6. Finally, try running KUnit outside UML. KUnit and KUnit tests can run be
|
||||
6. Finally, try running KUnit outside UML. KUnit and KUnit tests can be
|
||||
built into any kernel, or can be built as a module and loaded at runtime.
|
||||
Doing so should allow you to determine if UML is causing the issue you're
|
||||
seeing. When tests are built-in, they will execute when the kernel boots, and
|
||||
|
|
|
|||
|
|
@ -175,17 +175,17 @@ An example Kconfig entry:
|
|||
|
||||
.. code-block:: none
|
||||
|
||||
config FOO_KUNIT_TEST
|
||||
tristate "KUnit test for foo" if !KUNIT_ALL_TESTS
|
||||
depends on KUNIT
|
||||
default KUNIT_ALL_TESTS
|
||||
help
|
||||
This builds unit tests for foo.
|
||||
config FOO_KUNIT_TEST
|
||||
tristate "KUnit test for foo" if !KUNIT_ALL_TESTS
|
||||
depends on KUNIT
|
||||
default KUNIT_ALL_TESTS
|
||||
help
|
||||
This builds unit tests for foo.
|
||||
|
||||
For more information on KUnit and unit tests in general, please refer
|
||||
to the KUnit documentation in Documentation/dev-tools/kunit
|
||||
For more information on KUnit and unit tests in general, please refer
|
||||
to the KUnit documentation in Documentation/dev-tools/kunit/.
|
||||
|
||||
If unsure, say N
|
||||
If unsure, say N.
|
||||
|
||||
|
||||
Test File and Module Names
|
||||
|
|
|
|||
|
|
@ -92,7 +92,7 @@ behavior of a function called ``add``; the first parameter is always of type
|
|||
the second parameter, in this case, is what the value is expected to be; the
|
||||
last value is what the value actually is. If ``add`` passes all of these
|
||||
expectations, the test case, ``add_test_basic`` will pass; if any one of these
|
||||
expectations fail, the test case will fail.
|
||||
expectations fails, the test case will fail.
|
||||
|
||||
It is important to understand that a test case *fails* when any expectation is
|
||||
violated; however, the test will continue running, potentially trying other
|
||||
|
|
@ -202,7 +202,7 @@ Example:
|
|||
kunit_test_suite(example_test_suite);
|
||||
|
||||
In the above example the test suite, ``example_test_suite``, would run the test
|
||||
cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``,
|
||||
cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``;
|
||||
each would have ``example_test_init`` called immediately before it and would
|
||||
have ``example_test_exit`` called immediately after it.
|
||||
``kunit_test_suite(example_test_suite)`` registers the test suite with the
|
||||
|
|
@ -229,7 +229,7 @@ through some sort of indirection where a function is exposed as part of an API
|
|||
such that the definition of that function can be changed without affecting the
|
||||
rest of the code base. In the kernel this primarily comes from two constructs,
|
||||
classes, structs that contain function pointers that are provided by the
|
||||
implementer, and architecture specific functions which have definitions selected
|
||||
implementer, and architecture-specific functions which have definitions selected
|
||||
at compile time.
|
||||
|
||||
Classes
|
||||
|
|
@ -459,7 +459,7 @@ KUnit on non-UML architectures
|
|||
By default KUnit uses UML as a way to provide dependencies for code under test.
|
||||
Under most circumstances KUnit's usage of UML should be treated as an
|
||||
implementation detail of how KUnit works under the hood. Nevertheless, there
|
||||
are instances where being able to run architecture specific code or test
|
||||
are instances where being able to run architecture-specific code or test
|
||||
against real hardware is desirable. For these reasons KUnit supports running on
|
||||
other architectures.
|
||||
|
||||
|
|
@ -599,7 +599,7 @@ writing normal KUnit tests. One special caveat is that you have to reset
|
|||
hardware state in between test cases; if this is not possible, you may only be
|
||||
able to run one test case per invocation.
|
||||
|
||||
.. TODO(brendanhiggins@google.com): Add an actual example of an architecture
|
||||
.. TODO(brendanhiggins@google.com): Add an actual example of an architecture-
|
||||
dependent KUnit test.
|
||||
|
||||
KUnit debugfs representation
|
||||
|
|
|
|||
|
|
@ -1105,7 +1105,7 @@ do { \
|
|||
KUNIT_ASSERTION(test, \
|
||||
strcmp(__left, __right) op 0, \
|
||||
kunit_binary_str_assert, \
|
||||
KUNIT_INIT_BINARY_ASSERT_STRUCT(test, \
|
||||
KUNIT_INIT_BINARY_STR_ASSERT_STRUCT(test, \
|
||||
assert_type, \
|
||||
#op, \
|
||||
#left, \
|
||||
|
|
|
|||
|
|
@ -1 +0,0 @@
|
|||
test_data/* binary
|
||||
|
|
@ -11,7 +11,6 @@ import argparse
|
|||
import sys
|
||||
import os
|
||||
import time
|
||||
import shutil
|
||||
|
||||
from collections import namedtuple
|
||||
from enum import Enum, auto
|
||||
|
|
@ -44,11 +43,6 @@ class KunitStatus(Enum):
|
|||
BUILD_FAILURE = auto()
|
||||
TEST_FAILURE = auto()
|
||||
|
||||
def create_default_kunitconfig():
|
||||
if not os.path.exists(kunit_kernel.kunitconfig_path):
|
||||
shutil.copyfile('arch/um/configs/kunit_defconfig',
|
||||
kunit_kernel.kunitconfig_path)
|
||||
|
||||
def get_kernel_root_path():
|
||||
parts = sys.argv[0] if not __file__ else __file__
|
||||
parts = os.path.realpath(parts).split('tools/testing/kunit')
|
||||
|
|
@ -61,7 +55,6 @@ def config_tests(linux: kunit_kernel.LinuxSourceTree,
|
|||
kunit_parser.print_with_timestamp('Configuring KUnit Kernel ...')
|
||||
|
||||
config_start = time.time()
|
||||
create_default_kunitconfig()
|
||||
success = linux.build_reconfig(request.build_dir, request.make_options)
|
||||
config_end = time.time()
|
||||
if not success:
|
||||
|
|
@ -262,12 +255,12 @@ def main(argv, linux=None):
|
|||
if not os.path.exists(cli_args.build_dir):
|
||||
os.mkdir(cli_args.build_dir)
|
||||
|
||||
if not os.path.exists(kunit_kernel.kunitconfig_path):
|
||||
create_default_kunitconfig()
|
||||
|
||||
if not linux:
|
||||
linux = kunit_kernel.LinuxSourceTree()
|
||||
|
||||
linux.create_kunitconfig(cli_args.build_dir)
|
||||
linux.read_kunitconfig(cli_args.build_dir)
|
||||
|
||||
request = KunitRequest(cli_args.raw_output,
|
||||
cli_args.timeout,
|
||||
cli_args.jobs,
|
||||
|
|
@ -283,12 +276,12 @@ def main(argv, linux=None):
|
|||
not os.path.exists(cli_args.build_dir)):
|
||||
os.mkdir(cli_args.build_dir)
|
||||
|
||||
if not os.path.exists(kunit_kernel.kunitconfig_path):
|
||||
create_default_kunitconfig()
|
||||
|
||||
if not linux:
|
||||
linux = kunit_kernel.LinuxSourceTree()
|
||||
|
||||
linux.create_kunitconfig(cli_args.build_dir)
|
||||
linux.read_kunitconfig(cli_args.build_dir)
|
||||
|
||||
request = KunitConfigRequest(cli_args.build_dir,
|
||||
cli_args.make_options)
|
||||
result = config_tests(linux, request)
|
||||
|
|
@ -301,6 +294,9 @@ def main(argv, linux=None):
|
|||
if not linux:
|
||||
linux = kunit_kernel.LinuxSourceTree()
|
||||
|
||||
linux.create_kunitconfig(cli_args.build_dir)
|
||||
linux.read_kunitconfig(cli_args.build_dir)
|
||||
|
||||
request = KunitBuildRequest(cli_args.jobs,
|
||||
cli_args.build_dir,
|
||||
cli_args.alltests,
|
||||
|
|
@ -315,6 +311,9 @@ def main(argv, linux=None):
|
|||
if not linux:
|
||||
linux = kunit_kernel.LinuxSourceTree()
|
||||
|
||||
linux.create_kunitconfig(cli_args.build_dir)
|
||||
linux.read_kunitconfig(cli_args.build_dir)
|
||||
|
||||
exec_request = KunitExecRequest(cli_args.timeout,
|
||||
cli_args.build_dir,
|
||||
cli_args.alltests)
|
||||
|
|
@ -337,7 +336,7 @@ def main(argv, linux=None):
|
|||
kunit_output = f.read().splitlines()
|
||||
request = KunitParseRequest(cli_args.raw_output,
|
||||
kunit_output,
|
||||
cli_args.build_dir,
|
||||
None,
|
||||
cli_args.json)
|
||||
result = parse_tests(request)
|
||||
if result.status != KunitStatus.SUCCESS:
|
||||
|
|
|
|||
|
|
@ -6,10 +6,10 @@
|
|||
# Author: Felix Guo <felixguoxiuping@gmail.com>
|
||||
# Author: Brendan Higgins <brendanhiggins@google.com>
|
||||
|
||||
|
||||
import logging
|
||||
import subprocess
|
||||
import os
|
||||
import shutil
|
||||
import signal
|
||||
|
||||
from contextlib import ExitStack
|
||||
|
|
@ -18,8 +18,10 @@ import kunit_config
|
|||
import kunit_parser
|
||||
|
||||
KCONFIG_PATH = '.config'
|
||||
kunitconfig_path = '.kunitconfig'
|
||||
KUNITCONFIG_PATH = '.kunitconfig'
|
||||
DEFAULT_KUNITCONFIG_PATH = 'arch/um/configs/kunit_defconfig'
|
||||
BROKEN_ALLCONFIG_PATH = 'tools/testing/kunit/configs/broken_on_uml.config'
|
||||
OUTFILE_PATH = 'test.log'
|
||||
|
||||
class ConfigError(Exception):
|
||||
"""Represents an error trying to configure the Linux kernel."""
|
||||
|
|
@ -82,36 +84,51 @@ class LinuxSourceTreeOperations(object):
|
|||
if build_dir:
|
||||
command += ['O=' + build_dir]
|
||||
try:
|
||||
subprocess.check_output(command, stderr=subprocess.STDOUT)
|
||||
proc = subprocess.Popen(command,
|
||||
stderr=subprocess.PIPE,
|
||||
stdout=subprocess.DEVNULL)
|
||||
except OSError as e:
|
||||
raise BuildError('Could not call execute make: ' + str(e))
|
||||
except subprocess.CalledProcessError as e:
|
||||
raise BuildError(e.output.decode())
|
||||
raise BuildError('Could not call make command: ' + str(e))
|
||||
_, stderr = proc.communicate()
|
||||
if proc.returncode != 0:
|
||||
raise BuildError(stderr.decode())
|
||||
if stderr: # likely only due to build warnings
|
||||
print(stderr.decode())
|
||||
|
||||
def linux_bin(self, params, timeout, build_dir, outfile):
|
||||
def linux_bin(self, params, timeout, build_dir):
|
||||
"""Runs the Linux UML binary. Must be named 'linux'."""
|
||||
linux_bin = './linux'
|
||||
if build_dir:
|
||||
linux_bin = os.path.join(build_dir, 'linux')
|
||||
outfile = get_outfile_path(build_dir)
|
||||
with open(outfile, 'w') as output:
|
||||
process = subprocess.Popen([linux_bin] + params,
|
||||
stdout=output,
|
||||
stderr=subprocess.STDOUT)
|
||||
process.wait(timeout)
|
||||
|
||||
|
||||
def get_kconfig_path(build_dir):
|
||||
kconfig_path = KCONFIG_PATH
|
||||
if build_dir:
|
||||
kconfig_path = os.path.join(build_dir, KCONFIG_PATH)
|
||||
return kconfig_path
|
||||
|
||||
def get_kunitconfig_path(build_dir):
|
||||
kunitconfig_path = KUNITCONFIG_PATH
|
||||
if build_dir:
|
||||
kunitconfig_path = os.path.join(build_dir, KUNITCONFIG_PATH)
|
||||
return kunitconfig_path
|
||||
|
||||
def get_outfile_path(build_dir):
|
||||
outfile_path = OUTFILE_PATH
|
||||
if build_dir:
|
||||
outfile_path = os.path.join(build_dir, OUTFILE_PATH)
|
||||
return outfile_path
|
||||
|
||||
class LinuxSourceTree(object):
|
||||
"""Represents a Linux kernel source tree with KUnit tests."""
|
||||
|
||||
def __init__(self):
|
||||
self._kconfig = kunit_config.Kconfig()
|
||||
self._kconfig.read_from_file(kunitconfig_path)
|
||||
self._ops = LinuxSourceTreeOperations()
|
||||
signal.signal(signal.SIGINT, self.signal_handler)
|
||||
|
||||
|
|
@ -123,6 +140,16 @@ class LinuxSourceTree(object):
|
|||
return False
|
||||
return True
|
||||
|
||||
def create_kunitconfig(self, build_dir, defconfig=DEFAULT_KUNITCONFIG_PATH):
|
||||
kunitconfig_path = get_kunitconfig_path(build_dir)
|
||||
if not os.path.exists(kunitconfig_path):
|
||||
shutil.copyfile(defconfig, kunitconfig_path)
|
||||
|
||||
def read_kunitconfig(self, build_dir):
|
||||
kunitconfig_path = get_kunitconfig_path(build_dir)
|
||||
self._kconfig = kunit_config.Kconfig()
|
||||
self._kconfig.read_from_file(kunitconfig_path)
|
||||
|
||||
def validate_config(self, build_dir):
|
||||
kconfig_path = get_kconfig_path(build_dir)
|
||||
validated_kconfig = kunit_config.Kconfig()
|
||||
|
|
@ -178,8 +205,8 @@ class LinuxSourceTree(object):
|
|||
|
||||
def run_kernel(self, args=[], build_dir='', timeout=None):
|
||||
args.extend(['mem=1G'])
|
||||
outfile = 'test.log'
|
||||
self._ops.linux_bin(args, timeout, build_dir, outfile)
|
||||
self._ops.linux_bin(args, timeout, build_dir)
|
||||
outfile = get_outfile_path(build_dir)
|
||||
subprocess.call(['stty', 'sane'])
|
||||
with open(outfile, 'r') as file:
|
||||
for line in file:
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ from collections import namedtuple
|
|||
from datetime import datetime
|
||||
from enum import Enum, auto
|
||||
from functools import reduce
|
||||
from typing import List
|
||||
from typing import List, Optional, Tuple
|
||||
|
||||
TestResult = namedtuple('TestResult', ['status','suites','log'])
|
||||
|
||||
|
|
@ -54,6 +54,7 @@ kunit_end_re = re.compile('(List of all partitions:|'
|
|||
def isolate_kunit_output(kernel_output):
|
||||
started = False
|
||||
for line in kernel_output:
|
||||
line = line.rstrip() # line always has a trailing \n
|
||||
if kunit_start_re.search(line):
|
||||
prefix_len = len(line.split('TAP version')[0])
|
||||
started = True
|
||||
|
|
@ -65,7 +66,7 @@ def isolate_kunit_output(kernel_output):
|
|||
|
||||
def raw_output(kernel_output):
|
||||
for line in kernel_output:
|
||||
print(line)
|
||||
print(line.rstrip())
|
||||
|
||||
DIVIDER = '=' * 60
|
||||
|
||||
|
|
@ -151,7 +152,7 @@ def parse_diagnostic(lines: List[str], test_case: TestCase) -> bool:
|
|||
else:
|
||||
return False
|
||||
|
||||
def parse_test_case(lines: List[str]) -> TestCase:
|
||||
def parse_test_case(lines: List[str]) -> Optional[TestCase]:
|
||||
test_case = TestCase()
|
||||
save_non_diagnositic(lines, test_case)
|
||||
while parse_diagnostic(lines, test_case):
|
||||
|
|
@ -163,7 +164,7 @@ def parse_test_case(lines: List[str]) -> TestCase:
|
|||
|
||||
SUBTEST_HEADER = re.compile(r'^[\s]+# Subtest: (.*)$')
|
||||
|
||||
def parse_subtest_header(lines: List[str]) -> str:
|
||||
def parse_subtest_header(lines: List[str]) -> Optional[str]:
|
||||
consume_non_diagnositic(lines)
|
||||
if not lines:
|
||||
return None
|
||||
|
|
@ -176,7 +177,7 @@ def parse_subtest_header(lines: List[str]) -> str:
|
|||
|
||||
SUBTEST_PLAN = re.compile(r'[\s]+[0-9]+\.\.([0-9]+)')
|
||||
|
||||
def parse_subtest_plan(lines: List[str]) -> int:
|
||||
def parse_subtest_plan(lines: List[str]) -> Optional[int]:
|
||||
consume_non_diagnositic(lines)
|
||||
match = SUBTEST_PLAN.match(lines[0])
|
||||
if match:
|
||||
|
|
@ -230,7 +231,7 @@ def bubble_up_test_case_errors(test_suite: TestSuite) -> TestStatus:
|
|||
max_test_case_status = bubble_up_errors(lambda x: x.status, test_suite.cases)
|
||||
return max_status(max_test_case_status, test_suite.status)
|
||||
|
||||
def parse_test_suite(lines: List[str], expected_suite_index: int) -> TestSuite:
|
||||
def parse_test_suite(lines: List[str], expected_suite_index: int) -> Optional[TestSuite]:
|
||||
if not lines:
|
||||
return None
|
||||
consume_non_diagnositic(lines)
|
||||
|
|
@ -271,7 +272,7 @@ def parse_tap_header(lines: List[str]) -> bool:
|
|||
|
||||
TEST_PLAN = re.compile(r'[0-9]+\.\.([0-9]+)')
|
||||
|
||||
def parse_test_plan(lines: List[str]) -> int:
|
||||
def parse_test_plan(lines: List[str]) -> Optional[int]:
|
||||
consume_non_diagnositic(lines)
|
||||
match = TEST_PLAN.match(lines[0])
|
||||
if match:
|
||||
|
|
@ -310,7 +311,7 @@ def parse_test_result(lines: List[str]) -> TestResult:
|
|||
else:
|
||||
return TestResult(TestStatus.NO_TESTS, [], lines)
|
||||
|
||||
def print_and_count_results(test_result: TestResult) -> None:
|
||||
def print_and_count_results(test_result: TestResult) -> Tuple[int, int, int]:
|
||||
total_tests = 0
|
||||
failed_tests = 0
|
||||
crashed_tests = 0
|
||||
|
|
|
|||
|
|
@ -102,7 +102,7 @@ class KUnitParserTest(unittest.TestCase):
|
|||
'test_data/test_output_isolated_correctly.log')
|
||||
file = open(log_path)
|
||||
result = kunit_parser.isolate_kunit_output(file.readlines())
|
||||
self.assertContains('TAP version 14\n', result)
|
||||
self.assertContains('TAP version 14', result)
|
||||
self.assertContains(' # Subtest: example', result)
|
||||
self.assertContains(' 1..2', result)
|
||||
self.assertContains(' ok 1 - example_simple_test', result)
|
||||
|
|
@ -115,7 +115,7 @@ class KUnitParserTest(unittest.TestCase):
|
|||
'test_data/test_pound_sign.log')
|
||||
with open(log_path) as file:
|
||||
result = kunit_parser.isolate_kunit_output(file.readlines())
|
||||
self.assertContains('TAP version 14\n', result)
|
||||
self.assertContains('TAP version 14', result)
|
||||
self.assertContains(' # Subtest: kunit-resource-test', result)
|
||||
self.assertContains(' 1..5', result)
|
||||
self.assertContains(' ok 1 - kunit_resource_test_init_resources', result)
|
||||
|
|
|
|||
Loading…
Reference in New Issue