redonkable
/
remarkable-linux
1
0
Fork 0
Code Releases Activity

rcu: Eliminate NOCBs CPU-state Kconfig options 

The CONFIG_RCU_NOCB_CPU_ALL, CONFIG_RCU_NOCB_CPU_NONE, and
CONFIG_RCU_NOCB_CPU_ZERO Kconfig options are used only in testing and
are redundant with the rcu_nocbs= boot parameter.  This commit therefore
removes these three Kconfig options and adjusts the rcutorture scripts
to use the boot parameter instead.

Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
zero-colors
Paul E. McKenney 2017-05-15 16:26:34 -07:00
parent ae91aa0adb
commit 44c65ff2e3
11 changed files with 26 additions and 131 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
Documentation/kernel-per-CPU-kthreads.txt
View File

@ -109,13 +109,12 @@ SCHED_SOFTIRQ: Do all of the following:
on that CPU. If a thread that expects to run on the de-jittered on that CPU. If a thread that expects to run on the de-jittered
CPU awakens, the scheduler will send an IPI that can result in CPU awakens, the scheduler will send an IPI that can result in
a subsequent SCHED_SOFTIRQ. a subsequent SCHED_SOFTIRQ.
2. Build with CONFIG_RCU_NOCB_CPU=y, CONFIG_RCU_NOCB_CPU_ALL=y, 2. CONFIG_NO_HZ_FULL=y and ensure that the CPU to be de-jittered
CONFIG_NO_HZ_FULL=y, and, in addition, ensure that the CPU is marked as an adaptive-ticks CPU using the "nohz_full="
to be de-jittered is marked as an adaptive-ticks CPU using the boot parameter. This reduces the number of scheduler-clock
"nohz_full=" boot parameter. This reduces the number of interrupts that the de-jittered CPU receives, minimizing its
scheduler-clock interrupts that the de-jittered CPU receives, chances of being selected to do the load balancing work that
minimizing its chances of being selected to do the load balancing runs in SCHED_SOFTIRQ context.
work that runs in SCHED_SOFTIRQ context.
3. To the extent possible, keep the CPU out of the kernel when it 3. To the extent possible, keep the CPU out of the kernel when it
is non-idle, for example, by avoiding system calls and by is non-idle, for example, by avoiding system calls and by
forcing both kernel threads and interrupts to execute elsewhere. forcing both kernel threads and interrupts to execute elsewhere.
@ -135,11 +134,10 @@ HRTIMER_SOFTIRQ: Do all of the following:
RCU_SOFTIRQ: Do at least one of the following: RCU_SOFTIRQ: Do at least one of the following:
1. Offload callbacks and keep the CPU in either dyntick-idle or 1. Offload callbacks and keep the CPU in either dyntick-idle or
adaptive-ticks state by doing all of the following: adaptive-ticks state by doing all of the following:
a. Build with CONFIG_RCU_NOCB_CPU=y, CONFIG_RCU_NOCB_CPU_ALL=y, a. CONFIG_NO_HZ_FULL=y and ensure that the CPU to be
CONFIG_NO_HZ_FULL=y, and, in addition ensure that the CPU de-jittered is marked as an adaptive-ticks CPU using the
to be de-jittered is marked as an adaptive-ticks CPU using "nohz_full=" boot parameter. Bind the rcuo kthreads to
the "nohz_full=" boot parameter. Bind the rcuo kthreads housekeeping CPUs, which can tolerate OS jitter.
to housekeeping CPUs, which can tolerate OS jitter.
b. To the extent possible, keep the CPU out of the kernel b. To the extent possible, keep the CPU out of the kernel
when it is non-idle, for example, by avoiding system when it is non-idle, for example, by avoiding system
calls and by forcing both kernel threads and interrupts calls and by forcing both kernel threads and interrupts
@ -236,11 +234,10 @@ To reduce its OS jitter, do at least one of the following:
is feasible only if your workload never requires RCU priority is feasible only if your workload never requires RCU priority
boosting, for example, if you ensure frequent idle time on all boosting, for example, if you ensure frequent idle time on all
CPUs that might execute within the kernel. CPUs that might execute within the kernel.
3. Build with CONFIG_RCU_NOCB_CPU=y and CONFIG_RCU_NOCB_CPU_ALL=y, 3. Build with CONFIG_RCU_NOCB_CPU=y and boot with the rcu_nocbs=
which offloads all RCU callbacks to kthreads that can be moved boot parameter offloading RCU callbacks from all CPUs susceptible
off of CPUs susceptible to OS jitter. This approach prevents the to OS jitter. This approach prevents the rcuc/%u kthreads from
rcuc/%u kthreads from having any work to do, so that they are having any work to do, so that they are never awakened.
never awakened.
4. Ensure that the CPU never enters the kernel, and, in particular, 4. Ensure that the CPU never enters the kernel, and, in particular,
avoid initiating any CPU hotplug operations on this CPU. This is avoid initiating any CPU hotplug operations on this CPU. This is
another way of preventing any callbacks from being queued on the another way of preventing any callbacks from being queued on the

29
Documentation/timers/NO_HZ.txt
View File

@ -194,32 +194,9 @@ that the RCU callbacks are processed in a timely fashion.
Another approach is to offload RCU callback processing to "rcuo" kthreads Another approach is to offload RCU callback processing to "rcuo" kthreads
using the CONFIG_RCU_NOCB_CPU=y Kconfig option. The specific CPUs to using the CONFIG_RCU_NOCB_CPU=y Kconfig option. The specific CPUs to
offload may be selected via several methods: offload may be selected using The "rcu_nocbs=" kernel boot parameter,
which takes a comma-separated list of CPUs and CPU ranges, for example,
1. One of three mutually exclusive Kconfig options specify a "1,3-5" selects CPUs 1, 3, 4, and 5.
build-time default for the CPUs to offload:
a. The CONFIG_RCU_NOCB_CPU_NONE=y Kconfig option results in
no CPUs being offloaded.
b. The CONFIG_RCU_NOCB_CPU_ZERO=y Kconfig option causes
CPU 0 to be offloaded.
c. The CONFIG_RCU_NOCB_CPU_ALL=y Kconfig option causes all
CPUs to be offloaded. Note that the callbacks will be
offloaded to "rcuo" kthreads, and that those kthreads
will in fact run on some CPU. However, this approach
gives fine-grained control on exactly which CPUs the
callbacks run on, along with their scheduling priority
(including the default of SCHED_OTHER), and it further
allows this control to be varied dynamically at runtime.
2. The "rcu_nocbs=" kernel boot parameter, which takes a comma-separated
list of CPUs and CPU ranges, for example, "1,3-5" selects CPUs 1,
3, 4, and 5. The specified CPUs will be offloaded in addition to
any CPUs specified as offloaded by CONFIG_RCU_NOCB_CPU_ZERO=y or
CONFIG_RCU_NOCB_CPU_ALL=y. This means that the "rcu_nocbs=" boot
parameter has no effect for kernels built with RCU_NOCB_CPU_ALL=y.
The offloaded CPUs will never queue RCU callbacks, and therefore RCU The offloaded CPUs will never queue RCU callbacks, and therefore RCU
never prevents offloaded CPUs from entering either dyntick-idle mode never prevents offloaded CPUs from entering either dyntick-idle mode

53
init/Kconfig
View File

@ -709,59 +709,6 @@ config RCU_NOCB_CPU
Say Y here if you want to help to debug reduced OS jitter. Say Y here if you want to help to debug reduced OS jitter.
Say N here if you are unsure. Say N here if you are unsure.
choice
prompt "Build-forced no-CBs CPUs"
default RCU_NOCB_CPU_NONE
depends on RCU_NOCB_CPU
help
This option allows no-CBs CPUs (whose RCU callbacks are invoked
from kthreads rather than from softirq context) to be specified
at build time. Additional no-CBs CPUs may be specified by
the rcu_nocbs= boot parameter.
config RCU_NOCB_CPU_NONE
bool "No build_forced no-CBs CPUs"
help
This option does not force any of the CPUs to be no-CBs CPUs.
Only CPUs designated by the rcu_nocbs= boot parameter will be
no-CBs CPUs, whose RCU callbacks will be invoked by per-CPU
kthreads whose names begin with "rcuo". All other CPUs will
invoke their own RCU callbacks in softirq context.
Select this option if you want to choose no-CBs CPUs at
boot time, for example, to allow testing of different no-CBs
configurations without having to rebuild the kernel each time.
config RCU_NOCB_CPU_ZERO
bool "CPU 0 is a build_forced no-CBs CPU"
help
This option forces CPU 0 to be a no-CBs CPU, so that its RCU
callbacks are invoked by a per-CPU kthread whose name begins
with "rcuo". Additional CPUs may be designated as no-CBs
CPUs using the rcu_nocbs= boot parameter will be no-CBs CPUs.
All other CPUs will invoke their own RCU callbacks in softirq
context.
Select this if CPU 0 needs to be a no-CBs CPU for real-time
or energy-efficiency reasons, but the real reason it exists
is to ensure that randconfig testing covers mixed systems.
config RCU_NOCB_CPU_ALL
bool "All CPUs are build_forced no-CBs CPUs"
help
This option forces all CPUs to be no-CBs CPUs. The rcu_nocbs=
boot parameter will be ignored. All CPUs' RCU callbacks will
be executed in the context of per-CPU rcuo kthreads created for
this purpose. Assuming that the kthreads whose names start with
"rcuo" are bound to "housekeeping" CPUs, this reduces OS jitter
on the remaining CPUs, but might decrease memory locality during
RCU-callback invocation, thus potentially degrading throughput.
Select this if all CPUs need to be no-CBs CPUs for real-time
or energy-efficiency reasons.
endchoice
endmenu # "RCU Subsystem" endmenu # "RCU Subsystem"
config BUILD_BIN2C config BUILD_BIN2C

4
kernel/rcu/rcu.h
View File

@ -564,9 +564,7 @@ void rcu_bh_force_quiescent_state(void);
void rcu_sched_force_quiescent_state(void); void rcu_sched_force_quiescent_state(void);
#endif /* #else #ifdef CONFIG_TINY_RCU */ #endif /* #else #ifdef CONFIG_TINY_RCU */
#if defined(CONFIG_RCU_NOCB_CPU_ALL) #ifdef CONFIG_RCU_NOCB_CPU
static inline bool rcu_is_nocb_cpu(int cpu) { return true; }
#elif defined(CONFIG_RCU_NOCB_CPU)
bool rcu_is_nocb_cpu(int cpu); bool rcu_is_nocb_cpu(int cpu);
#else #else
static inline bool rcu_is_nocb_cpu(int cpu) { return false; } static inline bool rcu_is_nocb_cpu(int cpu) { return false; }

27
kernel/rcu/tree_plugin.h
View File

@ -1296,8 +1296,7 @@ static void rcu_prepare_kthreads(int cpu)
int rcu_needs_cpu(u64 basemono, u64 *nextevt) int rcu_needs_cpu(u64 basemono, u64 *nextevt)
{ {
*nextevt = KTIME_MAX; *nextevt = KTIME_MAX;
return IS_ENABLED(CONFIG_RCU_NOCB_CPU_ALL) return rcu_cpu_has_callbacks(NULL);
? 0 : rcu_cpu_has_callbacks(NULL);
} }
/* /*
@ -1409,10 +1408,6 @@ int rcu_needs_cpu(u64 basemono, u64 *nextevt)
unsigned long dj; unsigned long dj;
RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_needs_cpu() invoked with irqs enabled!!!"); RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_needs_cpu() invoked with irqs enabled!!!");
if (IS_ENABLED(CONFIG_RCU_NOCB_CPU_ALL)) {
*nextevt = KTIME_MAX;
return 0;
}
/* Snapshot to detect later posting of non-lazy callback. */ /* Snapshot to detect later posting of non-lazy callback. */
rdtp->nonlazy_posted_snap = rdtp->nonlazy_posted; rdtp->nonlazy_posted_snap = rdtp->nonlazy_posted;
@ -1462,8 +1457,7 @@ static void rcu_prepare_for_idle(void)
int tne; int tne;
RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_prepare_for_idle() invoked with irqs enabled!!!"); RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_prepare_for_idle() invoked with irqs enabled!!!");
if (IS_ENABLED(CONFIG_RCU_NOCB_CPU_ALL) || if (rcu_is_nocb_cpu(smp_processor_id()))
rcu_is_nocb_cpu(smp_processor_id()))
return; return;
/* Handle nohz enablement switches conservatively. */ /* Handle nohz enablement switches conservatively. */
@ -1518,8 +1512,7 @@ static void rcu_prepare_for_idle(void)
static void rcu_cleanup_after_idle(void) static void rcu_cleanup_after_idle(void)
{ {
RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_cleanup_after_idle() invoked with irqs enabled!!!"); RCU_LOCKDEP_WARN(!irqs_disabled(), "rcu_cleanup_after_idle() invoked with irqs enabled!!!");
if (IS_ENABLED(CONFIG_RCU_NOCB_CPU_ALL) || if (rcu_is_nocb_cpu(smp_processor_id()))
rcu_is_nocb_cpu(smp_processor_id()))
return; return;
if (rcu_try_advance_all_cbs()) if (rcu_try_advance_all_cbs())
invoke_rcu_core(); invoke_rcu_core();
@ -1786,7 +1779,6 @@ static void rcu_init_one_nocb(struct rcu_node *rnp)
init_swait_queue_head(&rnp->nocb_gp_wq[1]); init_swait_queue_head(&rnp->nocb_gp_wq[1]);
} }
#ifndef CONFIG_RCU_NOCB_CPU_ALL
/* Is the specified CPU a no-CBs CPU? */ /* Is the specified CPU a no-CBs CPU? */
bool rcu_is_nocb_cpu(int cpu) bool rcu_is_nocb_cpu(int cpu)
{ {
@ -1794,7 +1786,6 @@ bool rcu_is_nocb_cpu(int cpu)
return cpumask_test_cpu(cpu, rcu_nocb_mask); return cpumask_test_cpu(cpu, rcu_nocb_mask);
return false; return false;
} }
#endif /* #ifndef CONFIG_RCU_NOCB_CPU_ALL */
/* /*
* Kick the leader kthread for this NOCB group. * Kick the leader kthread for this NOCB group.
@ -2253,10 +2244,6 @@ void __init rcu_init_nohz(void)
bool need_rcu_nocb_mask = true; bool need_rcu_nocb_mask = true;
struct rcu_state *rsp; struct rcu_state *rsp;
#ifdef CONFIG_RCU_NOCB_CPU_NONE
need_rcu_nocb_mask = false;
#endif /* #ifndef CONFIG_RCU_NOCB_CPU_NONE */
#if defined(CONFIG_NO_HZ_FULL) #if defined(CONFIG_NO_HZ_FULL)
if (tick_nohz_full_running && cpumask_weight(tick_nohz_full_mask)) if (tick_nohz_full_running && cpumask_weight(tick_nohz_full_mask))
need_rcu_nocb_mask = true; need_rcu_nocb_mask = true;
@ -2272,14 +2259,6 @@ void __init rcu_init_nohz(void)
if (!have_rcu_nocb_mask) if (!have_rcu_nocb_mask)
return; return;
#ifdef CONFIG_RCU_NOCB_CPU_ZERO
pr_info("\tOffload RCU callbacks from CPU 0\n");
cpumask_set_cpu(0, rcu_nocb_mask);
#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ZERO */
#ifdef CONFIG_RCU_NOCB_CPU_ALL
pr_info("\tOffload RCU callbacks from all CPUs\n");
cpumask_copy(rcu_nocb_mask, cpu_possible_mask);
#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ALL */
#if defined(CONFIG_NO_HZ_FULL) #if defined(CONFIG_NO_HZ_FULL)
if (tick_nohz_full_running) if (tick_nohz_full_running)
cpumask_or(rcu_nocb_mask, rcu_nocb_mask, tick_nohz_full_mask); cpumask_or(rcu_nocb_mask, rcu_nocb_mask, tick_nohz_full_mask);

1
tools/testing/selftests/rcutorture/configs/rcu/TREE01
View File

@ -12,7 +12,6 @@ CONFIG_HOTPLUG_CPU=y
CONFIG_MAXSMP=y CONFIG_MAXSMP=y
CONFIG_CPUMASK_OFFSTACK=y CONFIG_CPUMASK_OFFSTACK=y
CONFIG_RCU_NOCB_CPU=y CONFIG_RCU_NOCB_CPU=y
CONFIG_RCU_NOCB_CPU_ZERO=y
CONFIG_DEBUG_LOCK_ALLOC=n CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_RCU_BOOST=n CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n CONFIG_DEBUG_OBJECTS_RCU_HEAD=n

1
tools/testing/selftests/rcutorture/configs/rcu/TREE01.boot
View File

@ -2,3 +2,4 @@ rcutorture.torture_type=rcu_bh maxcpus=8
rcutree.gp_preinit_delay=3 rcutree.gp_preinit_delay=3
rcutree.gp_init_delay=3 rcutree.gp_init_delay=3
rcutree.gp_cleanup_delay=3 rcutree.gp_cleanup_delay=3
rcu_nocbs=0

1
tools/testing/selftests/rcutorture/configs/rcu/TREE05
View File

@ -13,7 +13,6 @@ CONFIG_HOTPLUG_CPU=y
CONFIG_RCU_FANOUT=6 CONFIG_RCU_FANOUT=6
CONFIG_RCU_FANOUT_LEAF=6 CONFIG_RCU_FANOUT_LEAF=6
CONFIG_RCU_NOCB_CPU=y CONFIG_RCU_NOCB_CPU=y
CONFIG_RCU_NOCB_CPU_NONE=y
CONFIG_DEBUG_LOCK_ALLOC=y CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_PROVE_LOCKING=y CONFIG_PROVE_LOCKING=y
#CHECK#CONFIG_PROVE_RCU=y #CHECK#CONFIG_PROVE_RCU=y

1
tools/testing/selftests/rcutorture/configs/rcu/TREE08
View File

@ -15,7 +15,6 @@ CONFIG_HIBERNATION=n
CONFIG_RCU_FANOUT=3 CONFIG_RCU_FANOUT=3
CONFIG_RCU_FANOUT_LEAF=2 CONFIG_RCU_FANOUT_LEAF=2
CONFIG_RCU_NOCB_CPU=y CONFIG_RCU_NOCB_CPU=y
CONFIG_RCU_NOCB_CPU_ALL=y
CONFIG_DEBUG_LOCK_ALLOC=n CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_PROVE_LOCKING=n CONFIG_PROVE_LOCKING=n
CONFIG_RCU_BOOST=n CONFIG_RCU_BOOST=n

1
tools/testing/selftests/rcutorture/configs/rcu/TREE08.boot
View File

@ -2,3 +2,4 @@ rcutorture.torture_type=sched
rcupdate.rcu_self_test=1 rcupdate.rcu_self_test=1
rcupdate.rcu_self_test_sched=1 rcupdate.rcu_self_test_sched=1
rcutree.rcu_fanout_exact=1 rcutree.rcu_fanout_exact=1
rcu_nocbs=0-7

8
tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt
View File

@ -16,11 +16,9 @@ CONFIG_PROVE_RCU -- Hardwired to CONFIG_PROVE_LOCKING.
CONFIG_RCU_BOOST -- one of PREEMPT_RCU. CONFIG_RCU_BOOST -- one of PREEMPT_RCU.
CONFIG_RCU_FANOUT -- Cover hierarchy, but overlap with others. CONFIG_RCU_FANOUT -- Cover hierarchy, but overlap with others.
CONFIG_RCU_FANOUT_LEAF -- Do one non-default. CONFIG_RCU_FANOUT_LEAF -- Do one non-default.
CONFIG_RCU_FAST_NO_HZ -- Do one, but not with CONFIG_RCU_NOCB_CPU_ALL. CONFIG_RCU_FAST_NO_HZ -- Do one, but not with all nohz_full CPUs.
CONFIG_RCU_NOCB_CPU -- Do three, see below. CONFIG_RCU_NOCB_CPU -- Do three, one with no rcu_nocbs CPUs, one with
CONFIG_RCU_NOCB_CPU_ALL -- Do one. rcu_nocbs=0, and one with all rcu_nocbs CPUs.
CONFIG_RCU_NOCB_CPU_NONE -- Do one.
CONFIG_RCU_NOCB_CPU_ZERO -- Do one.
CONFIG_RCU_TRACE -- Do half. CONFIG_RCU_TRACE -- Do half.
CONFIG_SMP -- Need one !SMP for PREEMPT_RCU. CONFIG_SMP -- Need one !SMP for PREEMPT_RCU.
CONFIG_RCU_EXPERT=n -- Do a few, but these have to be vanilla configurations. CONFIG_RCU_EXPERT=n -- Do a few, but these have to be vanilla configurations.