d519329f72
The estimated utilization of a task is currently updated every time the task is dequeued. However, to keep overheads under control, PELT signals are effectively updated at maximum once every 1ms. Thus, for really short running tasks, it can happen that their util_avg value has not been updates since their last enqueue. If such tasks are also frequently running tasks (e.g. the kind of workload generated by hackbench) it can also happen that their util_avg is updated only every few activations. This means that updating util_est at every dequeue potentially introduces not necessary overheads and it's also conceptually wrong if the util_avg signal has never been updated during a task activation. Let's introduce a throttling mechanism on task's util_est updates to sync them with util_avg updates. To make the solution memory efficient, both in terms of space and load/store operations, we encode a synchronization flag into the LSB of util_est.enqueued. This makes util_est an even values only metric, which is still considered good enough for its purpose. The synchronization bit is (re)set by __update_load_avg_se() once the PELT signal of a task has been updated during its last activation. Such a throttling mechanism allows to keep under control util_est overheads in the wakeup hot path, thus making it a suitable mechanism which can be enabled also on high-intensity workload systems. Thus, this now switches on by default the estimation utilization scheduler feature. Suggested-by: Chris Redpath <chris.redpath@arm.com> Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Cc: Dietmar Eggemann <dietmar.eggemann@arm.com> Cc: Joel Fernandes <joelaf@google.com> Cc: Juri Lelli <juri.lelli@redhat.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Morten Rasmussen <morten.rasmussen@arm.com> Cc: Paul Turner <pjt@google.com> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Rafael J . Wysocki <rafael.j.wysocki@intel.com> Cc: Steve Muckle <smuckle@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Todd Kjos <tkjos@android.com> Cc: Vincent Guittot <vincent.guittot@linaro.org> Cc: Viresh Kumar <viresh.kumar@linaro.org> Link: http://lkml.kernel.org/r/20180309095245.11071-5-patrick.bellasi@arm.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
93 lines
2.4 KiB
C
93 lines
2.4 KiB
C
/* SPDX-License-Identifier: GPL-2.0 */
|
|
/*
|
|
* Only give sleepers 50% of their service deficit. This allows
|
|
* them to run sooner, but does not allow tons of sleepers to
|
|
* rip the spread apart.
|
|
*/
|
|
SCHED_FEAT(GENTLE_FAIR_SLEEPERS, true)
|
|
|
|
/*
|
|
* Place new tasks ahead so that they do not starve already running
|
|
* tasks
|
|
*/
|
|
SCHED_FEAT(START_DEBIT, true)
|
|
|
|
/*
|
|
* Prefer to schedule the task we woke last (assuming it failed
|
|
* wakeup-preemption), since its likely going to consume data we
|
|
* touched, increases cache locality.
|
|
*/
|
|
SCHED_FEAT(NEXT_BUDDY, false)
|
|
|
|
/*
|
|
* Prefer to schedule the task that ran last (when we did
|
|
* wake-preempt) as that likely will touch the same data, increases
|
|
* cache locality.
|
|
*/
|
|
SCHED_FEAT(LAST_BUDDY, true)
|
|
|
|
/*
|
|
* Consider buddies to be cache hot, decreases the likelyness of a
|
|
* cache buddy being migrated away, increases cache locality.
|
|
*/
|
|
SCHED_FEAT(CACHE_HOT_BUDDY, true)
|
|
|
|
/*
|
|
* Allow wakeup-time preemption of the current task:
|
|
*/
|
|
SCHED_FEAT(WAKEUP_PREEMPTION, true)
|
|
|
|
SCHED_FEAT(HRTICK, false)
|
|
SCHED_FEAT(DOUBLE_TICK, false)
|
|
SCHED_FEAT(LB_BIAS, true)
|
|
|
|
/*
|
|
* Decrement CPU capacity based on time not spent running tasks
|
|
*/
|
|
SCHED_FEAT(NONTASK_CAPACITY, true)
|
|
|
|
/*
|
|
* Queue remote wakeups on the target CPU and process them
|
|
* using the scheduler IPI. Reduces rq->lock contention/bounces.
|
|
*/
|
|
SCHED_FEAT(TTWU_QUEUE, true)
|
|
|
|
/*
|
|
* When doing wakeups, attempt to limit superfluous scans of the LLC domain.
|
|
*/
|
|
SCHED_FEAT(SIS_AVG_CPU, false)
|
|
SCHED_FEAT(SIS_PROP, true)
|
|
|
|
/*
|
|
* Issue a WARN when we do multiple update_rq_clock() calls
|
|
* in a single rq->lock section. Default disabled because the
|
|
* annotations are not complete.
|
|
*/
|
|
SCHED_FEAT(WARN_DOUBLE_CLOCK, false)
|
|
|
|
#ifdef HAVE_RT_PUSH_IPI
|
|
/*
|
|
* In order to avoid a thundering herd attack of CPUs that are
|
|
* lowering their priorities at the same time, and there being
|
|
* a single CPU that has an RT task that can migrate and is waiting
|
|
* to run, where the other CPUs will try to take that CPUs
|
|
* rq lock and possibly create a large contention, sending an
|
|
* IPI to that CPU and let that CPU push the RT task to where
|
|
* it should go may be a better scenario.
|
|
*/
|
|
SCHED_FEAT(RT_PUSH_IPI, true)
|
|
#endif
|
|
|
|
SCHED_FEAT(RT_RUNTIME_SHARE, true)
|
|
SCHED_FEAT(LB_MIN, false)
|
|
SCHED_FEAT(ATTACH_AGE_LOAD, true)
|
|
|
|
SCHED_FEAT(WA_IDLE, true)
|
|
SCHED_FEAT(WA_WEIGHT, true)
|
|
SCHED_FEAT(WA_BIAS, true)
|
|
|
|
/*
|
|
* UtilEstimation. Use estimated CPU utilization.
|
|
*/
|
|
SCHED_FEAT(UTIL_EST, true)
|