1
0
Fork 0
alistair23-linux/kernel/time
Konstantin Khlebnikov d1318034e9 clocksource: Prevent double add_timer_on() for watchdog_timer
commit febac332a8 upstream.

Kernel crashes inside QEMU/KVM are observed:

  kernel BUG at kernel/time/timer.c:1154!
  BUG_ON(timer_pending(timer) || !timer->function) in add_timer_on().

At the same time another cpu got:

  general protection fault: 0000 [#1] SMP PTI of poinson pointer 0xdead000000000200 in:

  __hlist_del at include/linux/list.h:681
  (inlined by) detach_timer at kernel/time/timer.c:818
  (inlined by) expire_timers at kernel/time/timer.c:1355
  (inlined by) __run_timers at kernel/time/timer.c:1686
  (inlined by) run_timer_softirq at kernel/time/timer.c:1699

Unfortunately kernel logs are badly scrambled, stacktraces are lost.

Printing the timer->function before the BUG_ON() pointed to
clocksource_watchdog().

The execution of clocksource_watchdog() can race with a sequence of
clocksource_stop_watchdog() .. clocksource_start_watchdog():

expire_timers()
 detach_timer(timer, true);
  timer->entry.pprev = NULL;
 raw_spin_unlock_irq(&base->lock);
 call_timer_fn
  clocksource_watchdog()

					clocksource_watchdog_kthread() or
					clocksource_unbind()

					spin_lock_irqsave(&watchdog_lock, flags);
					clocksource_stop_watchdog();
					 del_timer(&watchdog_timer);
					 watchdog_running = 0;
					spin_unlock_irqrestore(&watchdog_lock, flags);

					spin_lock_irqsave(&watchdog_lock, flags);
					clocksource_start_watchdog();
					 add_timer_on(&watchdog_timer, ...);
					 watchdog_running = 1;
					spin_unlock_irqrestore(&watchdog_lock, flags);

  spin_lock(&watchdog_lock);
  add_timer_on(&watchdog_timer, ...);
   BUG_ON(timer_pending(timer) || !timer->function);
    timer_pending() -> true
    BUG()

I.e. inside clocksource_watchdog() watchdog_timer could be already armed.

Check timer_pending() before calling add_timer_on(). This is sufficient as
all operations are synchronized by watchdog_lock.

Fixes: 75c5158f70 ("timekeeping: Update clocksource with stop_machine")
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/158048693917.4378.13823603769948933793.stgit@buzz
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2020-02-11 04:35:54 -08:00
..
Kconfig treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
Makefile timekeeping: Provide a generic update_vsyscall() implementation 2019-06-22 21:21:06 +02:00
alarmtimer.c alarmtimer: Unregister wakeup source when module get fails 2020-02-11 04:35:20 -08:00
clockevents.c tick: Remove outgoing CPU from broadcast masks 2019-03-23 18:26:43 +01:00
clocksource.c clocksource: Prevent double add_timer_on() for watchdog_timer 2020-02-11 04:35:54 -08:00
hrtimer.c hrtimer: Annotate lockless access to timer->state 2020-01-04 19:18:41 +01:00
itimer.c posix-cpu-timers: Switch thread group sampling to array 2019-08-28 11:50:39 +02:00
jiffies.c Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-05-06 14:50:46 -07:00
ntp.c ntp/y2038: Remove incorrect time_t truncation 2019-11-12 08:13:44 +01:00
ntp_internal.h ntp: Audit NTP parameters adjustment 2019-04-15 18:14:01 -04:00
posix-clock.c ptp: fix the race between the release of ptp_clock and cdev 2020-01-04 19:18:48 +01:00
posix-cpu-timers.c posix-cpu-timers: Fix two trivial comments 2019-10-23 14:48:24 +02:00
posix-stubs.c y2038: syscalls: rename y2038 compat syscalls 2019-02-07 00:13:27 +01:00
posix-timers.c hrtimer: Improve comments on handling priority inversion against softirq kthread 2019-08-20 22:05:46 +02:00
posix-timers.h posix-timers: Use a callback for cancel synchronization on PREEMPT_RT 2019-08-20 22:05:46 +02:00
sched_clock.c timers/sched_clock: Include local timekeeping.h for missing declarations 2019-10-23 14:48:23 +02:00
test_udelay.c time/debug: Remove license boilerplate 2018-11-23 11:51:21 +01:00
tick-broadcast-hrtimer.c tick: broadcast-hrtimer: Fix a race in bc_set_next 2019-09-27 14:45:55 +02:00
tick-broadcast.c tick: Fix typos in comments 2019-04-19 19:17:04 +02:00
tick-common.c Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-05-06 14:50:46 -07:00
tick-internal.h tick: Remove outgoing CPU from broadcast masks 2019-03-23 18:26:43 +01:00
tick-oneshot.c hrtimers/tick/clockevents: Remove sloppy license references 2018-11-23 11:51:21 +01:00
tick-sched.c tick/sched: Annotate lockless access to last_jiffies_update 2020-01-23 08:22:55 +01:00
tick-sched.h tick/sched: Update tick_sched struct documentation 2019-03-24 20:29:32 +01:00
time.c time: Zero the upper 32-bits in __kernel_timespec on 32-bit 2019-12-13 08:42:18 +01:00
timeconst.bc time: Add SPDX license identifiers 2018-11-23 11:51:20 +01:00
timeconv.c time: Add SPDX license identifiers 2018-11-23 11:51:20 +01:00
timecounter.c time: Remove license boilerplate 2018-11-23 11:51:21 +01:00
timekeeping.c timekeeping/vsyscall: Prevent math overflow in BOOTTIME update 2019-08-23 02:12:11 +02:00
timekeeping.h timers/sched_clock: Prevent generic sched_clock wrap caused by tick_freeze() 2019-04-18 14:34:53 +02:00
timekeeping_debug.c timekeeping/debug: No need to check return value of debugfs_create functions 2019-01-29 20:08:41 +01:00
timekeeping_internal.h timekeeping/ntp: Constify some function arguments 2018-07-19 17:08:05 -07:00
timer.c timer: Read jiffies once when forwarding base clk 2019-09-19 17:50:11 +02:00
timer_list.c timer_list: Guard procfs specific code 2019-06-23 00:08:52 +02:00
vsyscall.c timekeeping/vsyscall: Update VDSO data unconditionally 2019-11-04 23:02:53 +01:00