alistair23-linux/arch/arm64/include/asm
Linus Torvalds 2b047252d0 Fix TLB gather virtual address range invalidation corner cases
Ben Tebulin reported:

 "Since v3.7.2 on two independent machines a very specific Git
  repository fails in 9/10 cases on git-fsck due to an SHA1/memory
  failures.  This only occurs on a very specific repository and can be
  reproduced stably on two independent laptops.  Git mailing list ran
  out of ideas and for me this looks like some very exotic kernel issue"

and bisected the failure to the backport of commit 53a59fc67f ("mm:
limit mmu_gather batching to fix soft lockups on !CONFIG_PREEMPT").

That commit itself is not actually buggy, but what it does is to make it
much more likely to hit the partial TLB invalidation case, since it
introduces a new case in tlb_next_batch() that previously only ever
happened when running out of memory.

The real bug is that the TLB gather virtual memory range setup is subtly
buggered.  It was introduced in commit 597e1c3580 ("mm/mmu_gather:
enable tlb flush range in generic mmu_gather"), and the range handling
was already fixed at least once in commit e6c495a96c ("mm: fix the TLB
range flushed when __tlb_remove_page() runs out of slots"), but that fix
was not complete.

The problem with the TLB gather virtual address range is that it isn't
set up by the initial tlb_gather_mmu() initialization (which didn't get
the TLB range information), but it is set up ad-hoc later by the
functions that actually flush the TLB.  And so any such case that forgot
to update the TLB range entries would potentially miss TLB invalidates.

Rather than try to figure out exactly which particular ad-hoc range
setup was missing (I personally suspect it's the hugetlb case in
zap_huge_pmd(), which didn't have the same logic as zap_pte_range()
did), this patch just gets rid of the problem at the source: make the
TLB range information available to tlb_gather_mmu(), and initialize it
when initializing all the other tlb gather fields.

This makes the patch larger, but conceptually much simpler.  And the end
result is much more understandable; even if you want to play games with
partial ranges when invalidating the TLB contents in chunks, now the
range information is always there, and anybody who doesn't want to
bother with it won't introduce subtle bugs.

Ben verified that this fixes his problem.

Reported-bisected-and-tested-by: Ben Tebulin <tebulin@googlemail.com>
Build-testing-by: Stephen Rothwell <sfr@canb.auug.org.au>
Build-testing-by: Richard Weinberger <richard.weinberger@gmail.com>
Reviewed-by: Michal Hocko <mhocko@suse.cz>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Cc: stable@vger.kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-08-16 08:52:46 -07:00
..
xen arm64/xen: introduce asm/xen header files on arm64 2013-06-07 10:39:45 +00:00
arch_timer.h arm64: delete __cpuinit usage from all users 2013-07-14 19:36:52 -04:00
asm-offsets.h
assembler.h arm64: debug: fix mdscr.ss check when enabling debug exceptions 2013-05-17 18:24:19 +01:00
atomic.h arm64: atomics: fix grossly inconsistent asm constraints for exclusives 2013-02-11 18:16:41 +00:00
barrier.h arm64: Miscellaneous header files 2012-09-17 13:42:21 +01:00
bitops.h arm64: klib: Optimised atomic bitops 2013-03-21 17:39:31 +00:00
cache.h arm64: Cache maintenance routines 2012-09-17 13:42:00 +01:00
cacheflush.h arm64: Remove __flush_dcache_page() 2013-06-07 17:58:30 +01:00
cachetype.h arm64: Cache maintenance routines 2012-09-17 13:42:00 +01:00
cmpxchg.h arm64: Define cmpxchg64 and cmpxchg64_local for outside use 2013-04-23 11:47:26 +01:00
compat.h arm64: Fix compat types affecting struct compat_stat 2013-04-25 17:45:50 +01:00
compiler.h arm64: Miscellaneous header files 2012-09-17 13:42:21 +01:00
cputable.h arm64: CPU support 2012-09-17 13:41:59 +01:00
cputype.h arm64: Add defines for APM ARMv8 implementation 2013-06-20 17:03:59 +01:00
debug-monitors.h arm64: add '#ifdef CONFIG_COMPAT' for aarch32_break_handler() 2013-07-19 15:49:43 +01:00
device.h arm64: device: add iommu pointer to device archdata 2013-06-11 18:15:55 +01:00
dma-mapping.h arm64: Provide default implementation for dma_{alloc,free}_attrs 2013-06-07 17:57:23 +01:00
elf.h arm64: elf: fix core dumping to match what glibc expects 2013-01-22 17:50:59 +00:00
esr.h arm64: add explicit symbols to ESR_EL1 decoding 2013-04-17 15:58:25 +01:00
exception.h arm64: Use irqchip_init() for interrupt controller initialisation 2013-03-26 16:02:23 +00:00
exec.h arm64: Miscellaneous header files 2012-09-17 13:42:21 +01:00
fb.h arm64: Device specific operations 2012-09-17 13:42:04 +01:00
fpsimd.h arm64: elf: fix core dumping definitions for GP and FP registers 2012-11-08 16:06:20 +00:00
fpsimdmacros.h arm64: move FP-SIMD save/restore code to a macro 2012-12-05 11:26:50 +00:00
futex.h arm64: atomics: fix grossly inconsistent asm constraints for exclusives 2013-02-11 18:16:41 +00:00
hardirq.h arm64: Use irqchip_init() for interrupt controller initialisation 2013-03-26 16:02:23 +00:00
hugetlb.h ARM64: mm: HugeTLB support. 2013-06-14 09:52:40 +01:00
hw_breakpoint.h arm64: Debugging support 2012-09-17 13:42:14 +01:00
hwcap.h UAPI: (Scripted) Disintegrate arch/arm64/include/asm 2012-10-11 11:05:13 +01:00
hypervisor.h arm64/xen: introduce asm/xen header files on arm64 2013-06-07 10:39:45 +00:00
io.h arm64/xen: implement ioremap_cached on arm64 2013-06-07 10:39:50 +00:00
irq.h arm64: Use irqchip_init() for interrupt controller initialisation 2013-03-26 16:02:23 +00:00
irqflags.h arm64: IRQ handling 2012-09-17 13:42:02 +01:00
Kbuild arm64: compiling issue, need add include/asm/vga.h file 2013-04-25 17:45:47 +01:00
kvm_arm.h arm64: KVM: EL2 register definitions 2013-06-07 14:03:32 +01:00
kvm_asm.h arm64: KVM: define 32bit specific registers 2013-06-12 16:42:13 +01:00
kvm_coproc.h arm64: KVM: 32bit handling of coprocessor traps 2013-06-12 16:42:16 +01:00
kvm_emulate.h arm64: KVM: 32bit conditional execution emulation 2013-06-12 16:42:15 +01:00
kvm_host.h arm64: KVM: enable initialization of a 32bit vcpu 2013-06-12 16:42:18 +01:00
kvm_mmio.h arm64: KVM: MMIO access backend 2013-06-07 14:03:38 +01:00
kvm_mmu.h arm64: KVM: architecture specific MMU backend 2013-06-07 14:03:35 +01:00
kvm_psci.h arm64: KVM: PSCI implementation 2013-06-12 16:40:32 +01:00
linkage.h arm64: fix alignment padding in assembly code 2012-10-20 11:12:01 +01:00
memblock.h arm64: MMU initialisation 2012-09-17 13:41:56 +01:00
memory.h arm64: KVM: define HYP and Stage-2 translation page flags 2013-06-07 14:03:31 +01:00
mmu.h arm64: Add simple earlyprintk support 2013-01-22 17:51:01 +00:00
mmu_context.h arm64: mm: don't bother invalidating the icache in switch_mm 2013-06-07 18:00:11 +01:00
module.h arm64: Loadable modules 2012-09-17 13:42:19 +01:00
page.h arm64: MMU fault handling and page table management 2012-09-17 13:41:57 +01:00
perf_event.h arm64: perf: add guest vs host discrimination 2013-01-29 16:56:17 +00:00
pgalloc.h arm64: MMU fault handling and page table management 2012-09-17 13:41:57 +01:00
pgtable-2level-hwdef.h arm64: MMU definitions 2012-09-17 13:41:56 +01:00
pgtable-2level-types.h arm64: MMU definitions 2012-09-17 13:41:56 +01:00
pgtable-3level-hwdef.h arm64: MMU definitions 2012-09-17 13:41:56 +01:00
pgtable-3level-types.h arm64: MMU definitions 2012-09-17 13:41:56 +01:00
pgtable-hwdef.h Merge branch 'for-next/hugepages' of git://git.linaro.org/people/stevecapper/linux into upstream-hugepages 2013-07-01 11:20:58 +01:00
pgtable.h Main features: 2013-07-03 10:31:38 -07:00
pmu.h arm64: Performance counters support 2012-09-17 13:42:17 +01:00
proc-fns.h arm64: CPU support 2012-09-17 13:41:59 +01:00
processor.h Merge branch 'arch-microblaze' into no-rebases 2012-11-16 22:28:43 -05:00
prom.h arm64: Build infrastructure 2012-09-17 13:42:21 +01:00
psci.h arm64: psci: add support for PSCI invocations from the kernel 2013-01-29 16:56:37 +00:00
ptrace.h arm64: debug: consolidate software breakpoint handlers 2013-06-12 11:23:02 +01:00
shmparam.h arm64: ELF definitions 2012-09-17 13:42:07 +01:00
sigcontext.h UAPI: (Scripted) Disintegrate arch/arm64/include/asm 2012-10-11 11:05:13 +01:00
signal32.h arm64: 32-bit (compat) applications support 2012-09-17 13:42:12 +01:00
smp.h arm64: SMP: enable PSCI boot method 2013-01-29 16:56:37 +00:00
smp_plat.h arm64: kernel: initialise cpu_logical_map from the DT 2013-03-20 17:26:24 +00:00
sparsemem.h arm64: MMU definitions 2012-09-17 13:41:56 +01:00
spinlock.h arm64: spinlock: retry trylock operation if strex fails on free lock 2013-06-07 17:58:31 +01:00
spinlock_types.h arm64: SMP support 2012-09-17 13:42:06 +01:00
stacktrace.h arm64: Exception handling 2012-09-17 10:24:46 +01:00
stat.h UAPI: (Scripted) Disintegrate arch/arm64/include/asm 2012-10-11 11:05:13 +01:00
string.h arm64: klib: Optimised string functions 2013-03-21 17:39:30 +00:00
sync_bitops.h arm64/xen: introduce asm/xen header files on arm64 2013-06-07 10:39:45 +00:00
syscall.h arm64: Miscellaneous library functions 2012-09-17 13:42:18 +01:00
syscalls.h arm64: switch to generic sigaltstack 2013-02-14 09:17:29 -05:00
system_misc.h arm64: use common reboot infrastructure 2013-07-19 15:57:08 +01:00
thread_info.h arm64: Change kernel stack size to 16K 2013-07-26 13:37:29 +01:00
timex.h arm64: kernel: compiling issue, need delete read_current_timer() 2013-06-10 17:58:20 +01:00
tlb.h Fix TLB gather virtual address range invalidation corner cases 2013-08-16 08:52:46 -07:00
tlbflush.h ARM64: mm: THP support. 2013-06-14 09:52:41 +01:00
traps.h arm64: Exception handling 2012-09-17 10:24:46 +01:00
uaccess.h arm64: uaccess s/might_sleep/might_fault/ 2013-05-28 09:41:06 +02:00
ucontext.h arm64: fix padding computation in struct ucontext 2013-03-18 10:42:16 +00:00
unistd.h burying unused conditionals 2013-02-14 09:21:15 -05:00
unistd32.h unify compat fanotify_mark(2), switch to COMPAT_SYSCALL_DEFINE 2013-05-09 13:46:38 -04:00
vdso.h arm64: VDSO support 2012-09-17 13:42:09 +01:00
vdso_datapage.h arm64: VDSO support 2012-09-17 13:42:09 +01:00
virt.h arm64: virt: ensure visibility of __boot_cpu_mode 2013-07-23 11:00:24 +01:00