redonkable
/
alistair23-linux
1
0
Fork 0
Code Releases Activity
alistair23-linux/arch/arm64/include/asm
History
Will Deacon 22ec71615d arm64: io: Relax implicit barriers in default I/O accessors 
The arm64 implementation of the default I/O accessors requires barrier
instructions to satisfy the memory ordering requirements documented in
memory-barriers.txt [1], which are largely derived from the behaviour of
I/O accesses on x86.

Of particular interest are the requirements that a write to a device
must be ordered against prior writes to memory, and a read from a device
must be ordered against subsequent reads from memory. We satisfy these
requirements using various flavours of DSB: the most expensive barrier
we have, since it implies completion of prior accesses. This was deemed
necessary when we first implemented the accessors, since accesses to
different endpoints could propagate independently and therefore the only
way to enforce order is to rely on completion guarantees [2].

Since then, the Armv8 memory model has been retrospectively strengthened
to require "other-multi-copy atomicity", a property that requires memory
accesses from an observer to become visible to all other observers
simultaneously [3]. In other words, propagation of accesses is limited
to transitioning from locally observed to globally observed. It recently
became apparent that this change also has a subtle impact on our I/O
accessors for shared peripherals, allowing us to use the cheaper DMB
instruction instead.

As a concrete example, consider the following:

	memcpy(dma_buffer, data, bufsz);
	writel(DMA_START, dev->ctrl_reg);

A DMB ST instruction between the final write to the DMA buffer and the
write to the control register will ensure that the writes to the DMA
buffer are observed before the write to the control register by all
observers. Put another way, if an observer can see the write to the
control register, it can also see the writes to memory. This has always
been the case and is not sufficient to provide the ordering required by
Linux, since there is no guarantee that the master interface of the
DMA-capable device has observed either of the accesses. However, in an
other-multi-copy atomic world, we can infer two things:

  1. A write arriving at an endpoint shared between multiple CPUs is
     visible to all CPUs

  2. A write that is visible to all CPUs is also visible to all other
     observers in the shareability domain

Pieced together, this allows us to use DMB OSHST for our default I/O
write accessors and DMB OSHLD for our default I/O read accessors (the
outer-shareability is for handling non-cacheable mappings) for shared
devices. Memory-mapped, DMA-capable peripherals that are private to a
CPU (i.e. inaccessible to other CPUs) still require the DSB, however
these are few and far between and typically require special treatment
anyway which is outside of the scope of the portable driver API (e.g.
GIC, page-table walker, SPE profiler).

Note that our mandatory barriers remain as DSBs, since there are cases
where they are used to flush the store buffer of the CPU, e.g. when
publishing page table updates to the SMMU.

[1] https://git.kernel.org/linus/4614bbdee357
[2] https://www.youtube.com/watch?v=i6DayghhA8Q
[3] https://www.cl.cam.ac.uk/~pes20/armv8-mca/

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
 		2019-08-05 12:35:23 +01:00
..
vdso arm64: compat: vdso: Use legacy syscalls as fallback 2019-07-31 00:09:10 +02:00
xen arm64/xen: fix xen-swiotlb cache flushing 2019-01-23 22:14:56 +01:00
Kbuild treewide: Add SPDX license identifier - Kbuild 2019-05-30 11:32:33 -07:00
acenv.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
acpi.h arm64 updates for 5.3: 2019-07-08 09:54:55 -07:00
alternative.h arm64: alternative: Apply alternatives early in boot process 2019-02-06 10:05:20 +00:00
arch_gicv3.h arm64: Lower priority mask for GIC_PRIO_IRQON 2019-08-01 14:59:48 +01:00
arch_timer.h Merge branch 'timers/vdso' into timers/core 2019-07-03 10:50:21 +02:00
arm-cci.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
arm_dsu_pmu.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
asm-bug.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
asm-offsets.h
…
asm-prototypes.h arm64: asm-prototypes: Fix fat-fingered typo in comment 2019-01-10 11:11:46 +00:00
asm-uaccess.h arm64: Rename get_thread_info() 2019-02-26 16:57:59 +00:00
assembler.h arm64: Remove unused assembly macro 2019-08-05 11:06:33 +01:00
atomic.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
atomic_ll_sc.h Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-07-08 16:12:03 -07:00
atomic_lse.h Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-07-08 16:12:03 -07:00
barrier.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
bitops.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
bitrev.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
boot.h treewide: replace #include <asm/sizes.h> with #include <linux/sizes.h> 2019-05-14 19:52:52 -07:00
brk-imm.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
bug.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
cache.h arm64 updates for 5.3: 2019-07-08 09:54:55 -07:00
cacheflush.h arm64 updates for 5.3: 2019-07-08 09:54:55 -07:00
checksum.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
clocksource.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cmpxchg.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
compat.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
cpu.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
cpu_ops.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
cpucaps.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
cpufeature.h arm64: Remove unused cpucap_multi_entry_cap_cpu_enable() 2019-08-05 11:06:34 +01:00
cpuidle.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cputype.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
current.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
daifflags.h arm64: kprobes: Recover pstate.D in single-step exception handler 2019-08-02 11:55:50 +01:00
dcc.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 284 2019-06-05 17:36:37 +02:00
debug-monitors.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
device.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
dma-mapping.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
dmi.h arm64: dmi: Add SMBIOS/DMI support 2014-11-05 09:03:25 +01:00
efi.h arm64/efi: fix variable 'si' set but not used 2019-07-31 18:13:46 +01:00
elf.h arm64: vdso: Fix population of AT_SYSINFO_EHDR for compat vdso 2019-07-22 10:33:42 +01:00
esr.h arm64: esr: Add ESR exception class encoding for trapped ERET 2019-08-05 11:06:34 +01:00
exception.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
exec.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
extable.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
fb.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
fixmap.h firmware: arm_sdei: Add ACPI GHES registration helper 2019-02-11 11:07:49 +01:00
fpsimd.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
fpsimdmacros.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
ftrace.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
futex.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
hardirq.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
hugetlb.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 174 2019-05-30 11:26:41 -07:00
hw_breakpoint.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
hwcap.h arm64 updates for 5.3: 2019-07-08 09:54:55 -07:00
hypervisor.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
image.h docs: arm64: convert docs to ReST and rename to .rst 2019-06-14 14:20:27 -06:00
insn.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
io.h arm64: io: Relax implicit barriers in default I/O accessors 2019-08-05 12:35:23 +01:00
irq.h arm64: Use the new GENERIC_IRQ_MULTI_HANDLER 2018-08-03 12:14:09 +02:00
irq_work.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
irqflags.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
jump_label.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
kasan.h kasan: add tag related helper functions 2018-12-28 12:11:43 -08:00
kernel-pgtable.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
kexec.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
kgdb.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
kprobes.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 174 2019-05-30 11:26:41 -07:00
kvm_arm.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
kvm_asm.h KVM: arm64: Consume pending SError as early as possible 2019-07-05 13:03:29 +01:00
kvm_coproc.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
kvm_emulate.h KVM: arm64: Migrate _elx sysreg accessors to msr_s/mrs_s 2019-07-05 13:57:25 +01:00
kvm_host.h ARM: 2019-07-12 15:35:14 -07:00
kvm_hyp.h KVM: arm64: Migrate _elx sysreg accessors to msr_s/mrs_s 2019-07-05 13:57:25 +01:00
kvm_mmio.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
kvm_mmu.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
kvm_ptrauth.h KVM: arm/arm64: Context-switch ptrauth registers 2019-04-24 15:30:40 +01:00
kvm_ras.h arm64: KVM/mm: Move SEA handling behind a single 'claim' interface 2019-02-07 23:10:45 +01:00
linkage.h arm64: relax assembly code alignment from 16 byte to 4 byte 2017-09-18 11:20:19 +01:00
lse.h arm64: lse: Include compiler_types.h and export.h for out-of-line LL/SC 2018-03-27 13:14:49 +01:00
memory.h arm64/mm: fix variable 'tag' set but not used 2019-08-01 15:53:10 +01:00
mmu.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
mmu_context.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
mmzone.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
module.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
neon-intrinsics.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
neon.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
numa.h arm64: numa: separate out updates to percpu nodeid and NUMA node cpumap 2018-07-06 13:18:18 +01:00
page-def.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
page.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
paravirt.h x86/paravirt: Use a single ops structure 2018-09-03 16:50:35 +02:00
pci.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
percpu.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
perf_event.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
pgalloc.h arm64: switch to generic version of pte allocation 2019-07-12 11:05:45 -07:00
pgtable-hwdef.h arm64 updates for 5.3: 2019-07-08 09:54:55 -07:00
pgtable-prot.h arm64: mm: implement pte_devmap support 2019-07-16 19:23:25 -07:00
pgtable-types.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
pgtable.h arm64/mm: fix variable 'pud' set but not used 2019-08-01 15:00:27 +01:00
pointer_auth.h arm64: Use the correct style for SPDX License Identifier 2019-04-16 16:28:01 +01:00
preempt.h arm64: preempt: Provide our own implementation of asm/preempt.h 2018-12-07 12:35:53 +00:00
probes.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 174 2019-05-30 11:26:41 -07:00
proc-fns.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
processor.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
ptdump.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
ptrace.h arm64: Lower priority mask for GIC_PRIO_IRQON 2019-08-01 14:59:48 +01:00
sdei.h arm64: Use the correct style for SPDX License Identifier 2019-04-16 16:28:01 +01:00
seccomp.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
sections.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
shmparam.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
signal32.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
simd.h arm64 updates for 5.3: 2019-07-08 09:54:55 -07:00
smp.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
smp_plat.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
sparsemem.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
spinlock.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
spinlock_types.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
stack_pointer.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
stackprotector.h arm64: enable per-task stack canaries 2018-12-12 18:45:31 +00:00
stacktrace.h arm64: stacktrace: Better handle corrupted stacks 2019-07-22 11:44:15 +01:00
stage2_pgtable.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
stat.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
string.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
suspend.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
sync_bitops.h arm64, locking/atomics: Use instrumented atomics 2018-11-01 11:01:40 +01:00
syscall.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
syscall_wrapper.h arm64: use the correct function type in SYSCALL_DEFINE0 2019-05-29 13:45:59 +01:00
sysreg.h arm64: sysreg: Remove unused and rotting SCTLR_ELx field definitions 2019-08-05 11:06:34 +01:00
system_misc.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
thread_info.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
timex.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
tlb.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
tlbflush.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
topology.h sched/topology, arch/arm64: Rebuild the sched_domain hierarchy when the CPU capacity changes 2018-09-10 11:05:47 +02:00
traps.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
uaccess.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
unistd.h clone3-v5.3 2019-07-11 10:09:44 -07:00
unistd32.h clone3-v5.3 2019-07-11 10:09:44 -07:00
uprobes.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
vdso.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
vdso_datapage.h arm64: remove pointless __KERNEL__ guards 2019-08-05 11:06:33 +01:00
virt.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
vmap_stack.h arm64: Use the correct style for SPDX License Identifier 2019-04-16 16:28:01 +01:00
word-at-a-time.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234 2019-06-19 17:09:07 +02:00
xor.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00