redonkable
/
alistair23-linux
1
0
Fork 0
Code Releases Activity
alistair23-linux/arch/arm64/include/asm
History
Dave Martin e6b673b741 KVM: arm64: Optimise FPSIMD handling to reduce guest/host thrashing 
This patch refactors KVM to align the host and guest FPSIMD
save/restore logic with each other for arm64.  This reduces the
number of redundant save/restore operations that must occur, and
reduces the common-case IRQ blackout time during guest exit storms
by saving the host state lazily and optimising away the need to
restore the host state before returning to the run loop.

Four hooks are defined in order to enable this:

 * kvm_arch_vcpu_run_map_fp():
   Called on PID change to map necessary bits of current to Hyp.

 * kvm_arch_vcpu_load_fp():
   Set up FP/SIMD for entering the KVM run loop (parse as
   "vcpu_load fp").

 * kvm_arch_vcpu_ctxsync_fp():
   Get FP/SIMD into a safe state for re-enabling interrupts after a
   guest exit back to the run loop.

   For arm64 specifically, this involves updating the host kernel's
   FPSIMD context tracking metadata so that kernel-mode NEON use
   will cause the vcpu's FPSIMD state to be saved back correctly
   into the vcpu struct.  This must be done before re-enabling
   interrupts because kernel-mode NEON may be used by softirqs.

 * kvm_arch_vcpu_put_fp():
   Save guest FP/SIMD state back to memory and dissociate from the
   CPU ("vcpu_put fp").

Also, the arm64 FPSIMD context switch code is updated to enable it
to save back FPSIMD state for a vcpu, not just current.  A few
helpers drive this:

 * fpsimd_bind_state_to_cpu(struct user_fpsimd_state *fp):
   mark this CPU as having context fp (which may belong to a vcpu)
   currently loaded in its registers.  This is the non-task
   equivalent of the static function fpsimd_bind_to_cpu() in
   fpsimd.c.

 * task_fpsimd_save():
   exported to allow KVM to save the guest's FPSIMD state back to
   memory on exit from the run loop.

 * fpsimd_flush_state():
   invalidate any context's FPSIMD state that is currently loaded.
   Used to disassociate the vcpu from the CPU regs on run loop exit.

These changes allow the run loop to enable interrupts (and thus
softirqs that may use kernel-mode NEON) without having to save the
guest's FPSIMD state eagerly.

Some new vcpu_arch fields are added to make all this work.  Because
host FPSIMD state can now be saved back directly into current's
thread_struct as appropriate, host_cpu_context is no longer used
for preserving the FPSIMD state.  However, it is still needed for
preserving other things such as the host's system registers.  To
avoid ABI churn, the redundant storage space in host_cpu_context is
not removed for now.

arch/arm is not addressed by this patch and continues to use its
current save/restore logic.  It could provide implementations of
the helpers later if desired.

Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Christoffer Dall <christoffer.dall@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
 		2018-05-25 12:28:28 +01:00
..
xen License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
Kbuild arch: Remove clkdev.h asm-generic from Kbuild 2018-01-03 09:02:11 -08:00
acenv.h ARM64 / ACPI: Get RSDP and ACPI boot-time tables 2015-03-25 11:49:30 +00:00
acpi.h arm64: mm: Remove arch_apei_flush_tlb_one() 2017-11-07 12:13:33 +01:00
alternative.h arm64: alternatives: Add dynamic patching feature 2018-03-19 13:03:17 +00:00
arch_gicv3.h irqchip/gic-v3: Probe for SCR_EL3 being clear before resetting AP0Rn 2018-03-22 13:46:18 +00:00
arch_timer.h First batch of KVM changes for 4.15 2017-11-16 13:00:24 -08:00
arm-cci.h arm-cci: Get rid of secure transactions for PMU driver 2015-03-27 13:44:35 +00:00
arm_dsu_pmu.h perf: ARM DynamIQ Shared Unit PMU support 2018-01-02 16:43:12 +00:00
asm-bug.h arm64: asm-bug: Renumber macro local labels to avoid clashes 2017-10-25 15:57:15 +01:00
asm-offsets.h
…
asm-uaccess.h arm64: mm: Add additional parameter to uaccess_ttbr0_disable 2018-01-17 13:57:49 +01:00
assembler.h arm64: assembler: add macros to conditionally yield the NEON under PREEMPT 2018-04-11 18:50:34 +01:00
atomic.h locking/atomic: Remove linux/atomic.h:atomic_fetch_or() 2016-06-16 10:48:32 +02:00
atomic_ll_sc.h arm64: Remove redundant mov from LL/SC cmpxchg 2017-05-15 18:30:10 +01:00
atomic_lse.h arm64: atomics: Remove '&' from '+&' asm constraint in lse atomics 2017-07-20 10:20:54 +01:00
barrier.h arm64: Implement array_index_mask_nospec() 2018-02-06 22:53:30 +00:00
bitops.h
…
bitrev.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
boot.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
brk-imm.h arm64: move brk immediate argument definitions to separate header 2016-02-23 17:43:25 +00:00
bug.h arm64: Add ASM_BUG() 2017-08-08 16:28:13 +01:00
cache.h Revert "arm64: Revert L1_CACHE_SHIFT back to 6 (64-byte cache line size)" 2018-03-27 12:04:51 +01:00
cacheflush.h arm64: turn flush_dcache_mmap_lock into a no-op 2018-04-11 10:28:39 -07:00
checksum.h arm64: add missing conversion to __wsum in ip_fast_csum() 2017-06-29 16:32:43 +01:00
clocksource.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cmpxchg.h arm64: cmpxchg: Include linux/compiler.h in asm/cmpxchg.h 2018-03-27 13:15:49 +01:00
compat.h signal: unify compat_siginfo_t 2018-01-15 17:40:31 -06:00
compiler.h
…
cpu.h arm64/sve: Probe SVE capabilities and usable vector lengths 2017-11-03 15:24:17 +00:00
cpu_ops.h ARM64: kernel: unify ACPI and DT cpus initialization 2015-05-19 16:09:29 +01:00
cpucaps.h arm64: KVM: Use SMCCC_ARCH_WORKAROUND_1 for Falkor BP hardening 2018-04-11 18:49:30 +01:00
cpufeature.h arm64: capabilities: Handle shared entries 2018-03-26 18:01:43 +01:00
cpuidle.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
cputype.h arm64: Add MIDR encoding for Arm Cortex-A55 and Cortex-A35 2018-03-26 18:01:43 +01:00
current.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
daifflags.h arm64: Move the async/fiq helpers to explicitly set process context flags 2017-11-02 15:55:41 +00:00
dcc.h arm64: dcc: simplify accessors 2016-09-09 11:41:13 +01:00
debug-monitors.h arm64: Add uprobe support 2016-11-07 18:15:21 +00:00
device.h xen/arm,arm64: fix xen_dma_ops after 815dd18 "Consolidate get_dma_ops..." 2017-05-02 11:14:42 +02:00
dma-mapping.h dma-mapping: move dma_mark_clean to dma-direct.h 2018-01-10 16:41:12 +01:00
dmi.h
…
efi.h efi/arm64: Check whether x18 is preserved by runtime services calls 2018-03-09 08:58:22 +01:00
elf.h arm64: elf.h: Correct comment about READ_IMPLIES_EXEC propagation 2017-10-09 10:00:23 +01:00
esr.h arm64: fpsimd: Fix bad si_code for undiagnosed SIGFPE 2018-03-20 10:03:11 +00:00
exception.h arm64: kernel: Prepare for a DISR user 2018-01-16 15:07:12 +00:00
exec.h arm64: suspend: Reconfigure PSTATE after resume from idle 2016-10-20 09:50:54 +01:00
extable.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
fb.h
…
fixmap.h arm64: kaslr: Put kernel vectors address in separate data page 2017-12-11 13:41:20 +00:00
fpsimd.h KVM: arm64: Optimise FPSIMD handling to reduce guest/host thrashing 2018-05-25 12:28:28 +01:00
fpsimdmacros.h arm64/sve: Low-level SVE architectural state manipulation functions 2017-11-03 15:24:14 +00:00
ftrace.h arm64: Fix misspellings in comments. 2016-03-04 18:19:17 +00:00
futex.h arm64: futex: Mask __user pointers prior to dereference 2018-02-06 22:53:42 +00:00
hardirq.h arm64: kdump: implement machine_crash_shutdown() 2017-04-05 18:29:15 +01:00
hugetlb.h arm64: mm: Use READ_ONCE/WRITE_ONCE when accessing page tables 2018-02-16 18:13:57 +00:00
hw_breakpoint.h arm64: cpufeature: Make ID reg accessor naming less counterintuitive 2017-04-04 16:55:41 +01:00
hwcap.h arm64/HWCAP: Use system wide safe values 2015-10-21 15:35:58 +01:00
hypervisor.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
insn.h arm64; insn: Add encoder for the EXTR instruction 2018-03-19 13:05:10 +00:00
io.h ARM64: Implement pci_remap_cfgspace() interface 2017-04-24 13:53:13 -05:00
irq.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
irq_work.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
irqflags.h arm64: Move the async/fiq helpers to explicitly set process context flags 2017-11-02 15:55:41 +00:00
jump_label.h locking/static_keys: Add a new static_key interface 2015-08-03 11:34:15 +02:00
kasan.h kasan: clean up KASAN_SHADOW_SCALE_SHIFT usage 2018-02-06 18:32:43 -08:00
kernel-pgtable.h arm64: kpti: Make use of nG dependent on arm64_kernel_unmapped_at_el0() 2018-02-06 22:53:14 +00:00
kexec.h arm64: kdump: implement machine_crash_shutdown() 2017-04-05 18:29:15 +01:00
kgdb.h arm64: kgdb: Match pstate size with gdbserver protocol 2016-06-16 19:20:51 +01:00
kprobes.h kprobes: move kprobe declarations to asm-generic/kprobes.h 2017-02-27 18:43:45 -08:00
kvm_arm.h KVM: arm64: Move HCR_INT_OVERRIDE to default HCR_EL2 guest flag 2018-03-19 10:53:10 +00:00
kvm_asm.h KVM: arm64: Repurpose vcpu_arch.debug_flags for general-purpose flags 2018-05-25 12:28:13 +01:00
kvm_coproc.h
…
kvm_emulate.h KVM: arm64: Fix order of vcpu_write_sys_reg() arguments 2018-05-04 16:44:54 +01:00
kvm_host.h KVM: arm64: Optimise FPSIMD handling to reduce guest/host thrashing 2018-05-25 12:28:28 +01:00
kvm_hyp.h KVM: arm/arm64: Avoid VGICv3 save/restore on VHE with no IRQs 2018-03-19 10:53:21 +00:00
kvm_mmio.h KVM: arm/arm64: Export mmio_read/write_bus 2016-05-20 15:39:42 +02:00
kvm_mmu.h arm64: KVM: Allow mapping of vectors outside of the RAM region 2018-03-19 13:06:46 +00:00
linkage.h arm64: relax assembly code alignment from 16 byte to 4 byte 2017-09-18 11:20:19 +01:00
lse.h arm64: lse: Include compiler_types.h and export.h for out-of-line LL/SC 2018-03-27 13:14:49 +01:00
memblock.h
…
memory.h linux/const.h: move UL() macro to include/linux/const.h 2018-04-11 10:28:38 -07:00
mmu.h arm64: KVM: Allow mapping of vectors outside of the RAM region 2018-03-19 13:06:46 +00:00
mmu_context.h arm64: mm: Use READ_ONCE/WRITE_ONCE when accessing page tables 2018-02-16 18:13:57 +00:00
mmzone.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
module.h arm64/kernel: rename module_emit_adrp_veneer->module_emit_veneer_for_adrp 2018-04-24 19:07:35 +01:00
neon.h arm64: neon: Temporarily add a kernel_mode_begin_partial() definition 2017-08-04 15:10:12 +01:00
numa.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
page-def.h arm64: factor out PAGE_* and CONT_* definitions 2017-08-15 18:35:00 +01:00
page.h arm64: factor out PAGE_* and CONT_* definitions 2017-08-15 18:35:00 +01:00
paravirt.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
pci.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
percpu.h arm64: move percpu cmpxchg implementation from cmpxchg.h to percpu.h 2018-03-27 13:15:29 +01:00
perf_event.h arm64/bpf: correct broken uapi for BPF_PROG_TYPE_PERF_EVENT program type 2017-12-05 15:02:41 +01:00
pgalloc.h arm64: mm: Use READ_ONCE/WRITE_ONCE when accessing page tables 2018-02-16 18:13:57 +00:00
pgtable-hwdef.h arm64: kaslr: Set TCR_EL1.NFD1 when CONFIG_RANDOMIZE_BASE=y 2018-03-06 18:52:34 +00:00
pgtable-prot.h KVM changes for 4.16 2018-02-10 13:16:35 -08:00
pgtable-types.h arch, mm: convert all architectures to use 5level-fixup.h 2017-03-09 11:48:47 -08:00
pgtable.h arm64: mm: drop addr parameter from sync icache and dcache 2018-04-24 09:23:00 +01:00
probes.h arm64: Remove reference to asm/opcodes.h 2016-12-05 10:42:34 +00:00
proc-fns.h arm64: mm: Move ASID from TTBR0 to TTBR1 2017-12-11 13:40:25 +00:00
processor.h arm64: fpsimd: Eliminate task->mm checks 2018-05-25 12:27:55 +01:00
ptdump.h arm64: dump: Add checking for writable and exectuable pages 2016-11-07 18:15:04 +00:00
ptrace.h Merge branch 'arm64/exception-stack' of git://git.kernel.org/pub/scm/linux/kernel/git/mark/linux into for-next/core 2017-08-09 15:37:49 +01:00
sdei.h arm64: sdei: Add trampoline code for remapping the kernel 2018-01-14 18:49:50 +00:00
seccomp.h
…
sections.h arm64: mmu: add the entry trampolines start/end section markers into sections.h 2018-01-14 18:49:50 +00:00
shmparam.h arm64: fix COMPAT_SHMLBA definition for large pages 2015-12-02 14:00:10 +00:00
signal32.h arm64: compat: Remove leftover variable declaration 2017-08-10 18:17:32 +01:00
simd.h arm64: neon: Forbid when irqs are disabled 2017-08-09 15:05:59 +01:00
smp.h arm64: kexec: have own crash_smp_send_stop() for crash dump for nonpanic cores 2017-08-21 18:01:04 +01:00
smp_plat.h arm64: remove dead-code depending on CONFIG_UP_LATE_INIT 2015-07-29 18:32:09 +01:00
sparsemem.h arm64: add kconfig symbol to configure physical address size 2017-12-22 17:30:33 +00:00
spinlock.h arm64: spinlock: Fix theoretical trylock() A-B-A with LSE atomics 2018-02-06 22:53:11 +00:00
spinlock_types.h locking/qrwlock, arm64: Move rwlock implementation over to qrwlocks 2017-10-25 10:57:25 +02:00
stack_pointer.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
stackprotector.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
stacktrace.h arm64: fix unwind_frame() for filtered out fn for function graph tracing 2018-02-23 13:46:38 +00:00
stage2_pgtable-nopmd.h kvm: arm64: Get rid of fake page table levels 2016-04-21 14:58:21 +02:00
stage2_pgtable-nopud.h kvm: arm64: Get rid of fake page table levels 2016-04-21 14:58:21 +02:00
stage2_pgtable.h kvm: arm64: Get rid of fake page table levels 2016-04-21 14:58:21 +02:00
stat.h
…
string.h arm64: uaccess: Implement *_flushcache variants 2017-08-09 12:16:26 +01:00
suspend.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
sync_bitops.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
syscall.h
…
sysreg.h ARM: 2018-04-09 11:42:31 -07:00
system_misc.h arm64: Move show_unhandled_signals_ratelimited into traps.c 2018-03-06 18:52:31 +00:00
thread_info.h arm64: fpsimd: Avoid FPSIMD context leakage for the init task 2018-05-25 12:27:55 +01:00
timex.h arm64: Use physical counter for in-kernel reads when booted in EL2 2017-11-06 16:23:09 +01:00
tlb.h arm64: tlbflush: avoid flushing when fullmm == 1 2015-10-07 11:56:21 +01:00
tlbflush.h arm64: tlbflush: avoid writing RES0 bits 2018-03-28 15:20:17 +01:00
topology.h Merge branch 'pm-cpufreq' 2017-11-13 01:34:49 +01:00
traps.h arm64: Introduce arm64_force_sig_info and hook up in arm64_notify_die 2018-03-06 18:52:23 +00:00
uaccess.h arm64: uaccess: Formalise types for access_ok() 2018-02-19 13:59:58 +00:00
unistd.h Remove compat_sys_getdents64() 2017-04-17 12:52:22 -04:00
unistd32.h Remove compat_sys_getdents64() 2017-04-17 12:52:22 -04:00
uprobes.h arm64: Add uprobe support 2016-11-07 18:15:21 +00:00
vdso.h
…
vdso_datapage.h arm64: Add support for CLOCK_MONOTONIC_RAW in clock_gettime() vDSO 2016-07-12 16:06:32 +01:00
virt.h arm64: capabilities: Change scope of VHE to Boot CPU feature 2018-03-26 18:01:41 +01:00
vmap_stack.h arm64: Add vmap_stack header file 2018-01-13 10:45:03 +00:00
word-at-a-time.h Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00