redonkable
/
alistair23-linux
1
0
Fork 0
Code Releases Activity
alistair23-linux/arch/powerpc/kvm/book3s_interrupts.S

242 lines
5.6 KiB
ArmAsm
Raw Normal View History

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 266 Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation this program is distributed in the hope that it will be useful but without any warranty without even the implied warranty of merchantability or fitness for a particular purpose see the gnu general public license for more details you should have received a copy of the gnu general public license along with this program if not write to the free software foundation 51 franklin street fifth floor boston ma 02110 1301 usa extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 67 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Allison Randal <allison@lohutok.net> Reviewed-by: Richard Fontana <rfontana@redhat.com> Reviewed-by: Alexios Zavras <alexios.zavras@intel.com> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190529141333.953658117@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-29 08:12:40 -06:00
/* SPDX-License-Identifier: GPL-2.0-only */
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
/*
*
* Copyright SUSE Linux Products GmbH 2009
*
* Authors: Alexander Graf <agraf@suse.de>
*/
#include <asm/ppc_asm.h>
#include <asm/kvm_asm.h>
#include <asm/reg.h>
#include <asm/page.h>
#include <asm/asm-offsets.h>
#include <asm/exception-64s.h>
powerpc: move ASM_CONST and stringify_in_c() into asm-const.h This patch moves ASM_CONST() and stringify_in_c() into dedicated asm-const.h, then cleans all related inclusions. Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> [mpe: asm-compat.h should include asm-const.h] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-07-05 10:24:57 -06:00
#include <asm/asm-compat.h>
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
#if defined(CONFIG_PPC_BOOK3S_64)
powerpc: Define and use PPC64_ELF_ABI_v2/v1 We're approaching 20 locations where we need to check for ELF ABI v2. That's fine, except the logic is a bit awkward, because we have to check that _CALL_ELF is defined and then what its value is. So check it once in asm/types.h and define PPC64_ELF_ABI_v2 when ELF ABI v2 is detected. We also have a few places where what we're really trying to check is that we are using the 64-bit v1 ABI, ie. function descriptors. So also add a #define for that, which simplifies several checks. Signed-off-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2016-06-06 10:56:10 -06:00
#ifdef PPC64_ELF_ABI_v2
KVM: PPC: Book3S PR: Fix ABIv2 on LE We switched to ABIv2 on Little Endian systems now which gets rid of the dotted function names. Branch to the actual functions when we see such a system. Signed-off-by: Alexander Graf <agraf@suse.de>
2014-06-16 06:37:53 -06:00
#define FUNC(name) name
#else
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
#define FUNC(name) GLUE(.,name)
KVM: PPC: Book3S PR: Fix ABIv2 on LE We switched to ABIv2 on Little Endian systems now which gets rid of the dotted function names. Branch to the actual functions when we see such a system. Signed-off-by: Alexander Graf <agraf@suse.de>
2014-06-16 06:37:53 -06:00
#endif
KVM: PPC: Book3S PR: Keep volatile reg values in vcpu rather than shadow_vcpu Currently PR-style KVM keeps the volatile guest register values (R0 - R13, CR, LR, CTR, XER, PC) in a shadow_vcpu struct rather than the main kvm_vcpu struct. For 64-bit, the shadow_vcpu exists in two places, a kmalloc'd struct and in the PACA, and it gets copied back and forth in kvmppc_core_vcpu_load/put(), because the real-mode code can't rely on being able to access the kmalloc'd struct. This changes the code to copy the volatile values into the shadow_vcpu as one of the last things done before entering the guest. Similarly the values are copied back out of the shadow_vcpu to the kvm_vcpu immediately after exiting the guest. We arrange for interrupts to be still disabled at this point so that we can't get preempted on 64-bit and end up copying values from the wrong PACA. This means that the accessor functions in kvm_book3s.h for these registers are greatly simplified, and are same between PR and HV KVM. In places where accesses to shadow_vcpu fields are now replaced by accesses to the kvm_vcpu, we can also remove the svcpu_get/put pairs. Finally, on 64-bit, we don't need the kmalloc'd struct at all any more. With this, the time to read the PVR one million times in a loop went from 567.7ms to 575.5ms (averages of 6 values), an increase of about 1.4% for this worse-case test for guest entries and exits. The standard deviation of the measurements is about 11ms, so the difference is only marginally significant statistically. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-09-19 22:52:43 -06:00
#define GET_SHADOW_VCPU(reg) addi reg, r13, PACA_SVCPU
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
#elif defined(CONFIG_PPC_BOOK3S_32)
#define FUNC(name) name
KVM: PPC: Book3S PR: Keep volatile reg values in vcpu rather than shadow_vcpu Currently PR-style KVM keeps the volatile guest register values (R0 - R13, CR, LR, CTR, XER, PC) in a shadow_vcpu struct rather than the main kvm_vcpu struct. For 64-bit, the shadow_vcpu exists in two places, a kmalloc'd struct and in the PACA, and it gets copied back and forth in kvmppc_core_vcpu_load/put(), because the real-mode code can't rely on being able to access the kmalloc'd struct. This changes the code to copy the volatile values into the shadow_vcpu as one of the last things done before entering the guest. Similarly the values are copied back out of the shadow_vcpu to the kvm_vcpu immediately after exiting the guest. We arrange for interrupts to be still disabled at this point so that we can't get preempted on 64-bit and end up copying values from the wrong PACA. This means that the accessor functions in kvm_book3s.h for these registers are greatly simplified, and are same between PR and HV KVM. In places where accesses to shadow_vcpu fields are now replaced by accesses to the kvm_vcpu, we can also remove the svcpu_get/put pairs. Finally, on 64-bit, we don't need the kmalloc'd struct at all any more. With this, the time to read the PVR one million times in a loop went from 567.7ms to 575.5ms (averages of 6 values), an increase of about 1.4% for this worse-case test for guest entries and exits. The standard deviation of the measurements is about 11ms, so the difference is only marginally significant statistically. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-09-19 22:52:43 -06:00
#define GET_SHADOW_VCPU(reg) lwz reg, (THREAD + THREAD_KVM_SVCPU)(r2)
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
#endif /* CONFIG_PPC_BOOK3S_XX */
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
#define VCPU_LOAD_NVGPRS(vcpu) \
powerpc: Fix usage of register macros getting ready for %r0 change Anything that uses a constructed instruction (ie. from ppc-opcode.h), need to use the new R0 macro, as %r0 is not going to work. Also convert usages of macros where we are just determining an offset (usually for a load/store), like: std r14,STK_REG(r14)(r1) Can't use STK_REG(r14) as %r14 doesn't work in the STK_REG macro since it's just calculating an offset. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 07:33:10 -06:00
PPC_LL r14, VCPU_GPR(R14)(vcpu); \
PPC_LL r15, VCPU_GPR(R15)(vcpu); \
PPC_LL r16, VCPU_GPR(R16)(vcpu); \
PPC_LL r17, VCPU_GPR(R17)(vcpu); \
PPC_LL r18, VCPU_GPR(R18)(vcpu); \
PPC_LL r19, VCPU_GPR(R19)(vcpu); \
PPC_LL r20, VCPU_GPR(R20)(vcpu); \
PPC_LL r21, VCPU_GPR(R21)(vcpu); \
PPC_LL r22, VCPU_GPR(R22)(vcpu); \
PPC_LL r23, VCPU_GPR(R23)(vcpu); \
PPC_LL r24, VCPU_GPR(R24)(vcpu); \
PPC_LL r25, VCPU_GPR(R25)(vcpu); \
PPC_LL r26, VCPU_GPR(R26)(vcpu); \
PPC_LL r27, VCPU_GPR(R27)(vcpu); \
PPC_LL r28, VCPU_GPR(R28)(vcpu); \
PPC_LL r29, VCPU_GPR(R29)(vcpu); \
PPC_LL r30, VCPU_GPR(R30)(vcpu); \
PPC_LL r31, VCPU_GPR(R31)(vcpu); \
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
/*****************************************************************************
* *
* Guest entry / exit code that is in kernel module memory (highmem) *
* *
****************************************************************************/
/* Registers:
* r3: kvm_run pointer
* r4: vcpu pointer
*/
KVM: PPC: Move guest enter/exit down into subarch-specific code Instead of doing the kvm_guest_enter/exit() and local_irq_dis/enable() calls in powerpc.c, this moves them down into the subarch-specific book3s_pr.c and booke.c. This eliminates an extra local_irq_enable() call in book3s_pr.c, and will be needed for when we do SMT4 guest support in the book3s hypervisor mode code. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:19:50 -06:00
_GLOBAL(__kvmppc_vcpu_run)
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
kvm_start_entry:
/* Write correct stack frame */
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
mflr r0
PPC_STL r0,PPC_LR_STKOFF(r1)
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
/* Save host state to the stack */
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
PPC_STLU r1, -SWITCH_FRAME_SIZE(r1)
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
/* Save r3 (kvm_run) and r4 (vcpu) */
SAVE_2GPRS(3, r1)
/* Save non-volatile registers (r14 - r31) */
SAVE_NVGPRS(r1)
KVM: PPC: Save/Restore CR over vcpu_run On PPC, CR2-CR4 are nonvolatile, thus have to be saved across function calls. We didn't respect that for any architecture until Paul spotted it in his patch for Book3S-HV. This patch saves/restores CR for all KVM capable PPC hosts. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Paul Mackerras <paulus@samba.org>
2012-03-05 08:00:28 -07:00
/* Save CR */
mfcr r14
stw r14, _CCR(r1)
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
/* Save LR */
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
PPC_STL r0, _LINK(r1)
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
/* Load non-volatile guest state from the vcpu */
VCPU_LOAD_NVGPRS(r4)
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
kvm_start_lightweight:
KVM: PPC: Book3S PR: Keep volatile reg values in vcpu rather than shadow_vcpu Currently PR-style KVM keeps the volatile guest register values (R0 - R13, CR, LR, CTR, XER, PC) in a shadow_vcpu struct rather than the main kvm_vcpu struct. For 64-bit, the shadow_vcpu exists in two places, a kmalloc'd struct and in the PACA, and it gets copied back and forth in kvmppc_core_vcpu_load/put(), because the real-mode code can't rely on being able to access the kmalloc'd struct. This changes the code to copy the volatile values into the shadow_vcpu as one of the last things done before entering the guest. Similarly the values are copied back out of the shadow_vcpu to the kvm_vcpu immediately after exiting the guest. We arrange for interrupts to be still disabled at this point so that we can't get preempted on 64-bit and end up copying values from the wrong PACA. This means that the accessor functions in kvm_book3s.h for these registers are greatly simplified, and are same between PR and HV KVM. In places where accesses to shadow_vcpu fields are now replaced by accesses to the kvm_vcpu, we can also remove the svcpu_get/put pairs. Finally, on 64-bit, we don't need the kmalloc'd struct at all any more. With this, the time to read the PVR one million times in a loop went from 567.7ms to 575.5ms (averages of 6 values), an increase of about 1.4% for this worse-case test for guest entries and exits. The standard deviation of the measurements is about 11ms, so the difference is only marginally significant statistically. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-09-19 22:52:43 -06:00
/* Copy registers into shadow vcpu so we can access them in real mode */
KVM: PPC: Book3S PR: Fix svcpu copying with preemption enabled When copying between the vcpu and svcpu, we may get scheduled away onto a different host CPU which in turn means our svcpu pointer may change. That means we need to atomically copy to and from the svcpu with preemption disabled, so that all code around it always sees a coherent state. Reported-by: Simon Guo <wei.guo.simon@gmail.com> Fixes: 3d3319b45eea ("KVM: PPC: Book3S: PR: Enable interrupts earlier") Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2018-01-31 14:24:58 -07:00
mr r3, r4
KVM: PPC: Book3S PR: Keep volatile reg values in vcpu rather than shadow_vcpu Currently PR-style KVM keeps the volatile guest register values (R0 - R13, CR, LR, CTR, XER, PC) in a shadow_vcpu struct rather than the main kvm_vcpu struct. For 64-bit, the shadow_vcpu exists in two places, a kmalloc'd struct and in the PACA, and it gets copied back and forth in kvmppc_core_vcpu_load/put(), because the real-mode code can't rely on being able to access the kmalloc'd struct. This changes the code to copy the volatile values into the shadow_vcpu as one of the last things done before entering the guest. Similarly the values are copied back out of the shadow_vcpu to the kvm_vcpu immediately after exiting the guest. We arrange for interrupts to be still disabled at this point so that we can't get preempted on 64-bit and end up copying values from the wrong PACA. This means that the accessor functions in kvm_book3s.h for these registers are greatly simplified, and are same between PR and HV KVM. In places where accesses to shadow_vcpu fields are now replaced by accesses to the kvm_vcpu, we can also remove the svcpu_get/put pairs. Finally, on 64-bit, we don't need the kmalloc'd struct at all any more. With this, the time to read the PVR one million times in a loop went from 567.7ms to 575.5ms (averages of 6 values), an increase of about 1.4% for this worse-case test for guest entries and exits. The standard deviation of the measurements is about 11ms, so the difference is only marginally significant statistically. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-09-19 22:52:43 -06:00
bl FUNC(kvmppc_copy_to_svcpu)
nop
REST_GPR(4, r1)
KVM: PPC: Use PACA backed shadow vcpu We're being horribly racy right now. All the entry and exit code hijacks random fields from the PACA that could easily be used by different code in case we get interrupted, for example by a #MC or even page fault. After discussing this with Ben, we figured it's best to reserve some more space in the PACA and just shove off some vcpu state to there. That way we can drastically improve the readability of the code, make it less racy and less complex. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-07 18:58:03 -07:00
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
#ifdef CONFIG_PPC_BOOK3S_64
KVM: PPC: Book3S PR: Keep volatile reg values in vcpu rather than shadow_vcpu Currently PR-style KVM keeps the volatile guest register values (R0 - R13, CR, LR, CTR, XER, PC) in a shadow_vcpu struct rather than the main kvm_vcpu struct. For 64-bit, the shadow_vcpu exists in two places, a kmalloc'd struct and in the PACA, and it gets copied back and forth in kvmppc_core_vcpu_load/put(), because the real-mode code can't rely on being able to access the kmalloc'd struct. This changes the code to copy the volatile values into the shadow_vcpu as one of the last things done before entering the guest. Similarly the values are copied back out of the shadow_vcpu to the kvm_vcpu immediately after exiting the guest. We arrange for interrupts to be still disabled at this point so that we can't get preempted on 64-bit and end up copying values from the wrong PACA. This means that the accessor functions in kvm_book3s.h for these registers are greatly simplified, and are same between PR and HV KVM. In places where accesses to shadow_vcpu fields are now replaced by accesses to the kvm_vcpu, we can also remove the svcpu_get/put pairs. Finally, on 64-bit, we don't need the kmalloc'd struct at all any more. With this, the time to read the PVR one million times in a loop went from 567.7ms to 575.5ms (averages of 6 values), an increase of about 1.4% for this worse-case test for guest entries and exits. The standard deviation of the measurements is about 11ms, so the difference is only marginally significant statistically. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-09-19 22:52:43 -06:00
/* Get the dcbz32 flag */
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
PPC_LL r3, VCPU_HFLAGS(r4)
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
rldicl r3, r3, 0, 63 /* r3 &= 1 */
stb r3, HSTATE_RESTORE_HID5(r13)
KVM: PPC: Book3S PR: Load up SPRG3 register with guest value on guest entry Unlike the other general-purpose SPRs, SPRG3 can be read by usermode code, and is used in recent kernels to store the CPU and NUMA node numbers so that they can be read by VDSO functions. Thus we need to load the guest's SPRG3 value into the real SPRG3 register when entering the guest, and restore the host's value when exiting the guest. We don't need to save the guest SPRG3 value when exiting the guest as usermode code can't modify SPRG3. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-07-11 05:49:43 -06:00
/* Load up guest SPRG3 value, since it's user readable */
KVM: PPC: Make shared struct aka magic page guest endian The shared (magic) page is a data structure that contains often used supervisor privileged SPRs accessible via memory to the user to reduce the number of exits we have to take to read/write them. When we actually share this structure with the guest we have to maintain it in guest endianness, because some of the patch tricks only work with native endian load/store operations. Since we only share the structure with either host or guest in little endian on book3s_64 pr mode, we don't have to worry about booke or book3s hv. For booke, the shared struct stays big endian. For book3s_64 hv we maintain the struct in host native endian, since it never gets shared with the guest. For book3s_64 pr we introduce a variable that tells us which endianness the shared struct is in and route every access to it through helper inline functions that evaluate this variable. Signed-off-by: Alexander Graf <agraf@suse.de>
2014-04-24 05:46:24 -06:00
lwz r3, VCPU_SHAREDBE(r4)
cmpwi r3, 0
ld r5, VCPU_SHARED(r4)
beq sprg3_little_endian
sprg3_big_endian:
#ifdef __BIG_ENDIAN__
ld r3, VCPU_SHARED_SPRG3(r5)
#else
addi r5, r5, VCPU_SHARED_SPRG3
ldbrx r3, 0, r5
#endif
b after_sprg3_load
sprg3_little_endian:
#ifdef __LITTLE_ENDIAN__
ld r3, VCPU_SHARED_SPRG3(r5)
#else
addi r5, r5, VCPU_SHARED_SPRG3
ldbrx r3, 0, r5
#endif
after_sprg3_load:
KVM: PPC: Book3S PR: Load up SPRG3 register with guest value on guest entry Unlike the other general-purpose SPRs, SPRG3 can be read by usermode code, and is used in recent kernels to store the CPU and NUMA node numbers so that they can be read by VDSO functions. Thus we need to load the guest's SPRG3 value into the real SPRG3 register when entering the guest, and restore the host's value when exiting the guest. We don't need to save the guest SPRG3 value when exiting the guest as usermode code can't modify SPRG3. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-07-11 05:49:43 -06:00
mtspr SPRN_SPRG3, r3
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
#endif /* CONFIG_PPC_BOOK3S_64 */
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
PPC_LL r4, VCPU_SHADOW_MSR(r4) /* get shadow_msr */
KVM: PPC: Use PACA backed shadow vcpu We're being horribly racy right now. All the entry and exit code hijacks random fields from the PACA that could easily be used by different code in case we get interrupted, for example by a #MC or even page fault. After discussing this with Ben, we figured it's best to reserve some more space in the PACA and just shove off some vcpu state to there. That way we can drastically improve the readability of the code, make it less racy and less complex. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-01-07 18:58:03 -07:00
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
/* Jump to segment patching handler and into our guest */
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
bl FUNC(kvmppc_entry_trampoline)
nop
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
/*
* This is the handler in module memory. It gets jumped at from the
* lowmem trampoline code, so it's basically the guest exit code.
*
*/
/*
* Register usage at this point:
*
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
* R1 = host R1
* R2 = host R2
* R12 = exit handler id
* R13 = PACA
* SVCPU.* = guest *
KVM: PPC: Book3S: PR: Enable interrupts earlier Now that the svcpu sync is interrupt aware we can enable interrupts earlier in the exit code path again, moving 32bit and 64bit closer together. While at it, document the fact that we're always executing the exit path with interrupts enabled so that the next person doesn't trap over this. Signed-off-by: Alexander Graf <agraf@suse.de>
2013-11-28 18:32:31 -07:00
* MSR.EE = 1
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
*
*/
KVM: PPC: Book3S: PR: Don't clobber our exit handler id We call a C helper to save all svcpu fields into our vcpu. The C ABI states that r12 is considered volatile. However, we keep our exit handler id in r12 currently. So we need to save it away into a non-volatile register instead that definitely does get preserved across the C call. This bug usually didn't hit anyone yet since gcc is smart enough to generate code that doesn't even need r12 which means it stayed identical throughout the call by sheer luck. But we can't rely on that. Signed-off-by: Alexander Graf <agraf@suse.de>
2013-11-28 18:24:18 -07:00
PPC_LL r3, GPR4(r1) /* vcpu pointer */
/*
* kvmppc_copy_from_svcpu can clobber volatile registers, save
* the exit handler id to the vcpu and restore it from there later.
*/
stw r12, VCPU_TRAP(r3)
KVM: PPC: Book3S PR: Keep volatile reg values in vcpu rather than shadow_vcpu Currently PR-style KVM keeps the volatile guest register values (R0 - R13, CR, LR, CTR, XER, PC) in a shadow_vcpu struct rather than the main kvm_vcpu struct. For 64-bit, the shadow_vcpu exists in two places, a kmalloc'd struct and in the PACA, and it gets copied back and forth in kvmppc_core_vcpu_load/put(), because the real-mode code can't rely on being able to access the kmalloc'd struct. This changes the code to copy the volatile values into the shadow_vcpu as one of the last things done before entering the guest. Similarly the values are copied back out of the shadow_vcpu to the kvm_vcpu immediately after exiting the guest. We arrange for interrupts to be still disabled at this point so that we can't get preempted on 64-bit and end up copying values from the wrong PACA. This means that the accessor functions in kvm_book3s.h for these registers are greatly simplified, and are same between PR and HV KVM. In places where accesses to shadow_vcpu fields are now replaced by accesses to the kvm_vcpu, we can also remove the svcpu_get/put pairs. Finally, on 64-bit, we don't need the kmalloc'd struct at all any more. With this, the time to read the PVR one million times in a loop went from 567.7ms to 575.5ms (averages of 6 values), an increase of about 1.4% for this worse-case test for guest entries and exits. The standard deviation of the measurements is about 11ms, so the difference is only marginally significant statistically. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-09-19 22:52:43 -06:00
/* Transfer reg values from shadow vcpu back to vcpu struct */
KVM: PPC: Book3S: PR: Don't clobber our exit handler id We call a C helper to save all svcpu fields into our vcpu. The C ABI states that r12 is considered volatile. However, we keep our exit handler id in r12 currently. So we need to save it away into a non-volatile register instead that definitely does get preserved across the C call. This bug usually didn't hit anyone yet since gcc is smart enough to generate code that doesn't even need r12 which means it stayed identical throughout the call by sheer luck. But we can't rely on that. Signed-off-by: Alexander Graf <agraf@suse.de>
2013-11-28 18:24:18 -07:00
KVM: PPC: Book3S PR: Keep volatile reg values in vcpu rather than shadow_vcpu Currently PR-style KVM keeps the volatile guest register values (R0 - R13, CR, LR, CTR, XER, PC) in a shadow_vcpu struct rather than the main kvm_vcpu struct. For 64-bit, the shadow_vcpu exists in two places, a kmalloc'd struct and in the PACA, and it gets copied back and forth in kvmppc_core_vcpu_load/put(), because the real-mode code can't rely on being able to access the kmalloc'd struct. This changes the code to copy the volatile values into the shadow_vcpu as one of the last things done before entering the guest. Similarly the values are copied back out of the shadow_vcpu to the kvm_vcpu immediately after exiting the guest. We arrange for interrupts to be still disabled at this point so that we can't get preempted on 64-bit and end up copying values from the wrong PACA. This means that the accessor functions in kvm_book3s.h for these registers are greatly simplified, and are same between PR and HV KVM. In places where accesses to shadow_vcpu fields are now replaced by accesses to the kvm_vcpu, we can also remove the svcpu_get/put pairs. Finally, on 64-bit, we don't need the kmalloc'd struct at all any more. With this, the time to read the PVR one million times in a loop went from 567.7ms to 575.5ms (averages of 6 values), an increase of about 1.4% for this worse-case test for guest entries and exits. The standard deviation of the measurements is about 11ms, so the difference is only marginally significant statistically. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-09-19 22:52:43 -06:00
bl FUNC(kvmppc_copy_from_svcpu)
nop
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
KVM: PPC: Book3S PR: Load up SPRG3 register with guest value on guest entry Unlike the other general-purpose SPRs, SPRG3 can be read by usermode code, and is used in recent kernels to store the CPU and NUMA node numbers so that they can be read by VDSO functions. Thus we need to load the guest's SPRG3 value into the real SPRG3 register when entering the guest, and restore the host's value when exiting the guest. We don't need to save the guest SPRG3 value when exiting the guest as usermode code can't modify SPRG3. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-07-11 05:49:43 -06:00
#ifdef CONFIG_PPC_BOOK3S_64
/*
* Reload kernel SPRG3 value.
* No need to save guest value as usermode can't modify SPRG3.
*/
powerpc/booke64: Use SPRG7 for VDSO Previously SPRG3 was marked for use by both VDSO and critical interrupts (though critical interrupts were not fully implemented). In commit 8b64a9dfb091f1eca8b7e58da82f1e7d1d5fe0ad ("powerpc/booke64: Use SPRG0/3 scratch for bolted TLB miss & crit int"), Mihai Caraman made an attempt to resolve this conflict by restoring the VDSO value early in the critical interrupt, but this has some issues: - It's incompatible with EXCEPTION_COMMON which restores r13 from the by-then-overwritten scratch (this cost me some debugging time). - It forces critical exceptions to be a special case handled differently from even machine check and debug level exceptions. - It didn't occur to me that it was possible to make this work at all (by doing a final "ld r13, PACA_EXCRIT+EX_R13(r13)") until after I made (most of) this patch. :-) It might be worth investigating using a load rather than SPRG on return from all exceptions (except TLB misses where the scratch never leaves the SPRG) -- it could save a few cycles. Until then, let's stick with SPRG for all exceptions. Since we cannot use SPRG4-7 for scratch without corrupting the state of a KVM guest, move VDSO to SPRG7 on book3e. Since neither SPRG4-7 nor critical interrupts exist on book3s, SPRG3 is still used for VDSO there. Signed-off-by: Scott Wood <scottwood@freescale.com> Cc: Mihai Caraman <mihai.caraman@freescale.com> Cc: Anton Blanchard <anton@samba.org> Cc: Paul Mackerras <paulus@samba.org> Cc: kvm-ppc@vger.kernel.org
2014-03-10 16:29:38 -06:00
ld r3, PACA_SPRG_VDSO(r13)
mtspr SPRN_SPRG_VDSO_WRITE, r3
KVM: PPC: Book3S PR: Load up SPRG3 register with guest value on guest entry Unlike the other general-purpose SPRs, SPRG3 can be read by usermode code, and is used in recent kernels to store the CPU and NUMA node numbers so that they can be read by VDSO functions. Thus we need to load the guest's SPRG3 value into the real SPRG3 register when entering the guest, and restore the host's value when exiting the guest. We don't need to save the guest SPRG3 value when exiting the guest as usermode code can't modify SPRG3. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-07-11 05:49:43 -06:00
#endif /* CONFIG_PPC_BOOK3S_64 */
KVM: PPC: Book3S PR: Keep volatile reg values in vcpu rather than shadow_vcpu Currently PR-style KVM keeps the volatile guest register values (R0 - R13, CR, LR, CTR, XER, PC) in a shadow_vcpu struct rather than the main kvm_vcpu struct. For 64-bit, the shadow_vcpu exists in two places, a kmalloc'd struct and in the PACA, and it gets copied back and forth in kvmppc_core_vcpu_load/put(), because the real-mode code can't rely on being able to access the kmalloc'd struct. This changes the code to copy the volatile values into the shadow_vcpu as one of the last things done before entering the guest. Similarly the values are copied back out of the shadow_vcpu to the kvm_vcpu immediately after exiting the guest. We arrange for interrupts to be still disabled at this point so that we can't get preempted on 64-bit and end up copying values from the wrong PACA. This means that the accessor functions in kvm_book3s.h for these registers are greatly simplified, and are same between PR and HV KVM. In places where accesses to shadow_vcpu fields are now replaced by accesses to the kvm_vcpu, we can also remove the svcpu_get/put pairs. Finally, on 64-bit, we don't need the kmalloc'd struct at all any more. With this, the time to read the PVR one million times in a loop went from 567.7ms to 575.5ms (averages of 6 values), an increase of about 1.4% for this worse-case test for guest entries and exits. The standard deviation of the measurements is about 11ms, so the difference is only marginally significant statistically. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-09-19 22:52:43 -06:00
/* R7 = vcpu */
PPC_LL r7, GPR4(r1)
powerpc: Fix usage of register macros getting ready for %r0 change Anything that uses a constructed instruction (ie. from ppc-opcode.h), need to use the new R0 macro, as %r0 is not going to work. Also convert usages of macros where we are just determining an offset (usually for a load/store), like: std r14,STK_REG(r14)(r1) Can't use STK_REG(r14) as %r14 doesn't work in the STK_REG macro since it's just calculating an offset. Signed-off-by: Michael Neuling <mikey@neuling.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2012-06-25 07:33:10 -06:00
PPC_STL r14, VCPU_GPR(R14)(r7)
PPC_STL r15, VCPU_GPR(R15)(r7)
PPC_STL r16, VCPU_GPR(R16)(r7)
PPC_STL r17, VCPU_GPR(R17)(r7)
PPC_STL r18, VCPU_GPR(R18)(r7)
PPC_STL r19, VCPU_GPR(R19)(r7)
PPC_STL r20, VCPU_GPR(R20)(r7)
PPC_STL r21, VCPU_GPR(R21)(r7)
PPC_STL r22, VCPU_GPR(R22)(r7)
PPC_STL r23, VCPU_GPR(R23)(r7)
PPC_STL r24, VCPU_GPR(R24)(r7)
PPC_STL r25, VCPU_GPR(R25)(r7)
PPC_STL r26, VCPU_GPR(R26)(r7)
PPC_STL r27, VCPU_GPR(R27)(r7)
PPC_STL r28, VCPU_GPR(R28)(r7)
PPC_STL r29, VCPU_GPR(R29)(r7)
PPC_STL r30, VCPU_GPR(R30)(r7)
PPC_STL r31, VCPU_GPR(R31)(r7)
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
/* Pass the exit number as 3rd argument to kvmppc_handle_exit */
KVM: PPC: Book3S: PR: Don't clobber our exit handler id We call a C helper to save all svcpu fields into our vcpu. The C ABI states that r12 is considered volatile. However, we keep our exit handler id in r12 currently. So we need to save it away into a non-volatile register instead that definitely does get preserved across the C call. This bug usually didn't hit anyone yet since gcc is smart enough to generate code that doesn't even need r12 which means it stayed identical throughout the call by sheer luck. But we can't rely on that. Signed-off-by: Alexander Graf <agraf@suse.de>
2013-11-28 18:24:18 -07:00
lwz r5, VCPU_TRAP(r7)
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
/* Restore r3 (kvm_run) and r4 (vcpu) */
REST_2GPRS(3, r1)
kvm: powerpc: Add kvmppc_ops callback This patch add a new callback kvmppc_ops. This will help us in enabling both HV and PR KVM together in the same kernel. The actual change to enable them together is done in the later patch in the series. Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> [agraf: squash in booke changes] Signed-off-by: Alexander Graf <agraf@suse.de>
2013-10-07 10:47:53 -06:00
bl FUNC(kvmppc_handle_exit_pr)
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
/* If RESUME_GUEST, get back in the loop */
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
cmpwi r3, RESUME_GUEST
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
beq kvm_loop_lightweight
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
cmpwi r3, RESUME_GUEST_NV
beq kvm_loop_heavyweight
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
kvm_exit_loop:
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
PPC_LL r4, _LINK(r1)
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
mtlr r4
KVM: PPC: Save/Restore CR over vcpu_run On PPC, CR2-CR4 are nonvolatile, thus have to be saved across function calls. We didn't respect that for any architecture until Paul spotted it in his patch for Book3S-HV. This patch saves/restores CR for all KVM capable PPC hosts. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Paul Mackerras <paulus@samba.org>
2012-03-05 08:00:28 -07:00
lwz r14, _CCR(r1)
mtcr r14
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
/* Restore non-volatile host registers (r14 - r31) */
REST_NVGPRS(r1)
addi r1, r1, SWITCH_FRAME_SIZE
blr
kvm_loop_heavyweight:
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
KVM: PPC: Make highmem code generic Since we now have several fields in the shadow VCPU, we also change the internal calling convention between the different entry/exit code layers. Let's reflect that in the IR=1 code and make sure we use "long" defines for long field access. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:47 -06:00
PPC_LL r4, _LINK(r1)
PPC_STL r4, (PPC_LR_STKOFF + SWITCH_FRAME_SIZE)(r1)
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
/* Load vcpu and cpu_run */
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
REST_2GPRS(3, r1)
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
/* Load non-volatile guest state from the vcpu */
VCPU_LOAD_NVGPRS(r4)
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
/* Jump back into the beginning of this function */
b kvm_start_lightweight
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
kvm_loop_lightweight:
Add book3s_64 highmem asm code This is the of entry / exit code. In order to switch between host and guest context, we need to switch register state and call the exit code handler on exit. This assembly file does exactly that. To finally enter the guest it calls into book3s_64_slb.S. On exit it gets jumped at from book3s_64_slb.S too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2009-10-29 23:47:07 -06:00
KVM: PPC: Enable lightweight exits again The PowerPC C ABI defines that registers r14-r31 need to be preserved across function calls. Since our exit handler is written in C, we can make use of that and don't need to reload r14-r31 on every entry/exit cycle. This technique is also used in the BookE code and is called "lightweight exits" there. To follow the tradition, it's called the same in Book3S. So far this optimization was disabled though, as the code didn't do what it was expected to do, but failed to work. This patch fixes and enables lightweight exits again. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
2010-01-04 14:19:25 -07:00
/* We'll need the vcpu pointer */
REST_GPR(4, r1)
/* Jump back into the beginning of this function */
b kvm_start_lightweight