redonkable
/
alistair23-linux
1
0
Fork 0
Code Releases Activity
alistair23-linux/arch/powerpc/kvm/book3s_segment.S

417 lines
10 KiB
ArmAsm
Raw Normal View History

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 266 Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation this program is distributed in the hope that it will be useful but without any warranty without even the implied warranty of merchantability or fitness for a particular purpose see the gnu general public license for more details you should have received a copy of the gnu general public license along with this program if not write to the free software foundation 51 franklin street fifth floor boston ma 02110 1301 usa extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 67 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Allison Randal <allison@lohutok.net> Reviewed-by: Richard Fontana <rfontana@redhat.com> Reviewed-by: Alexios Zavras <alexios.zavras@intel.com> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190529141333.953658117@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-29 08:12:40 -06:00
/* SPDX-License-Identifier: GPL-2.0-only */
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
/*
*
* Copyright SUSE Linux Products GmbH 2010
*
* Authors: Alexander Graf <agraf@suse.de>
*/
/* Real mode helpers */
powerpc: move ASM_CONST and stringify_in_c() into asm-const.h This patch moves ASM_CONST() and stringify_in_c() into dedicated asm-const.h, then cleans all related inclusions. Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> [mpe: asm-compat.h should include asm-const.h] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-07-05 10:24:57 -06:00
#include <asm/asm-compat.h>
powerpc: clean inclusions of asm/feature-fixups.h files not using feature fixup don't need asm/feature-fixups.h files using feature fixup need asm/feature-fixups.h Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-07-05 10:25:01 -06:00
#include <asm/feature-fixups.h>
powerpc: move ASM_CONST and stringify_in_c() into asm-const.h This patch moves ASM_CONST() and stringify_in_c() into dedicated asm-const.h, then cleans all related inclusions. Signed-off-by: Christophe Leroy <christophe.leroy@c-s.fr> [mpe: asm-compat.h should include asm-const.h] Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-07-05 10:24:57 -06:00
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
#if defined(CONFIG_PPC_BOOK3S_64)
#define GET_SHADOW_VCPU(reg) \
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
mr reg, r13
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
#elif defined(CONFIG_PPC_BOOK3S_32)
#define GET_SHADOW_VCPU(reg) \
tophys(reg, r2); \
lwz reg, (THREAD + THREAD_KVM_SVCPU)(reg); \
tophys(reg, reg)
#endif
/* Disable for nested KVM */
#define USE_QUICK_LAST_INST
/* Get helper functions for subarch specific functionality */
#if defined(CONFIG_PPC_BOOK3S_64)
#include "book3s_64_slb.S"
#elif defined(CONFIG_PPC_BOOK3S_32)
#include "book3s_32_sr.S"
#endif
/******************************************************************************
* *
* Entry code *
* *
*****************************************************************************/
.global kvmppc_handler_trampoline_enter
kvmppc_handler_trampoline_enter:
/* Required state:
*
* MSR = ~IR|DR
* R1 = host R1
* R2 = host R2
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
* R4 = guest shadow MSR
* R5 = normal host MSR
* R6 = current host MSR (EE, IR, DR off)
* LR = highmem guest exit code
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
* all other volatile GPRS = free
* SVCPU[CR] = guest CR
* SVCPU[XER] = guest XER
* SVCPU[CTR] = guest CTR
* SVCPU[LR] = guest LR
*/
/* r3 = shadow vcpu */
GET_SHADOW_VCPU(r3)
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
/* Save guest exit handler address and MSR */
mflr r0
PPC_STL r0, HSTATE_VMHANDLER(r3)
PPC_STL r5, HSTATE_HOST_MSR(r3)
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
/* Save R1/R2 in the PACA (64-bit) or shadow_vcpu (32-bit) */
PPC_STL r1, HSTATE_HOST_R1(r3)
PPC_STL r2, HSTATE_HOST_R2(r3)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
/* Activate guest mode, so faults get handled by KVM */
li r11, KVM_GUEST_MODE_GUEST
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
stb r11, HSTATE_IN_GUEST(r3)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
/* Switch to guest segment. This is subarch specific. */
LOAD_GUEST_SEGMENTS
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
#ifdef CONFIG_PPC_BOOK3S_64
KVM: PPC: Book3S PR: Handle Facility interrupt and FSCR POWER8 introduced a new interrupt type called "Facility unavailable interrupt" which contains its status message in a new register called FSCR. Handle these exits and try to emulate instructions for unhandled facilities. Follow-on patches enable KVM to expose specific facilities into the guest. Signed-off-by: Alexander Graf <agraf@suse.de>
2014-04-29 08:48:44 -06:00
BEGIN_FTR_SECTION
/* Save host FSCR */
mfspr r8, SPRN_FSCR
std r8, HSTATE_HOST_FSCR(r13)
/* Set FSCR during guest execution */
ld r9, SVCPU_SHADOW_FSCR(r13)
mtspr SPRN_FSCR, r9
END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
/* Some guests may need to have dcbz set to 32 byte length.
*
* Usually we ensure that by patching the guest's instructions
* to trap on dcbz and emulate it in the hypervisor.
*
* If we can, we should tell the CPU to use 32 byte dcbz though,
* because that's a lot faster.
*/
lbz r0, HSTATE_RESTORE_HID5(r3)
cmpwi r0, 0
beq no_dcbz32_on
mfspr r0,SPRN_HID5
ori r0, r0, 0x80 /* XXX HID5_dcbz32 = 0x80 */
mtspr SPRN_HID5,r0
no_dcbz32_on:
#endif /* CONFIG_PPC_BOOK3S_64 */
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
/* Enter guest */
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
PPC_LL r8, SVCPU_CTR(r3)
PPC_LL r9, SVCPU_LR(r3)
lwz r10, SVCPU_CR(r3)
KVM: PPC: Book3S: correct width in XER handling In 64 bit kernels, the Fixed Point Exception Register (XER) is a 64 bit field (e.g. in kvm_regs and kvm_vcpu_arch) and in most places it is accessed as such. This patch corrects places where it is accessed as a 32 bit field by a 64 bit kernel. In some cases this is via a 32 bit load or store instruction which, depending on endianness, will cause either the lower or upper 32 bits to be missed. In another case it is cast as a u32, causing the upper 32 bits to be cleared. This patch corrects those places by extending the access methods to 64 bits. Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com> Reviewed-by: Laurent Vivier <lvivier@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Tested-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alexander Graf <agraf@suse.de>
2015-05-26 17:56:57 -06:00
PPC_LL r11, SVCPU_XER(r3)
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
mtctr r8
mtlr r9
mtcr r10
mtxer r11
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
/* Move SRR0 and SRR1 into the respective regs */
PPC_LL r9, SVCPU_PC(r3)
/* First clear RI in our current MSR value */
li r0, MSR_RI
andc r6, r6, r0
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
KVM: PPC: Add support for Book3S processors in hypervisor mode This adds support for KVM running on 64-bit Book 3S processors, specifically POWER7, in hypervisor mode. Using hypervisor mode means that the guest can use the processor's supervisor mode. That means that the guest can execute privileged instructions and access privileged registers itself without trapping to the host. This gives excellent performance, but does mean that KVM cannot emulate a processor architecture other than the one that the hardware implements. This code assumes that the guest is running paravirtualized using the PAPR (Power Architecture Platform Requirements) interface, which is the interface that IBM's PowerVM hypervisor uses. That means that existing Linux distributions that run on IBM pSeries machines will also run under KVM without modification. In order to communicate the PAPR hypercalls to qemu, this adds a new KVM_EXIT_PAPR_HCALL exit code to include/linux/kvm.h. Currently the choice between book3s_hv support and book3s_pr support (i.e. the existing code, which runs the guest in user mode) has to be made at kernel configuration time, so a given kernel binary can only do one or the other. This new book3s_hv code doesn't support MMIO emulation at present. Since we are running paravirtualized guests, this isn't a serious restriction. With the guest running in supervisor mode, most exceptions go straight to the guest. We will never get data or instruction storage or segment interrupts, alignment interrupts, decrementer interrupts, program interrupts, single-step interrupts, etc., coming to the hypervisor from the guest. Therefore this introduces a new KVMTEST_NONHV macro for the exception entry path so that we don't have to do the KVM test on entry to those exception handlers. We do however get hypervisor decrementer, hypervisor data storage, hypervisor instruction storage, and hypervisor emulation assist interrupts, so we have to handle those. In hypervisor mode, real-mode accesses can access all of RAM, not just a limited amount. Therefore we put all the guest state in the vcpu.arch and use the shadow_vcpu in the PACA only for temporary scratch space. We allocate the vcpu with kzalloc rather than vzalloc, and we don't use anything in the kvmppc_vcpu_book3s struct, so we don't allocate it. We don't have a shared page with the guest, but we still need a kvm_vcpu_arch_shared struct to store the values of various registers, so we include one in the vcpu_arch struct. The POWER7 processor has a restriction that all threads in a core have to be in the same partition. MMU-on kernel code counts as a partition (partition 0), so we have to do a partition switch on every entry to and exit from the guest. At present we require the host and guest to run in single-thread mode because of this hardware restriction. This code allocates a hashed page table for the guest and initializes it with HPTEs for the guest's Virtual Real Memory Area (VRMA). We require that the guest memory is allocated using 16MB huge pages, in order to simplify the low-level memory management. This also means that we can get away without tracking paging activity in the host for now, since huge pages can't be paged or swapped. This also adds a few new exports needed by the book3s_hv code. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:21:34 -06:00
PPC_LL r0, SVCPU_R0(r3)
PPC_LL r1, SVCPU_R1(r3)
PPC_LL r2, SVCPU_R2(r3)
PPC_LL r5, SVCPU_R5(r3)
PPC_LL r7, SVCPU_R7(r3)
PPC_LL r8, SVCPU_R8(r3)
PPC_LL r10, SVCPU_R10(r3)
PPC_LL r11, SVCPU_R11(r3)
PPC_LL r12, SVCPU_R12(r3)
PPC_LL r13, SVCPU_R13(r3)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
KVM: PPC: Book3S: PR: Optimize entry path By shuffling a few instructions around we can execute more memory loads in parallel, giving us a small performance boost. With this patch and a simple priviledged SPR access loop guest, I get a speed bump from 2013052 to 2035607 exits per second. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-04-25 06:28:23 -06:00
MTMSR_EERI(r6)
mtsrr0 r9
mtsrr1 r4
PPC_LL r4, SVCPU_R4(r3)
PPC_LL r6, SVCPU_R6(r3)
PPC_LL r9, SVCPU_R9(r3)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
PPC_LL r3, (SVCPU_R3)(r3)
powerpc/64s: Simple RFI macro conversions This commit does simple conversions of rfi/rfid to the new macros that include the expected destination context. By simple we mean cases where there is a single well known destination context, and it's simply a matter of substituting the instruction for the appropriate macro. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-01-09 09:07:15 -07:00
RFI_TO_GUEST
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
kvmppc_handler_trampoline_enter_end:
/******************************************************************************
* *
* Exit code *
* *
*****************************************************************************/
kvm: powerpc: book3s: Cleanup interrupt handling code With this patch if HV is included, interrupts come in to the HV version of the kvmppc_interrupt code, which then jumps to the PR handler, renamed to kvmppc_interrupt_pr, if the guest is a PR guest. This helps in enabling both HV and PR, which we do in later patch Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com> Signed-off-by: Alexander Graf <agraf@suse.de>
2013-10-07 10:47:55 -06:00
.global kvmppc_interrupt_pr
kvmppc_interrupt_pr:
KVM: PPC: Book3S: Change interrupt call to reduce scratch space use on HV Change the calling convention to put the trap number together with CR in two halves of r12, which frees up HSTATE_SCRATCH2 in the HV handler. The 64-bit PR handler entry translates the calling convention back to match the previous call convention (i.e., shared with 32-bit), for simplicity. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Acked-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2016-12-21 11:29:25 -07:00
/* 64-bit entry. Register usage at this point:
*
* SPRG_SCRATCH0 = guest R13
* R12 = (guest CR << 32) | exit handler id
* R13 = PACA
* HSTATE.SCRATCH0 = guest R12
KVM: PPC: Book3S: 64-bit CONFIG_RELOCATABLE support for interrupts 64-bit Book3S exception handlers must find the dynamic kernel base to add to the target address when branching beyond __end_interrupts, in order to support kernel running at non-0 physical address. Support this in KVM by branching with CTR, similarly to regular interrupt handlers. The guest CTR saved in HSTATE_SCRATCH1 and restored after the branch. Without this, the host kernel hangs and crashes randomly when it is running at a non-0 address and a KVM guest is started. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Acked-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-01-26 21:00:34 -07:00
* HSTATE.SCRATCH1 = guest CTR if RELOCATABLE
KVM: PPC: Book3S: Change interrupt call to reduce scratch space use on HV Change the calling convention to put the trap number together with CR in two halves of r12, which frees up HSTATE_SCRATCH2 in the HV handler. The 64-bit PR handler entry translates the calling convention back to match the previous call convention (i.e., shared with 32-bit), for simplicity. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Acked-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2016-12-21 11:29:25 -07:00
*/
#ifdef CONFIG_PPC64
/* Match 32-bit entry */
KVM: PPC: Book3S: 64-bit CONFIG_RELOCATABLE support for interrupts 64-bit Book3S exception handlers must find the dynamic kernel base to add to the target address when branching beyond __end_interrupts, in order to support kernel running at non-0 physical address. Support this in KVM by branching with CTR, similarly to regular interrupt handlers. The guest CTR saved in HSTATE_SCRATCH1 and restored after the branch. Without this, the host kernel hangs and crashes randomly when it is running at a non-0 address and a KVM guest is started. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Acked-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-01-26 21:00:34 -07:00
#ifdef CONFIG_RELOCATABLE
std r9, HSTATE_SCRATCH2(r13)
ld r9, HSTATE_SCRATCH1(r13)
mtctr r9
ld r9, HSTATE_SCRATCH2(r13)
#endif
KVM: PPC: Book3S: Change interrupt call to reduce scratch space use on HV Change the calling convention to put the trap number together with CR in two halves of r12, which frees up HSTATE_SCRATCH2 in the HV handler. The 64-bit PR handler entry translates the calling convention back to match the previous call convention (i.e., shared with 32-bit), for simplicity. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Acked-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2016-12-21 11:29:25 -07:00
rotldi r12, r12, 32 /* Flip R12 halves for stw */
stw r12, HSTATE_SCRATCH1(r13) /* CR is now in the low half */
srdi r12, r12, 32 /* shift trap into low half */
#endif
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:18:26 -06:00
KVM: PPC: Book3S: Change interrupt call to reduce scratch space use on HV Change the calling convention to put the trap number together with CR in two halves of r12, which frees up HSTATE_SCRATCH2 in the HV handler. The 64-bit PR handler entry translates the calling convention back to match the previous call convention (i.e., shared with 32-bit), for simplicity. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Acked-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2016-12-21 11:29:25 -07:00
.global kvmppc_handler_trampoline_exit
kvmppc_handler_trampoline_exit:
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
/* Register usage at this point:
*
KVM: PPC: Book3S: Change interrupt call to reduce scratch space use on HV Change the calling convention to put the trap number together with CR in two halves of r12, which frees up HSTATE_SCRATCH2 in the HV handler. The 64-bit PR handler entry translates the calling convention back to match the previous call convention (i.e., shared with 32-bit), for simplicity. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Acked-by: Paul Mackerras <paulus@ozlabs.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2016-12-21 11:29:25 -07:00
* SPRG_SCRATCH0 = guest R13
* R12 = exit handler id
* R13 = shadow vcpu (32-bit) or PACA (64-bit)
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
* HSTATE.SCRATCH0 = guest R12
* HSTATE.SCRATCH1 = guest CR
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
*/
/* Save registers */
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
PPC_STL r0, SVCPU_R0(r13)
PPC_STL r1, SVCPU_R1(r13)
PPC_STL r2, SVCPU_R2(r13)
PPC_STL r3, SVCPU_R3(r13)
PPC_STL r4, SVCPU_R4(r13)
PPC_STL r5, SVCPU_R5(r13)
PPC_STL r6, SVCPU_R6(r13)
PPC_STL r7, SVCPU_R7(r13)
PPC_STL r8, SVCPU_R8(r13)
PPC_STL r9, SVCPU_R9(r13)
PPC_STL r10, SVCPU_R10(r13)
PPC_STL r11, SVCPU_R11(r13)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
/* Restore R1/R2 so we can handle faults */
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
PPC_LL r1, HSTATE_HOST_R1(r13)
PPC_LL r2, HSTATE_HOST_R2(r13)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
/* Save guest PC and MSR */
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:18:26 -06:00
#ifdef CONFIG_PPC64
BEGIN_FTR_SECTION
KVM: PPC: Book3S: PR: Fix hsrr code When jumping back into the kernel to code that knows that it would be using HSRR registers instead of SRR registers, we need to make sure we pass it all information on where to jump to in HSRR registers. Unfortunately, we used r10 to store the information to distinguish between the HSRR and SRR case. That register got clobbered in between though, rendering the later comparison invalid. Instead, let's use cr1 to store this information. That way we don't need yet another register and everyone's happy. This fixes PR KVM on POWER7 bare metal for me. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-05-09 19:58:50 -06:00
andi. r0, r12, 0x2
cmpwi cr1, r0, 0
powerpc: Base support for exceptions using HSRR0/1 Pass the register type to the prolog, also provides alternate "HV" version of hardware interrupt (0x500) and adjust LPES accordingly We tag those interrupts by setting bit 0x2 in the trap number Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2011-04-04 22:20:31 -06:00
beq 1f
mfspr r3,SPRN_HSRR0
mfspr r4,SPRN_HSRR1
andi. r12,r12,0x3ffd
b 2f
powerpc, KVM: Split HVMODE_206 cpu feature bit into separate HV and architecture bits This replaces the single CPU_FTR_HVMODE_206 bit with two bits, one to indicate that we have a usable hypervisor mode, and another to indicate that the processor conforms to PowerISA version 2.06. We also add another bit to indicate that the processor conforms to ISA version 2.01 and set that for PPC970 and derivatives. Some PPC970 chips (specifically those in Apple machines) have a hypervisor mode in that MSR[HV] is always 1, but the hypervisor mode is not useful in the sense that there is no way to run any code in supervisor mode (HV=0 PR=0). On these processors, the LPES0 and LPES1 bits in HID4 are always 0, and we use that as a way of detecting that hypervisor mode is not useful. Where we have a feature section in assembly code around code that only applies on POWER7 in hypervisor mode, we use a construct like END_FTR_SECTION_IFSET(CPU_FTR_HVMODE | CPU_FTR_ARCH_206) The definition of END_FTR_SECTION_IFSET is such that the code will be enabled (not overwritten with nops) only if all bits in the provided mask are set. Note that the CPU feature check in __tlbie() only needs to check the ARCH_206 bit, not the HVMODE bit, because __tlbie() can only get called if we are running bare-metal, i.e. in hypervisor mode. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:26:11 -06:00
END_FTR_SECTION_IFSET(CPU_FTR_HVMODE)
powerpc, KVM: Rework KVM checks in first-level interrupt handlers Instead of branching out-of-line with the DO_KVM macro to check if we are in a KVM guest at the time of an interrupt, this moves the KVM check inline in the first-level interrupt handlers. This speeds up the non-KVM case and makes sure that none of the interrupt handlers are missing the check. Because the first-level interrupt handlers are now larger, some things had to be move out of line in exceptions-64s.S. This all necessitated some minor changes to the interrupt entry code in KVM. This also streamlines the book3s_32 KVM test. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:18:26 -06:00
#endif
powerpc: Base support for exceptions using HSRR0/1 Pass the register type to the prolog, also provides alternate "HV" version of hardware interrupt (0x500) and adjust LPES accordingly We tag those interrupts by setting bit 0x2 in the trap number Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2011-04-04 22:20:31 -06:00
1: mfsrr0 r3
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
mfsrr1 r4
powerpc: Base support for exceptions using HSRR0/1 Pass the register type to the prolog, also provides alternate "HV" version of hardware interrupt (0x500) and adjust LPES accordingly We tag those interrupts by setting bit 0x2 in the trap number Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2011-04-04 22:20:31 -06:00
2:
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
PPC_STL r3, SVCPU_PC(r13)
PPC_STL r4, SVCPU_SHADOW_SRR1(r13)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
/* Get scratch'ed off registers */
powerpc: Always use SPRN_SPRG_HSCRATCH0 when running in HV mode This uses feature sections to arrange that we always use HSPRG1 as the scratch register in the interrupt entry code rather than SPRG2 when we're running in hypervisor mode on POWER7. This will ensure that we don't trash the guest's SPRG2 when we are running KVM guests. To simplify the code, we define GET_SCRATCH0() and SET_SCRATCH0() macros like the GET_PACA/SET_PACA macros. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
2011-04-04 21:59:58 -06:00
GET_SCRATCH0(r9)
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
PPC_LL r8, HSTATE_SCRATCH0(r13)
lwz r7, HSTATE_SCRATCH1(r13)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
PPC_STL r9, SVCPU_R13(r13)
PPC_STL r8, SVCPU_R12(r13)
stw r7, SVCPU_CR(r13)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
/* Save more register state */
mfxer r5
mfdar r6
mfdsisr r7
mfctr r8
mflr r9
KVM: PPC: Book3S: correct width in XER handling In 64 bit kernels, the Fixed Point Exception Register (XER) is a 64 bit field (e.g. in kvm_regs and kvm_vcpu_arch) and in most places it is accessed as such. This patch corrects places where it is accessed as a 32 bit field by a 64 bit kernel. In some cases this is via a 32 bit load or store instruction which, depending on endianness, will cause either the lower or upper 32 bits to be missed. In another case it is cast as a u32, causing the upper 32 bits to be cleared. This patch corrects those places by extending the access methods to 64 bits. Signed-off-by: Sam Bobroff <sam.bobroff@au1.ibm.com> Reviewed-by: Laurent Vivier <lvivier@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Tested-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alexander Graf <agraf@suse.de>
2015-05-26 17:56:57 -06:00
PPC_STL r5, SVCPU_XER(r13)
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
PPC_STL r6, SVCPU_FAULT_DAR(r13)
stw r7, SVCPU_FAULT_DSISR(r13)
PPC_STL r8, SVCPU_CTR(r13)
PPC_STL r9, SVCPU_LR(r13)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
/*
* In order for us to easily get the last instruction,
* we got the #vmexit at, we exploit the fact that the
* virtual layout is still the same here, so we can just
* ld from the guest's PC address
*/
/* We only load the last instruction when it's safe */
cmpwi r12, BOOK3S_INTERRUPT_DATA_STORAGE
beq ld_last_inst
cmpwi r12, BOOK3S_INTERRUPT_PROGRAM
beq ld_last_inst
KVM: PPC: Read out syscall instruction on trap We have a few traps where we cache the instruction that cause the trap for analysis later on. Since we now need to be able to distinguish between SC 0 and SC 1 system calls and the only way to find out which is which is by looking at the instruction, we also read out the instruction causing the system call. Signed-off-by: Alexander Graf <agraf@suse.de>
2011-08-08 08:11:36 -06:00
cmpwi r12, BOOK3S_INTERRUPT_SYSCALL
beq ld_last_prev_inst
KVM: PPC: Make Alignment interrupts work again In the process of merging Book3S_32 and 64 I somehow ended up having the alignment interrupt handler take last_inst, but the fetching code not fetching it. So we ended up with stale last_inst values. Let's just enable last_inst fetching for alignment interrupts too. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-19 18:49:49 -06:00
cmpwi r12, BOOK3S_INTERRUPT_ALIGNMENT
beq- ld_last_inst
KVM: PPC: Book3S: PR: Handle EMUL_ASSIST In addition to normal "priviledged instruction" traps, we can also receive "emulation assist" traps on newer hardware that has the HV bit set. Handle that one the same way as a privileged instruction, including the instruction fetching. That way we don't execute old instructions that we happen to still leave in that field when an emul assist trap comes. This fixes -M mac99 / -M g3beige on p7 bare metal for me. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-05-09 19:54:58 -06:00
#ifdef CONFIG_PPC64
BEGIN_FTR_SECTION
cmpwi r12, BOOK3S_INTERRUPT_H_EMUL_ASSIST
beq- ld_last_inst
END_FTR_SECTION_IFSET(CPU_FTR_HVMODE)
KVM: PPC: Book3S PR: Handle Facility interrupt and FSCR POWER8 introduced a new interrupt type called "Facility unavailable interrupt" which contains its status message in a new register called FSCR. Handle these exits and try to emulate instructions for unhandled facilities. Follow-on patches enable KVM to expose specific facilities into the guest. Signed-off-by: Alexander Graf <agraf@suse.de>
2014-04-29 08:48:44 -06:00
BEGIN_FTR_SECTION
cmpwi r12, BOOK3S_INTERRUPT_FAC_UNAVAIL
beq- ld_last_inst
END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
KVM: PPC: Book3S: PR: Handle EMUL_ASSIST In addition to normal "priviledged instruction" traps, we can also receive "emulation assist" traps on newer hardware that has the HV bit set. Handle that one the same way as a privileged instruction, including the instruction fetching. That way we don't execute old instructions that we happen to still leave in that field when an emul assist trap comes. This fixes -M mac99 / -M g3beige on p7 bare metal for me. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-05-09 19:54:58 -06:00
#endif
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
b no_ld_last_inst
KVM: PPC: Read out syscall instruction on trap We have a few traps where we cache the instruction that cause the trap for analysis later on. Since we now need to be able to distinguish between SC 0 and SC 1 system calls and the only way to find out which is which is by looking at the instruction, we also read out the instruction causing the system call. Signed-off-by: Alexander Graf <agraf@suse.de>
2011-08-08 08:11:36 -06:00
ld_last_prev_inst:
addi r3, r3, -4
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
ld_last_inst:
/* Save off the guest instruction we're at */
/* In case lwz faults */
li r0, KVM_INST_FETCH_FAILED
#ifdef USE_QUICK_LAST_INST
/* Set guest mode to 'jump over instruction' so if lwz faults
* we'll just continue at the next IP. */
li r9, KVM_GUEST_MODE_SKIP
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
stb r9, HSTATE_IN_GUEST(r13)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
/* 1) enable paging for data */
mfmsr r9
ori r11, r9, MSR_DR /* Enable paging for data */
mtmsr r11
sync
/* 2) fetch the instruction */
lwz r0, 0(r3)
/* 3) disable paging again */
mtmsr r9
sync
#endif
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
stw r0, SVCPU_LAST_INST(r13)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
no_ld_last_inst:
/* Unset guest mode */
li r9, KVM_GUEST_MODE_NONE
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
stb r9, HSTATE_IN_GUEST(r13)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
/* Switch back to host MMU */
LOAD_HOST_SEGMENTS
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
#ifdef CONFIG_PPC_BOOK3S_64
lbz r5, HSTATE_RESTORE_HID5(r13)
cmpwi r5, 0
beq no_dcbz32_off
li r4, 0
mfspr r5,SPRN_HID5
rldimi r5,r4,6,56
mtspr SPRN_HID5,r5
no_dcbz32_off:
KVM: PPC: Book3S PR: Handle Facility interrupt and FSCR POWER8 introduced a new interrupt type called "Facility unavailable interrupt" which contains its status message in a new register called FSCR. Handle these exits and try to emulate instructions for unhandled facilities. Follow-on patches enable KVM to expose specific facilities into the guest. Signed-off-by: Alexander Graf <agraf@suse.de>
2014-04-29 08:48:44 -06:00
BEGIN_FTR_SECTION
/* Save guest FSCR on a FAC_UNAVAIL interrupt */
cmpwi r12, BOOK3S_INTERRUPT_FAC_UNAVAIL
bne+ no_fscr_save
mfspr r7, SPRN_FSCR
std r7, SVCPU_SHADOW_FSCR(r13)
no_fscr_save:
/* Restore host FSCR */
ld r8, HSTATE_HOST_FSCR(r13)
mtspr SPRN_FSCR, r8
END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S)
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
#endif /* CONFIG_PPC_BOOK3S_64 */
/*
* For some interrupts, we need to call the real Linux
* handler, so it can do work for us. This has to happen
* as if the interrupt arrived from the kernel though,
* so let's fake it here where most state is restored.
*
* Having set up SRR0/1 with the address where we want
* to continue with relocation on (potentially in module
* space), we either just go straight there with rfi[d],
KVM: PPC: Fix PR KVM on POWER7 bare metal When running on a system that is HV capable, some interrupts use HSRR SPRs instead of the normal SRR SPRs. These are also used in the Linux handlers to jump back to code after an interrupt got processed. Unfortunately, in our "jump back to the real host handler after we've done the context switch" code, we were only setting the SRR SPRs, rendering Linux to jump back to some invalid IP after it's processed the interrupt. This fixes random crashes on p7 opal mode with PR KVM for me. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-04-27 08:33:35 -06:00
* or we jump to an interrupt handler if there is an
* interrupt to be handled first. In the latter case,
* the rfi[d] at the end of the interrupt handler will
* get us back to where we want to continue.
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
*/
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
/* Register usage at this point:
*
* R1 = host R1
* R2 = host R2
KVM: PPC: Fix PR KVM on POWER7 bare metal When running on a system that is HV capable, some interrupts use HSRR SPRs instead of the normal SRR SPRs. These are also used in the Linux handlers to jump back to code after an interrupt got processed. Unfortunately, in our "jump back to the real host handler after we've done the context switch" code, we were only setting the SRR SPRs, rendering Linux to jump back to some invalid IP after it's processed the interrupt. This fixes random crashes on p7 opal mode with PR KVM for me. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-04-27 08:33:35 -06:00
* R10 = raw exit handler id
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
* R12 = exit handler id
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
* R13 = shadow vcpu (32-bit) or PACA (64-bit)
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
* SVCPU.* = guest *
*
*/
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
PPC_LL r6, HSTATE_HOST_MSR(r13)
KVM: PPC: Book3S PR: Avoid changing TS bits when exiting guest PR KVM host usually runs with TM enabled in its host MSR value, and with non-transactional TS value. When a guest with TM active traps into PR KVM host, the rfid at the tail of kvmppc_interrupt_pr() will try to switch TS bits from S0 (Suspended & TM disabled) to N1 (Non-transactional & TM enabled). That will leads to TM Bad Thing interrupt. This patch manually sets target TS bits unchanged to avoid this exception. Signed-off-by: Simon Guo <wei.guo.simon@gmail.com> Signed-off-by: Paul Mackerras <paulus@ozlabs.org>
2018-05-23 01:01:55 -06:00
#ifdef CONFIG_PPC_TRANSACTIONAL_MEM
/*
* We don't want to change MSR[TS] bits via rfi here.
* The actual TM handling logic will be in host with
* recovered DR/IR bits after HSTATE_VMHANDLER.
* And MSR_TM can be enabled in HOST_MSR so rfid may
* not suppress this change and can lead to exception.
* Manually set MSR to prevent TS state change here.
*/
mfmsr r7
rldicl r7, r7, 64 - MSR_TS_S_LG, 62
rldimi r6, r7, MSR_TS_S_LG, 63 - MSR_TS_T_LG
#endif
KVM: PPC: Split host-state fields out of kvmppc_book3s_shadow_vcpu There are several fields in struct kvmppc_book3s_shadow_vcpu that temporarily store bits of host state while a guest is running, rather than anything relating to the particular guest or vcpu. This splits them out into a new kvmppc_host_state structure and modifies the definitions in asm-offsets.c to suit. On 32-bit, we have a kvmppc_host_state structure inside the kvmppc_book3s_shadow_vcpu since the assembly code needs to be able to get to them both with one pointer. On 64-bit they are separate fields in the PACA. This means that on 64-bit we don't need to copy the kvmppc_host_state in and out on vcpu load/unload, and in future will mean that the book3s_hv code doesn't need a shadow_vcpu struct in the PACA at all. That does mean that we have to be careful not to rely on any values persisting in the hstate field of the paca across any point where we could block or get preempted. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-06-28 18:20:58 -06:00
PPC_LL r8, HSTATE_VMHANDLER(r13)
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
KVM: PPC: Fix PR KVM on POWER7 bare metal When running on a system that is HV capable, some interrupts use HSRR SPRs instead of the normal SRR SPRs. These are also used in the Linux handlers to jump back to code after an interrupt got processed. Unfortunately, in our "jump back to the real host handler after we've done the context switch" code, we were only setting the SRR SPRs, rendering Linux to jump back to some invalid IP after it's processed the interrupt. This fixes random crashes on p7 opal mode with PR KVM for me. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-04-27 08:33:35 -06:00
#ifdef CONFIG_PPC64
BEGIN_FTR_SECTION
KVM: PPC: Book3S: PR: Fix hsrr code When jumping back into the kernel to code that knows that it would be using HSRR registers instead of SRR registers, we need to make sure we pass it all information on where to jump to in HSRR registers. Unfortunately, we used r10 to store the information to distinguish between the HSRR and SRR case. That register got clobbered in between though, rendering the later comparison invalid. Instead, let's use cr1 to store this information. That way we don't need yet another register and everyone's happy. This fixes PR KVM on POWER7 bare metal for me. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-05-09 19:58:50 -06:00
beq cr1, 1f
KVM: PPC: Fix PR KVM on POWER7 bare metal When running on a system that is HV capable, some interrupts use HSRR SPRs instead of the normal SRR SPRs. These are also used in the Linux handlers to jump back to code after an interrupt got processed. Unfortunately, in our "jump back to the real host handler after we've done the context switch" code, we were only setting the SRR SPRs, rendering Linux to jump back to some invalid IP after it's processed the interrupt. This fixes random crashes on p7 opal mode with PR KVM for me. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-04-27 08:33:35 -06:00
mtspr SPRN_HSRR1, r6
mtspr SPRN_HSRR0, r8
END_FTR_SECTION_IFSET(CPU_FTR_HVMODE)
#endif
1: /* Restore host msr -> SRR1 */
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
mtsrr1 r6
/* Load highmem handler address */
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
mtsrr0 r8
KVM: PPC: book3s_pr: Simplify transitions between virtual and real mode This simplifies the way that the book3s_pr makes the transition to real mode when entering the guest. We now call kvmppc_entry_trampoline (renamed from kvmppc_rmcall) in the base kernel using a normal function call instead of doing an indirect call through a pointer in the vcpu. If kvm is a module, the module loader takes care of generating a trampoline as it does for other calls to functions outside the module. kvmppc_entry_trampoline then disables interrupts and jumps to kvmppc_handler_trampoline_enter in real mode using an rfi[d]. That then uses the link register as the address to return to (potentially in module space) when the guest exits. This also simplifies the way that we call the Linux interrupt handler when we exit the guest due to an external, decrementer or performance monitor interrupt. Instead of turning on the MMU, then deciding that we need to call the Linux handler and turning the MMU back off again, we now go straight to the handler at the point where we would turn the MMU on. The handler will then return to the virtual-mode code (potentially in the module). Along the way, this moves the setting and clearing of the HID5 DCBZ32 bit into real-mode interrupts-off code, and also makes sure that we clear the MSR[RI] bit before loading values into SRR0/1. The net result is that we no longer need any code addresses to be stored in vcpu->arch. Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2011-07-23 01:41:44 -06:00
/* RFI into the highmem handler, or jump to interrupt handler */
KVM: PPC: Fix PR KVM on POWER7 bare metal When running on a system that is HV capable, some interrupts use HSRR SPRs instead of the normal SRR SPRs. These are also used in the Linux handlers to jump back to code after an interrupt got processed. Unfortunately, in our "jump back to the real host handler after we've done the context switch" code, we were only setting the SRR SPRs, rendering Linux to jump back to some invalid IP after it's processed the interrupt. This fixes random crashes on p7 opal mode with PR KVM for me. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-04-27 08:33:35 -06:00
cmpwi r12, BOOK3S_INTERRUPT_EXTERNAL
beqa BOOK3S_INTERRUPT_EXTERNAL
cmpwi r12, BOOK3S_INTERRUPT_DECREMENTER
beqa BOOK3S_INTERRUPT_DECREMENTER
cmpwi r12, BOOK3S_INTERRUPT_PERFMON
beqa BOOK3S_INTERRUPT_PERFMON
KVM: PPC: Book3S PR: Cope with doorbell interrupts When the PR host is running on a POWER8 machine in POWER8 mode, it will use doorbell interrupts for IPIs. If one of them arrives while we are in the guest, we pop out of the guest with trap number 0xA00, which isn't handled by kvmppc_handle_exit_pr, leading to the following BUG_ON: [ 331.436215] exit_nr=0xa00 | pc=0x1d2c | msr=0x800000000000d032 [ 331.437522] ------------[ cut here ]------------ [ 331.438296] kernel BUG at arch/powerpc/kvm/book3s_pr.c:982! [ 331.439063] Oops: Exception in kernel mode, sig: 5 [#2] [ 331.439819] SMP NR_CPUS=1024 NUMA pSeries [ 331.440552] Modules linked in: tun nf_conntrack_netbios_ns nf_conntrack_broadcast ipt_MASQUERADE ip6t_REJECT xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw virtio_net kvm binfmt_misc ibmvscsi scsi_transport_srp scsi_tgt virtio_blk [ 331.447614] CPU: 11 PID: 1296 Comm: qemu-system-ppc Tainted: G D 3.11.7-200.2.fc19.ppc64p7 #1 [ 331.448920] task: c0000003bdc8c000 ti: c0000003bd32c000 task.ti: c0000003bd32c000 [ 331.450088] NIP: d0000000025d6b9c LR: d0000000025d6b98 CTR: c0000000004cfdd0 [ 331.451042] REGS: c0000003bd32f420 TRAP: 0700 Tainted: G D (3.11.7-200.2.fc19.ppc64p7) [ 331.452331] MSR: 800000000282b032 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI> CR: 28004824 XER: 20000000 [ 331.454616] SOFTE: 1 [ 331.455106] CFAR: c000000000848bb8 [ 331.455726] GPR00: d0000000025d6b98 c0000003bd32f6a0 d0000000026017b8 0000000000000032 GPR04: c0000000018627f8 c000000001873208 320d0a3030303030 3030303030643033 GPR08: c000000000c490a8 0000000000000000 0000000000000000 0000000000000002 GPR12: 0000000028004822 c00000000fdc6300 0000000000000000 00000100076ec310 GPR16: 000000002ae343b8 00003ffffd397398 0000000000000000 0000000000000000 GPR20: 00000100076f16f4 00000100076ebe60 0000000000000008 ffffffffffffffff GPR24: 0000000000000000 0000008001041e60 0000000000000000 0000008001040ce8 GPR28: c0000003a2d80000 0000000000000a00 0000000000000001 c0000003a2681810 [ 331.466504] NIP [d0000000025d6b9c] .kvmppc_handle_exit_pr+0x75c/0xa80 [kvm] [ 331.466999] LR [d0000000025d6b98] .kvmppc_handle_exit_pr+0x758/0xa80 [kvm] [ 331.467517] Call Trace: [ 331.467909] [c0000003bd32f6a0] [d0000000025d6b98] .kvmppc_handle_exit_pr+0x758/0xa80 [kvm] (unreliable) [ 331.468553] [c0000003bd32f750] [d0000000025d98f0] kvm_start_lightweight+0xb4/0xc4 [kvm] [ 331.469189] [c0000003bd32f920] [d0000000025d7648] .kvmppc_vcpu_run_pr+0xd8/0x270 [kvm] [ 331.469838] [c0000003bd32f9c0] [d0000000025cf748] .kvmppc_vcpu_run+0xc8/0xf0 [kvm] [ 331.470790] [c0000003bd32fa50] [d0000000025cc19c] .kvm_arch_vcpu_ioctl_run+0x5c/0x1b0 [kvm] [ 331.471401] [c0000003bd32fae0] [d0000000025c4888] .kvm_vcpu_ioctl+0x478/0x730 [kvm] [ 331.472026] [c0000003bd32fc90] [c00000000026192c] .do_vfs_ioctl+0x4dc/0x7a0 [ 331.472561] [c0000003bd32fd80] [c000000000261cc4] .SyS_ioctl+0xd4/0xf0 [ 331.473095] [c0000003bd32fe30] [c000000000009ed8] syscall_exit+0x0/0x98 [ 331.473633] Instruction dump: [ 331.473766] 4bfff9b4 2b9d0800 419efc18 60000000 60420000 3d220000 e8bf11a0 e8df12a8 [ 331.474733] 7fa4eb78 e8698660 48015165 e8410028 <0fe00000> 813f00e4 3ba00000 39290001 [ 331.475386] ---[ end trace 49fc47d994c1f8f2 ]--- [ 331.479817] This fixes the problem by making kvmppc_handle_exit_pr() recognize the interrupt. We also need to jump to the doorbell interrupt handler in book3s_segment.S to handle the interrupt on the way out of the guest. Having done that, there's nothing further to be done in kvmppc_handle_exit_pr(). Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Alexander Graf <agraf@suse.de>
2014-01-08 03:25:36 -07:00
cmpwi r12, BOOK3S_INTERRUPT_DOORBELL
beqa BOOK3S_INTERRUPT_DOORBELL
KVM: PPC: Fix PR KVM on POWER7 bare metal When running on a system that is HV capable, some interrupts use HSRR SPRs instead of the normal SRR SPRs. These are also used in the Linux handlers to jump back to code after an interrupt got processed. Unfortunately, in our "jump back to the real host handler after we've done the context switch" code, we were only setting the SRR SPRs, rendering Linux to jump back to some invalid IP after it's processed the interrupt. This fixes random crashes on p7 opal mode with PR KVM for me. Signed-off-by: Alexander Graf <agraf@suse.de>
2012-04-27 08:33:35 -06:00
powerpc/64s: Simple RFI macro conversions This commit does simple conversions of rfi/rfid to the new macros that include the expected destination context. By simple we mean cases where there is a single well known destination context, and it's simply a matter of substituting the instruction for the appropriate macro. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-01-09 09:07:15 -07:00
RFI_TO_KERNEL
KVM: PPC: Add generic segment switching code This is the code that will later be used instead of book3s_64_slb.S. It does the last step of guest entry and the first generic steps of guest exiting, once we have determined the interrupt is a KVM interrupt. It also reads the last used instruction from the guest virtual address space if necessary, to speed up that path. The new thing about this file is that it makes use of generic long load and store functions and calls a macro to fill in the actual segment switching code. That still needs to be done differently for book3s_32 and book3s_64. Signed-off-by: Alexander Graf <agraf@suse.de> Signed-off-by: Avi Kivity <avi@redhat.com>
2010-04-15 16:11:35 -06:00
kvmppc_handler_trampoline_exit_end: