2019-05-27 00:55:01 -06:00
|
|
|
/* SPDX-License-Identifier: GPL-2.0-or-later */
|
2007-08-21 21:46:44 -06:00
|
|
|
#ifndef _ASM_POWERPC_EXCEPTION_H
|
|
|
|
|
#define _ASM_POWERPC_EXCEPTION_H
|
|
|
|
|
/*
|
|
|
|
|
* Extracted from head_64.S
|
|
|
|
|
*
|
|
|
|
|
* PowerPC version
|
|
|
|
|
* Copyright (C) 1995-1996 Gary Thomas (gdt@linuxppc.org)
|
|
|
|
|
*
|
|
|
|
|
* Rewritten by Cort Dougan (cort@cs.nmt.edu) for PReP
|
|
|
|
|
* Copyright (C) 1996 Cort Dougan <cort@cs.nmt.edu>
|
|
|
|
|
* Adapted for Power Macintosh by Paul Mackerras.
|
|
|
|
|
* Low-level exception handlers and MMU support
|
|
|
|
|
* rewritten by Paul Mackerras.
|
|
|
|
|
* Copyright (C) 1996 Paul Mackerras.
|
|
|
|
|
*
|
|
|
|
|
* Adapted for 64bit PowerPC by Dave Engebretsen, Peter Bergner, and
|
|
|
|
|
* Mike Corrigan {engebret|bergner|mikejc}@us.ibm.com
|
|
|
|
|
*
|
|
|
|
|
* This file contains the low-level support and setup for the
|
|
|
|
|
* PowerPC-64 platform, including trap and interrupt dispatch.
|
|
|
|
|
*/
|
|
|
|
|
/*
|
|
|
|
|
* The following macros define the code that appears as
|
|
|
|
|
* the prologue to each of the exception handlers. They
|
|
|
|
|
* are split into two parts to allow a single kernel binary
|
|
|
|
|
* to be used for pSeries and iSeries.
|
|
|
|
|
*
|
|
|
|
|
* We make as much of the exception code common between native
|
|
|
|
|
* exception handlers (including pSeries LPAR) and iSeries LPAR
|
|
|
|
|
* implementations as possible.
|
|
|
|
|
*/
|
2018-07-05 10:25:01 -06:00
|
|
|
#include <asm/feature-fixups.h>
|
2007-08-21 21:46:44 -06:00
|
|
|
|
2019-06-22 07:15:35 -06:00
|
|
|
/* PACA save area size in u64 units (exgen, exmc, etc) */
|
2017-05-21 07:15:50 -06:00
|
|
|
#if defined(CONFIG_RELOCATABLE)
|
2019-06-22 07:15:35 -06:00
|
|
|
#define EX_SIZE 10
|
2017-05-21 07:15:50 -06:00
|
|
|
#else
|
2019-06-22 07:15:35 -06:00
|
|
|
#define EX_SIZE 9
|
2017-05-21 07:15:50 -06:00
|
|
|
#endif
|
2017-05-21 07:15:48 -06:00
|
|
|
|
2017-09-28 22:26:53 -06:00
|
|
|
/*
|
|
|
|
|
* maximum recursive depth of MCE exceptions
|
|
|
|
|
*/
|
|
|
|
|
#define MAX_MCE_DEPTH 4
|
|
|
|
|
|
2019-06-22 07:15:11 -06:00
|
|
|
#ifdef __ASSEMBLY__
|
2017-05-21 07:15:49 -06:00
|
|
|
|
2018-05-21 17:00:00 -06:00
|
|
|
#define STF_ENTRY_BARRIER_SLOT \
|
|
|
|
|
STF_ENTRY_BARRIER_FIXUP_SECTION; \
|
|
|
|
|
nop; \
|
|
|
|
|
nop; \
|
|
|
|
|
nop
|
|
|
|
|
|
|
|
|
|
#define STF_EXIT_BARRIER_SLOT \
|
|
|
|
|
STF_EXIT_BARRIER_FIXUP_SECTION; \
|
|
|
|
|
nop; \
|
|
|
|
|
nop; \
|
|
|
|
|
nop; \
|
|
|
|
|
nop; \
|
|
|
|
|
nop; \
|
|
|
|
|
nop
|
|
|
|
|
|
2020-11-19 16:35:13 -07:00
|
|
|
#define ENTRY_FLUSH_SLOT \
|
|
|
|
|
ENTRY_FLUSH_FIXUP_SECTION; \
|
|
|
|
|
nop; \
|
|
|
|
|
nop; \
|
|
|
|
|
nop;
|
|
|
|
|
|
2018-05-21 17:00:00 -06:00
|
|
|
/*
|
|
|
|
|
* r10 must be free to use, r13 must be paca
|
|
|
|
|
*/
|
|
|
|
|
#define INTERRUPT_TO_KERNEL \
|
2020-11-19 16:35:13 -07:00
|
|
|
STF_ENTRY_BARRIER_SLOT; \
|
|
|
|
|
ENTRY_FLUSH_SLOT
|
2018-05-21 17:00:00 -06:00
|
|
|
|
powerpc/64s: Add support for RFI flush of L1-D cache
On some CPUs we can prevent the Meltdown vulnerability by flushing the
L1-D cache on exit from kernel to user mode, and from hypervisor to
guest.
This is known to be the case on at least Power7, Power8 and Power9. At
this time we do not know the status of the vulnerability on other CPUs
such as the 970 (Apple G5), pasemi CPUs (AmigaOne X1000) or Freescale
CPUs. As more information comes to light we can enable this, or other
mechanisms on those CPUs.
The vulnerability occurs when the load of an architecturally
inaccessible memory region (eg. userspace load of kernel memory) is
speculatively executed to the point where its result can influence the
address of a subsequent speculatively executed load.
In order for that to happen, the first load must hit in the L1,
because before the load is sent to the L2 the permission check is
performed. Therefore if no kernel addresses hit in the L1 the
vulnerability can not occur. We can ensure that is the case by
flushing the L1 whenever we return to userspace. Similarly for
hypervisor vs guest.
In order to flush the L1-D cache on exit, we add a section of nops at
each (h)rfi location that returns to a lower privileged context, and
patch that with some sequence. Newer firmwares are able to advertise
to us that there is a special nop instruction that flushes the L1-D.
If we do not see that advertised, we fall back to doing a displacement
flush in software.
For guest kernels we support migration between some CPU versions, and
different CPUs may use different flush instructions. So that we are
prepared to migrate to a machine with a different flush instruction
activated, we may have to patch more than one flush instruction at
boot if the hypervisor tells us to.
In the end this patch is mostly the work of Nicholas Piggin and
Michael Ellerman. However a cast of thousands contributed to analysis
of the issue, earlier versions of the patch, back ports testing etc.
Many thanks to all of them.
Tested-by: Jon Masters <jcm@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-01-09 09:07:15 -07:00
|
|
|
/*
|
|
|
|
|
* Macros for annotating the expected destination of (h)rfid
|
|
|
|
|
*
|
|
|
|
|
* The nop instructions allow us to insert one or more instructions to flush the
|
|
|
|
|
* L1-D cache when returning to userspace or a guest.
|
|
|
|
|
*/
|
|
|
|
|
#define RFI_FLUSH_SLOT \
|
|
|
|
|
RFI_FLUSH_FIXUP_SECTION; \
|
|
|
|
|
nop; \
|
|
|
|
|
nop; \
|
|
|
|
|
nop
|
2018-01-09 09:07:15 -07:00
|
|
|
|
|
|
|
|
#define RFI_TO_KERNEL \
|
|
|
|
|
rfid
|
|
|
|
|
|
|
|
|
|
#define RFI_TO_USER \
|
2018-05-21 17:00:00 -06:00
|
|
|
STF_EXIT_BARRIER_SLOT; \
|
powerpc/64s: Add support for RFI flush of L1-D cache
On some CPUs we can prevent the Meltdown vulnerability by flushing the
L1-D cache on exit from kernel to user mode, and from hypervisor to
guest.
This is known to be the case on at least Power7, Power8 and Power9. At
this time we do not know the status of the vulnerability on other CPUs
such as the 970 (Apple G5), pasemi CPUs (AmigaOne X1000) or Freescale
CPUs. As more information comes to light we can enable this, or other
mechanisms on those CPUs.
The vulnerability occurs when the load of an architecturally
inaccessible memory region (eg. userspace load of kernel memory) is
speculatively executed to the point where its result can influence the
address of a subsequent speculatively executed load.
In order for that to happen, the first load must hit in the L1,
because before the load is sent to the L2 the permission check is
performed. Therefore if no kernel addresses hit in the L1 the
vulnerability can not occur. We can ensure that is the case by
flushing the L1 whenever we return to userspace. Similarly for
hypervisor vs guest.
In order to flush the L1-D cache on exit, we add a section of nops at
each (h)rfi location that returns to a lower privileged context, and
patch that with some sequence. Newer firmwares are able to advertise
to us that there is a special nop instruction that flushes the L1-D.
If we do not see that advertised, we fall back to doing a displacement
flush in software.
For guest kernels we support migration between some CPU versions, and
different CPUs may use different flush instructions. So that we are
prepared to migrate to a machine with a different flush instruction
activated, we may have to patch more than one flush instruction at
boot if the hypervisor tells us to.
In the end this patch is mostly the work of Nicholas Piggin and
Michael Ellerman. However a cast of thousands contributed to analysis
of the issue, earlier versions of the patch, back ports testing etc.
Many thanks to all of them.
Tested-by: Jon Masters <jcm@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-01-09 09:07:15 -07:00
|
|
|
RFI_FLUSH_SLOT; \
|
|
|
|
|
rfid; \
|
|
|
|
|
b rfi_flush_fallback
|
2018-01-09 09:07:15 -07:00
|
|
|
|
|
|
|
|
#define RFI_TO_USER_OR_KERNEL \
|
2018-05-21 17:00:00 -06:00
|
|
|
STF_EXIT_BARRIER_SLOT; \
|
powerpc/64s: Add support for RFI flush of L1-D cache
On some CPUs we can prevent the Meltdown vulnerability by flushing the
L1-D cache on exit from kernel to user mode, and from hypervisor to
guest.
This is known to be the case on at least Power7, Power8 and Power9. At
this time we do not know the status of the vulnerability on other CPUs
such as the 970 (Apple G5), pasemi CPUs (AmigaOne X1000) or Freescale
CPUs. As more information comes to light we can enable this, or other
mechanisms on those CPUs.
The vulnerability occurs when the load of an architecturally
inaccessible memory region (eg. userspace load of kernel memory) is
speculatively executed to the point where its result can influence the
address of a subsequent speculatively executed load.
In order for that to happen, the first load must hit in the L1,
because before the load is sent to the L2 the permission check is
performed. Therefore if no kernel addresses hit in the L1 the
vulnerability can not occur. We can ensure that is the case by
flushing the L1 whenever we return to userspace. Similarly for
hypervisor vs guest.
In order to flush the L1-D cache on exit, we add a section of nops at
each (h)rfi location that returns to a lower privileged context, and
patch that with some sequence. Newer firmwares are able to advertise
to us that there is a special nop instruction that flushes the L1-D.
If we do not see that advertised, we fall back to doing a displacement
flush in software.
For guest kernels we support migration between some CPU versions, and
different CPUs may use different flush instructions. So that we are
prepared to migrate to a machine with a different flush instruction
activated, we may have to patch more than one flush instruction at
boot if the hypervisor tells us to.
In the end this patch is mostly the work of Nicholas Piggin and
Michael Ellerman. However a cast of thousands contributed to analysis
of the issue, earlier versions of the patch, back ports testing etc.
Many thanks to all of them.
Tested-by: Jon Masters <jcm@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-01-09 09:07:15 -07:00
|
|
|
RFI_FLUSH_SLOT; \
|
|
|
|
|
rfid; \
|
|
|
|
|
b rfi_flush_fallback
|
2018-01-09 09:07:15 -07:00
|
|
|
|
|
|
|
|
#define RFI_TO_GUEST \
|
2018-05-21 17:00:00 -06:00
|
|
|
STF_EXIT_BARRIER_SLOT; \
|
powerpc/64s: Add support for RFI flush of L1-D cache
On some CPUs we can prevent the Meltdown vulnerability by flushing the
L1-D cache on exit from kernel to user mode, and from hypervisor to
guest.
This is known to be the case on at least Power7, Power8 and Power9. At
this time we do not know the status of the vulnerability on other CPUs
such as the 970 (Apple G5), pasemi CPUs (AmigaOne X1000) or Freescale
CPUs. As more information comes to light we can enable this, or other
mechanisms on those CPUs.
The vulnerability occurs when the load of an architecturally
inaccessible memory region (eg. userspace load of kernel memory) is
speculatively executed to the point where its result can influence the
address of a subsequent speculatively executed load.
In order for that to happen, the first load must hit in the L1,
because before the load is sent to the L2 the permission check is
performed. Therefore if no kernel addresses hit in the L1 the
vulnerability can not occur. We can ensure that is the case by
flushing the L1 whenever we return to userspace. Similarly for
hypervisor vs guest.
In order to flush the L1-D cache on exit, we add a section of nops at
each (h)rfi location that returns to a lower privileged context, and
patch that with some sequence. Newer firmwares are able to advertise
to us that there is a special nop instruction that flushes the L1-D.
If we do not see that advertised, we fall back to doing a displacement
flush in software.
For guest kernels we support migration between some CPU versions, and
different CPUs may use different flush instructions. So that we are
prepared to migrate to a machine with a different flush instruction
activated, we may have to patch more than one flush instruction at
boot if the hypervisor tells us to.
In the end this patch is mostly the work of Nicholas Piggin and
Michael Ellerman. However a cast of thousands contributed to analysis
of the issue, earlier versions of the patch, back ports testing etc.
Many thanks to all of them.
Tested-by: Jon Masters <jcm@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-01-09 09:07:15 -07:00
|
|
|
RFI_FLUSH_SLOT; \
|
|
|
|
|
rfid; \
|
|
|
|
|
b rfi_flush_fallback
|
2018-01-09 09:07:15 -07:00
|
|
|
|
|
|
|
|
#define HRFI_TO_KERNEL \
|
|
|
|
|
hrfid
|
|
|
|
|
|
|
|
|
|
#define HRFI_TO_USER \
|
2018-05-21 17:00:00 -06:00
|
|
|
STF_EXIT_BARRIER_SLOT; \
|
powerpc/64s: Add support for RFI flush of L1-D cache
On some CPUs we can prevent the Meltdown vulnerability by flushing the
L1-D cache on exit from kernel to user mode, and from hypervisor to
guest.
This is known to be the case on at least Power7, Power8 and Power9. At
this time we do not know the status of the vulnerability on other CPUs
such as the 970 (Apple G5), pasemi CPUs (AmigaOne X1000) or Freescale
CPUs. As more information comes to light we can enable this, or other
mechanisms on those CPUs.
The vulnerability occurs when the load of an architecturally
inaccessible memory region (eg. userspace load of kernel memory) is
speculatively executed to the point where its result can influence the
address of a subsequent speculatively executed load.
In order for that to happen, the first load must hit in the L1,
because before the load is sent to the L2 the permission check is
performed. Therefore if no kernel addresses hit in the L1 the
vulnerability can not occur. We can ensure that is the case by
flushing the L1 whenever we return to userspace. Similarly for
hypervisor vs guest.
In order to flush the L1-D cache on exit, we add a section of nops at
each (h)rfi location that returns to a lower privileged context, and
patch that with some sequence. Newer firmwares are able to advertise
to us that there is a special nop instruction that flushes the L1-D.
If we do not see that advertised, we fall back to doing a displacement
flush in software.
For guest kernels we support migration between some CPU versions, and
different CPUs may use different flush instructions. So that we are
prepared to migrate to a machine with a different flush instruction
activated, we may have to patch more than one flush instruction at
boot if the hypervisor tells us to.
In the end this patch is mostly the work of Nicholas Piggin and
Michael Ellerman. However a cast of thousands contributed to analysis
of the issue, earlier versions of the patch, back ports testing etc.
Many thanks to all of them.
Tested-by: Jon Masters <jcm@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-01-09 09:07:15 -07:00
|
|
|
RFI_FLUSH_SLOT; \
|
|
|
|
|
hrfid; \
|
|
|
|
|
b hrfi_flush_fallback
|
2018-01-09 09:07:15 -07:00
|
|
|
|
|
|
|
|
#define HRFI_TO_USER_OR_KERNEL \
|
2018-05-21 17:00:00 -06:00
|
|
|
STF_EXIT_BARRIER_SLOT; \
|
powerpc/64s: Add support for RFI flush of L1-D cache
On some CPUs we can prevent the Meltdown vulnerability by flushing the
L1-D cache on exit from kernel to user mode, and from hypervisor to
guest.
This is known to be the case on at least Power7, Power8 and Power9. At
this time we do not know the status of the vulnerability on other CPUs
such as the 970 (Apple G5), pasemi CPUs (AmigaOne X1000) or Freescale
CPUs. As more information comes to light we can enable this, or other
mechanisms on those CPUs.
The vulnerability occurs when the load of an architecturally
inaccessible memory region (eg. userspace load of kernel memory) is
speculatively executed to the point where its result can influence the
address of a subsequent speculatively executed load.
In order for that to happen, the first load must hit in the L1,
because before the load is sent to the L2 the permission check is
performed. Therefore if no kernel addresses hit in the L1 the
vulnerability can not occur. We can ensure that is the case by
flushing the L1 whenever we return to userspace. Similarly for
hypervisor vs guest.
In order to flush the L1-D cache on exit, we add a section of nops at
each (h)rfi location that returns to a lower privileged context, and
patch that with some sequence. Newer firmwares are able to advertise
to us that there is a special nop instruction that flushes the L1-D.
If we do not see that advertised, we fall back to doing a displacement
flush in software.
For guest kernels we support migration between some CPU versions, and
different CPUs may use different flush instructions. So that we are
prepared to migrate to a machine with a different flush instruction
activated, we may have to patch more than one flush instruction at
boot if the hypervisor tells us to.
In the end this patch is mostly the work of Nicholas Piggin and
Michael Ellerman. However a cast of thousands contributed to analysis
of the issue, earlier versions of the patch, back ports testing etc.
Many thanks to all of them.
Tested-by: Jon Masters <jcm@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-01-09 09:07:15 -07:00
|
|
|
RFI_FLUSH_SLOT; \
|
|
|
|
|
hrfid; \
|
|
|
|
|
b hrfi_flush_fallback
|
2018-01-09 09:07:15 -07:00
|
|
|
|
|
|
|
|
#define HRFI_TO_GUEST \
|
2018-05-21 17:00:00 -06:00
|
|
|
STF_EXIT_BARRIER_SLOT; \
|
powerpc/64s: Add support for RFI flush of L1-D cache
On some CPUs we can prevent the Meltdown vulnerability by flushing the
L1-D cache on exit from kernel to user mode, and from hypervisor to
guest.
This is known to be the case on at least Power7, Power8 and Power9. At
this time we do not know the status of the vulnerability on other CPUs
such as the 970 (Apple G5), pasemi CPUs (AmigaOne X1000) or Freescale
CPUs. As more information comes to light we can enable this, or other
mechanisms on those CPUs.
The vulnerability occurs when the load of an architecturally
inaccessible memory region (eg. userspace load of kernel memory) is
speculatively executed to the point where its result can influence the
address of a subsequent speculatively executed load.
In order for that to happen, the first load must hit in the L1,
because before the load is sent to the L2 the permission check is
performed. Therefore if no kernel addresses hit in the L1 the
vulnerability can not occur. We can ensure that is the case by
flushing the L1 whenever we return to userspace. Similarly for
hypervisor vs guest.
In order to flush the L1-D cache on exit, we add a section of nops at
each (h)rfi location that returns to a lower privileged context, and
patch that with some sequence. Newer firmwares are able to advertise
to us that there is a special nop instruction that flushes the L1-D.
If we do not see that advertised, we fall back to doing a displacement
flush in software.
For guest kernels we support migration between some CPU versions, and
different CPUs may use different flush instructions. So that we are
prepared to migrate to a machine with a different flush instruction
activated, we may have to patch more than one flush instruction at
boot if the hypervisor tells us to.
In the end this patch is mostly the work of Nicholas Piggin and
Michael Ellerman. However a cast of thousands contributed to analysis
of the issue, earlier versions of the patch, back ports testing etc.
Many thanks to all of them.
Tested-by: Jon Masters <jcm@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-01-09 09:07:15 -07:00
|
|
|
RFI_FLUSH_SLOT; \
|
|
|
|
|
hrfid; \
|
|
|
|
|
b hrfi_flush_fallback
|
2018-01-09 09:07:15 -07:00
|
|
|
|
|
|
|
|
#define HRFI_TO_UNKNOWN \
|
2018-05-21 17:00:00 -06:00
|
|
|
STF_EXIT_BARRIER_SLOT; \
|
powerpc/64s: Add support for RFI flush of L1-D cache
On some CPUs we can prevent the Meltdown vulnerability by flushing the
L1-D cache on exit from kernel to user mode, and from hypervisor to
guest.
This is known to be the case on at least Power7, Power8 and Power9. At
this time we do not know the status of the vulnerability on other CPUs
such as the 970 (Apple G5), pasemi CPUs (AmigaOne X1000) or Freescale
CPUs. As more information comes to light we can enable this, or other
mechanisms on those CPUs.
The vulnerability occurs when the load of an architecturally
inaccessible memory region (eg. userspace load of kernel memory) is
speculatively executed to the point where its result can influence the
address of a subsequent speculatively executed load.
In order for that to happen, the first load must hit in the L1,
because before the load is sent to the L2 the permission check is
performed. Therefore if no kernel addresses hit in the L1 the
vulnerability can not occur. We can ensure that is the case by
flushing the L1 whenever we return to userspace. Similarly for
hypervisor vs guest.
In order to flush the L1-D cache on exit, we add a section of nops at
each (h)rfi location that returns to a lower privileged context, and
patch that with some sequence. Newer firmwares are able to advertise
to us that there is a special nop instruction that flushes the L1-D.
If we do not see that advertised, we fall back to doing a displacement
flush in software.
For guest kernels we support migration between some CPU versions, and
different CPUs may use different flush instructions. So that we are
prepared to migrate to a machine with a different flush instruction
activated, we may have to patch more than one flush instruction at
boot if the hypervisor tells us to.
In the end this patch is mostly the work of Nicholas Piggin and
Michael Ellerman. However a cast of thousands contributed to analysis
of the issue, earlier versions of the patch, back ports testing etc.
Many thanks to all of them.
Tested-by: Jon Masters <jcm@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2018-01-09 09:07:15 -07:00
|
|
|
RFI_FLUSH_SLOT; \
|
|
|
|
|
hrfid; \
|
|
|
|
|
b hrfi_flush_fallback
|
2018-01-09 09:07:15 -07:00
|
|
|
|
2020-11-19 16:35:14 -07:00
|
|
|
#else /* __ASSEMBLY__ */
|
|
|
|
|
/* Prototype for function defined in exceptions-64s.S */
|
|
|
|
|
void do_uaccess_flush(void);
|
2019-06-22 07:15:11 -06:00
|
|
|
#endif /* __ASSEMBLY__ */
|
2007-08-21 21:46:44 -06:00
|
|
|
|
|
|
|
|
#endif /* _ASM_POWERPC_EXCEPTION_H */
|