processor debug state PSTATE.D is unmasked in smp call
clear_os_lock for secondary cpus. So debug state is still
masked in normal kernel context. With this patch, unmask
debug state on secondary boot for the cpus in normal kernel
context. Now kgdb tests passed with multicore.
Signed-off-by: Vijaya Kumar K <Vijaya.Kumar@caviumnetworks.com>
Acked-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
So any FIQ handling is superfluous at the moment. The functions to
disable/enable FIQs is kept around if ever someone needs them in the
future, but existing calling sites including arch_cpu_idle_prepare()
may go for now.
Signed-off-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
This patch implements optimised percpu variable accesses using the
el1 r/w thread register (tpidr_el1) along the same lines as arch/arm/.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
On platforms with power management capabilities, timers that are shut
down when a CPU enters deep C-states must be emulated using an always-on
timer and a timer IPI to relay the timer IRQ to target CPUs on an SMP
system.
This patch enables the generic clockevents broadcast infrastructure for
arm64, by providing the required Kconfig entries and adding the timer
IPI infrastructure.
Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
The asynchronous aborts are generally fatal for the kernel but they can
be masked via the pstate A bit. If a system error happens while in
kernel mode, it won't be visible until returning to user space. This
patch enables this kind of abort early to help identifying the cause.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
The non-IPI interrupts are displayed only for the online cpus from
show_interrupts in kernel/irq/proc.c before calling arch_show_interrupts().
As a result, the column headers and the IPI count don't match if any
CPU is offline.
This patch fixes show_ipi_list to display IPIs for online CPUs only.
Signed-off-by: Sudeep KarkadaNagesha <sudeep.karkadanagesha@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Commit 53ae3acd (arm64: Only enable local interrupts after the CPU
is marked online) moved the enabling of the GIC after the CPUs are
marked online.
This has some interesting effect:
[...]
[<ffffffc0002eefd8>] gic_raise_softirq+0xf8/0x160
[<ffffffc000088f58>] smp_send_reschedule+0x38/0x40
[<ffffffc0000c8728>] resched_task+0x84/0xc0
[<ffffffc0000c8cdc>] check_preempt_curr+0x58/0x98
[<ffffffc0000c8d38>] ttwu_do_wakeup+0x1c/0xf4
[<ffffffc0000c8f90>] ttwu_do_activate.constprop.84+0x64/0x70
[<ffffffc0000cad30>] try_to_wake_up+0x1d4/0x2b4
[<ffffffc0000cae6c>] default_wake_function+0x10/0x18
[<ffffffc0000c5ca4>] __wake_up_common+0x60/0xa0
[<ffffffc0000c7784>] complete+0x48/0x64
[<ffffffc000088bec>] secondary_start_kernel+0xe8/0x110
[...]
Here, we end-up calling gic_raise_softirq without having initialized
the interrupt controller for this CPU. While this goes unnoticed
with GICv2 (the distributor is always accessible), it explodes with
GICv3.
The fix is to move the call to notify_cpu_starting before we set
the secondary CPU online.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
This patch adds the basic infrastructure necessary to support
CPU_HOTPLUG on arm64, based on the arm implementation. Actual hotplug
support will depend on an implementation's cpu_operations (e.g. PSCI).
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
With the advent of CPU_HOTPLUG, the enable-method property for CPU0 may
tells us something useful (i.e. how to hotplug it back on), so we must
read it along with all the enable-method for all the other CPUs. Even
on UP the enable-method may tell us useful information (e.g. if a core
has some mechanism that might be usable for cpuidle), so we should
always read it.
This patch factors out the reading of the enable method, and ensures
that CPU0's enable method is read regardless of whether the kernel is
built with SMP support.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
The arm64 kernel has an internal holding pen, which is necessary for
some systems where we can't bring CPUs online individually and must hold
multiple CPUs in a safe area until the kernel is able to handle them.
The current SMP infrastructure for arm64 is closely coupled to this
holding pen, and alternative boot methods must launch CPUs into the pen,
where they sit before they are launched into the kernel proper.
With PSCI (and possibly other future boot methods), we can bring CPUs
online individually, and need not perform the secondary_holding_pen
dance. Instead, this patch factors the holding pen management code out
to the spin-table boot method code, as it is the only boot method
requiring the pen.
A new entry point for secondaries, secondary_entry is added for other
boot methods to use, which bypasses the holding pen and its associated
overhead when bringing CPUs online. The smp.pen.text section is also
removed, as the pen can live in head.text without problem.
The cpu_operations structure is extended with two new functions,
cpu_boot and cpu_postboot, for bringing a cpu into the kernel and
performing any post-boot cleanup required by a bootmethod (e.g.
resetting the secondary_holding_pen_release to INVALID_HWID).
Documentation is added for cpu_operations.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
For hotplug support, we're going to want a place to store operations
that do more than bring CPUs online, and it makes sense to group these
with our current smp_enable_ops. For cpuidle support, we'll want to
group additional functions, and we may want them even for UP kernels.
This patch renames smp_enable_ops to the more general cpu_operations,
and pulls the definitions out of smp code such that they can be used in
UP kernels. While we're at it, fix up instances of the cpu parameter to
be an unsigned int, drop the init markings and rename the *_cpu
functions to cpu_* to reduce future churn when cpu_operations is
extended.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
We always use a timer-backed delay loop for arm64, so don't bother
reporting a bogomips value which appears to confuse some people.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
- Fixes (user cache maintenance fault handling, !COMPAT compilation, CPU
online and interrupt hanlding).
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iQIcBAABAgAGBQJR6WGUAAoJEGvWsS0AyF7xaMQP/3WSpcEUQ+k8wCbjzkbpnQJ3
Z0Ufz0XeBUgmaZNwYFjnpVTm/R04F1gcsoA5qE//6iMkbcpbM2sWVN/uQvY3l+fQ
haJmCWZ7PIXxm3vbKrcSBiJ+WKpvUkzlL+Q1upmQWFCxkP6noejCsvazM5lPgt1q
1Vr5/9q87TOfxG8Udki3pPqRazd4YwQJx6JCZ46P+mlqQk9gxDoQ7RRXy9Viv+SV
6Jq5Lt8Qj7iXmq8DDTYdF7DHE7rnIhdCUhJu7UN1G/O7U4s9nlUbv55fHMY76ZAn
BddgNtIsMsvfIxlYZ6f0r2ccrgPHaDTLVW/q0VkwdP8jD2EQWAK7gI21OgQm0ebl
OL+t29zdx9nidpI+cCTlwejh8i7vRsoxgYki5qfnYR3SHL5HhQSHrUTZvEFr+u4I
ceXnDmTZ46HmPSCC6/5cFiXxsw1zbBxSB7rNFvXmF2Jr7F3TvAxCWvrIfmrmYdrC
bw4UMBB15SaJud3maqVGhj6aVo4bEand4g++Dk0ytXbGq+5Ke5CktmPOmzNLVu9p
R8tHDyp9szFP0eqqF1/XTZx2rsHvop7D1QUpQVgFC+mRWI3gjtFPtShW1GgbQQ+P
t7gvHJQ2SF1BRTSf6KH/cqXDgHzpg3VuP2moM7YbR0/qvRf7Eh/M3Hrg/s6xCeKr
ywmW7/+McIz3kDsk1gUM
=M4kj
-----END PGP SIGNATURE-----
Merge tag 'arm64-stable' of git://git.kernel.org/pub/scm/linux/kernel/git/cmarinas/linux-aarch64
Pull arm64 fixes from Catalin Marinas:
- Post -rc1 update to the common reboot infrastructure.
- Fixes (user cache maintenance fault handling, !COMPAT compilation,
CPU online and interrupt hanlding).
* tag 'arm64-stable' of git://git.kernel.org/pub/scm/linux/kernel/git/cmarinas/linux-aarch64:
arm64: use common reboot infrastructure
arm64: mm: don't treat user cache maintenance faults as writes
arm64: add '#ifdef CONFIG_COMPAT' for aarch32_break_handler()
arm64: Only enable local interrupts after the CPU is marked online
There is a slight chance that (timer) interrupts are triggered before a
secondary CPU has been marked online with implications on softirq thread
affinity.
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Reported-by: Kirill Tkhai <tkhai@yandex.ru>
The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications. For example, the fix in
commit 5e427ec2d0 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.
After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out. Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.
Note that some harmless section mismatch warnings may result, since
notify_cpu_starting() and cpu_up() are arch independent (kernel/cpu.c)
are flagged as __cpuinit -- so if we remove the __cpuinit from
arch specific callers, we will also get section mismatch warnings.
As an intermediate step, we intend to turn the linux/init.h cpuinit
content into no-ops as early as possible, since that will get rid
of these warnings. In any case, they are temporary and harmless.
This removes all the arch/arm64 uses of the __cpuinit macros from
all C files. Currently arm64 does not have any __CPUINIT used in
assembly files.
[1] https://lkml.org/lkml/2013/5/20/589
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
- Versatile Express SoC (model) support - DT files and Kconfig entries
(there are no arch/arm64/mach-* directories). The bulk of the code has
already been moved to drivers/ as part of the ARM SoC clean-up.
- Basic multi-cluster support (CPU logical map initialised from the DT).
- Simple earlyprintk support for UART 8250/16550 and FastModel console
output.
- Optimised kernel library bitops and string functions.
- Automatic initialisation of the irqchip and clocks via DT.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iQIcBAABAgAGBQJRf+a/AAoJEGvWsS0AyF7x4cAP/199we5Q3IvXnOiyoiJSwk/v
FJ8i+B/cICd8LiSRhpUkDYDWboBWq+LkG+SxmhPGjvWp0GXt6RFgSuT7okQrQy4l
a3UpRNLAc7HRamehreWqdIXuD1JwkrUwRC5hxVjQBrI0Q2JeQgEbghtFwiK2qRAg
yLo9PPRfrBHq3jEJXhaiHyb/oQm9jNUbgJ5KSes6gjgwBtnsATiaSyLe5FbZqzIV
aYPKblFMUjR38TQE7uAxWtA8hPDG558dkiBJfs1GHRtkGt7odrEvAdscxw0mjDSd
rvP+XW8URVDYUhZVSaDKB8z4l436jCSAg3GYCWf/CC+sQDdxBBa1Lp5ADg/xQLrh
L+oICtDeX3v3LGhGs0x4RG4hdvyCATyHI+P/bIKA9sWUWbKTTCqEQahG0puyiun+
7nB89xGKZDIcy7yc+2UivHXIsTIRXETLmEFDKciit3f7ua4maoIvJ8oOHkJaPJRQ
imWRRg1BVMvq3L1IPCwuoIirasXzcrAkONw5Gw5ZHKVps0+xarNd4T/RWp1rmjXt
R4ZdgWNMyC3PHoP7CwG9h7Ia4CUYuaxqnmWQRALrHahr1Cxb/rnU98xadjkkBTwX
Xh1b6fdz4yKXgVBiirfPPRA3xjSYJkwEBqFwO5IAPhYulUw10U56+K0I77ixYQ8G
IDtjzKiHu/P05Nwncgn6
=Uvte
-----END PGP SIGNATURE-----
Merge tag 'arm64-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/cmarinas/linux-aarch64
Pull arm64 update from Catalin Marinas:
"Main features:
- Versatile Express SoC (model) support - DT files and Kconfig
entries (there are no arch/arm64/mach-* directories). The bulk of
the code has already been moved to drivers/ as part of the ARM SoC
clean-up.
- Basic multi-cluster support (CPU logical map initialised from the
DT)
- Simple earlyprintk support for UART 8250/16550 and FastModel
console output
- Optimised kernel library bitops and string functions.
- Automatic initialisation of the irqchip and clocks via DT"
* tag 'arm64-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/cmarinas/linux-aarch64: (26 commits)
arm64: Use acquire/release semantics instead of explicit DMB
arm64: klib: bitops: fix unpredictable stxr usage
arm64: vexpress: Enable ARMv8 RTSM model (SoC) support
arm64: vexpress: Add dts files for the ARMv8 RTSM models
arm64: Survive invalid cpu enable-methods
arm64: mm: Correct show_pte behaviour
arm64: Fix compat types affecting struct compat_stat
arm64: Execute DSB during thread switching for TLB/cache maintenance
arm64: compiling issue, need add include/asm/vga.h file
arm64: smp: honour #address-size when parsing CPU reg property
arm64: Define cmpxchg64 and cmpxchg64_local for outside use
arm64: Define readq and writeq for driver module using
arm64: Fix task tracing
arm64: add explicit symbols to ESR_EL1 decoding
arm64: Use irqchip_init() for interrupt controller initialisation
arm64: psci: Use the MPIDR values from cpu_logical_map for cpu ids.
arm64: klib: Optimised atomic bitops
arm64: klib: Optimised string functions
arm64: klib: Optimised memory functions
arm64: head: match all affinity levels in the pen of the secondaries
...
Currently, if you pass the kernel a dtb where a cpu node has an
unsupported enable-method property (e.g. "not-psci"), it'll explode
horribly, as it iterates over the enable_ops array incorrectly. It
increments the pointer *at* the current element, rather than
incrementing the pointer *to* the current element. As the first two
elements pointed to structures that were contiguous in memory, this
happened to be equivalent. However the third element is NULL, so when
the list is exhausted, smp_get_enable_ops generates the wrong pointer,
and dereferences an arbitrary portion of memory, which currently happens
to contain zero.
This patch fixes this by indirecting the pointer one level, so we
iterate over the array elements correctly, avoiding the below panic:
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
For systems where the top 32-bits of the MPIDR are all zero, we should
allow the device-tree to specify an #address-size of 0x1 for the CPU reg
property and then zero extend the value there.
Without this patch, kvmtool breaks with the recent mpidr parsing code
introduced in 4c7aa00213 ("arm64: kernel: initialise cpu_logical_map
from the DT").
Acked-by: Javi Merino <javi.merino@arm.com>
Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
When booting the kernel, the cpu logical id map must be initialised
using device tree data passed by FW or through an embedded blob.
This patch parses the reg property in device tree "cpu" nodes,
retrieves the corresponding CPUs hardware identifiers (MPIDR) and
initialises the cpu logical map accordingly.
The device tree HW identifiers are considered valid if all CPU nodes
contain a "reg" property, there are no duplicate "reg" entries and the
DT defines a CPU node whose "reg" property defines affinity levels
that matches those of the boot CPU.
The primary CPU is assigned cpu logical number 0 to keep the current
convention valid.
Based on a0ae024050 (ARM: kernel: add
device tree init map function).
Signed-off-by: Javi Merino <javi.merino@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Change the prototype of write_pen_release() accordingly and clarify
that's holding the hardware id of the secondary that's going to boot.
This is in preparation of getting HWIDs parsed from the DT.
Signed-off-by: Javi Merino <javi.merino@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Wire the PSCI implementation into the SMP secondary startup
code.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
In order to introduce PSCI support, let the SMP code handle
multiple enabling methods. This also allow CPUs to be booted
using different methods (though this feels a bit weird...).
In the process, move the spin-table code to its own file.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Commit 149c24151e ("ARM: SMP: use a timing out completion for cpu
hotplug") modified arm's CPU up path to use completions. It seems that
we only got half of this patch for arm64, so add the missing call to
complete.
Reported-by: Jon Brawn <jon.brawn@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
This patch adds SMP initialisation and spinlocks implementation for
AArch64. The spinlock support uses the new load-acquire/store-release
instructions to avoid explicit barriers. The architecture also specifies
that an event is automatically generated when clearing the exclusive
monitor state to wake up processors in WFE, so there is no need for an
explicit DSB/SEV instruction sequence. The SEVL instruction is used to
set the exclusive monitor locally as there is no conditional WFE and a
branch is more expensive.
For the SMP booting protocol, see Documentation/arm64/booting.txt.
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Tony Lindgren <tony@atomide.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
Acked-by: Olof Johansson <olof@lixom.net>
Acked-by: Santosh Shilimkar <santosh.shilimkar@ti.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Vijaya Kumar K