1
0
Fork 0
alistair23-linux/arch/mips/include/asm/vdso.h

137 lines
3.7 KiB
C
Raw Normal View History

/*
MIPS: Initial implementation of a VDSO Add an initial implementation of a proper (i.e. an ELF shared library) VDSO. With this commit it does not export any symbols, it only replaces the current signal return trampoline page. A later commit will add user implementations of gettimeofday()/clock_gettime(). To support both new toolchains and old ones which don't generate ABI flags section, we define its content manually and then use a tool (genvdso) to patch up the section to have the correct name and type. genvdso also extracts symbol offsets ({,rt_}sigreturn) needed by the kernel, and generates a C file containing a "struct mips_vdso_image" containing both the VDSO data and these offsets. This C file is compiled into the kernel. On 64-bit kernels we require a different VDSO for each supported ABI, so we may build up to 3 different VDSOs. The VDSO to use is selected by the mips_abi structure. A kernel/user shared data page is created and mapped below the VDSO image. This is currently empty, but will be used by the user time function implementations which are added later. [markos.chandras@imgtec.com: - Add more comments - Move abi detection in genvdso.h since it's the get_symbol function that needs it. - Add an R6 specific way to calculate the base address of VDSO in order to avoid the branch instruction which affects performance. - Do not patch .gnu.attributes since it's not needed for dynamic linking. - Simplify Makefile a little bit. - checkpatch fixes - Restrict VDSO support for binutils < 2.25 for pre-R6 - Include atomic64.h for O32 variant on MIPS64] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: Matthew Fortune <matthew.fortune@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11337/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:54:38 -06:00
* Copyright (C) 2015 Imagination Technologies
* Author: Alex Smith <alex.smith@imgtec.com>
*
MIPS: Initial implementation of a VDSO Add an initial implementation of a proper (i.e. an ELF shared library) VDSO. With this commit it does not export any symbols, it only replaces the current signal return trampoline page. A later commit will add user implementations of gettimeofday()/clock_gettime(). To support both new toolchains and old ones which don't generate ABI flags section, we define its content manually and then use a tool (genvdso) to patch up the section to have the correct name and type. genvdso also extracts symbol offsets ({,rt_}sigreturn) needed by the kernel, and generates a C file containing a "struct mips_vdso_image" containing both the VDSO data and these offsets. This C file is compiled into the kernel. On 64-bit kernels we require a different VDSO for each supported ABI, so we may build up to 3 different VDSOs. The VDSO to use is selected by the mips_abi structure. A kernel/user shared data page is created and mapped below the VDSO image. This is currently empty, but will be used by the user time function implementations which are added later. [markos.chandras@imgtec.com: - Add more comments - Move abi detection in genvdso.h since it's the get_symbol function that needs it. - Add an R6 specific way to calculate the base address of VDSO in order to avoid the branch instruction which affects performance. - Do not patch .gnu.attributes since it's not needed for dynamic linking. - Simplify Makefile a little bit. - checkpatch fixes - Restrict VDSO support for binutils < 2.25 for pre-R6 - Include atomic64.h for O32 variant on MIPS64] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: Matthew Fortune <matthew.fortune@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11337/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:54:38 -06:00
* This program is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by the
* Free Software Foundation; either version 2 of the License, or (at your
* option) any later version.
*/
#ifndef __ASM_VDSO_H
#define __ASM_VDSO_H
MIPS: Initial implementation of a VDSO Add an initial implementation of a proper (i.e. an ELF shared library) VDSO. With this commit it does not export any symbols, it only replaces the current signal return trampoline page. A later commit will add user implementations of gettimeofday()/clock_gettime(). To support both new toolchains and old ones which don't generate ABI flags section, we define its content manually and then use a tool (genvdso) to patch up the section to have the correct name and type. genvdso also extracts symbol offsets ({,rt_}sigreturn) needed by the kernel, and generates a C file containing a "struct mips_vdso_image" containing both the VDSO data and these offsets. This C file is compiled into the kernel. On 64-bit kernels we require a different VDSO for each supported ABI, so we may build up to 3 different VDSOs. The VDSO to use is selected by the mips_abi structure. A kernel/user shared data page is created and mapped below the VDSO image. This is currently empty, but will be used by the user time function implementations which are added later. [markos.chandras@imgtec.com: - Add more comments - Move abi detection in genvdso.h since it's the get_symbol function that needs it. - Add an R6 specific way to calculate the base address of VDSO in order to avoid the branch instruction which affects performance. - Do not patch .gnu.attributes since it's not needed for dynamic linking. - Simplify Makefile a little bit. - checkpatch fixes - Restrict VDSO support for binutils < 2.25 for pre-R6 - Include atomic64.h for O32 variant on MIPS64] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: Matthew Fortune <matthew.fortune@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11337/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:54:38 -06:00
#include <linux/mm_types.h>
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime() Add user-mode implementations of gettimeofday() and clock_gettime() to the VDSO. This is currently usable with 2 clocksources: the CP0 count register, which is accessible to user-mode via RDHWR on R2 and later cores, or the MIPS Global Interrupt Controller (GIC) timer, which provides a "user-mode visible" section containing a mirror of its counter registers. This section must be mapped into user memory, which is done below the VDSO data page. When a supported clocksource is not in use, the VDSO functions will return -ENOSYS, which causes libc to fall back on the standard syscall path. When support for neither of these clocksources is compiled into the kernel at all, the VDSO still provides clock_gettime(), as the coarse realtime/monotonic clocks can still be implemented. However, gettimeofday() is not provided in this case as nothing can be done without a suitable clocksource. This causes the symbol lookup to fail in libc and it will then always use the standard syscall path. This patch includes a workaround for a bug in QEMU which results in RDHWR on the CP0 count register always returning a constant (incorrect) value. A fix for this has been submitted, and the workaround can be removed after the fix has been in stable releases for a reasonable amount of time. A simple performance test which calls gettimeofday() 1000 times in a loop and calculates the average execution time gives the following results on a Malta + I6400 (running at 20MHz): - Syscall: ~31000 ns - VDSO (GIC): ~15000 ns - VDSO (CP0): ~9500 ns [markos.chandras@imgtec.com: - Minor code re-arrangements in order for mappings to be made in the order they appear to the process' address space. - Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef - Use gic_get_usm_range so we can do the GIC mapping in the arch/mips/kernel/vdso instead of the GIC irqchip driver] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-kernel@vger.kernel.org Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11338/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:57:44 -06:00
#include <asm/barrier.h>
MIPS: Initial implementation of a VDSO Add an initial implementation of a proper (i.e. an ELF shared library) VDSO. With this commit it does not export any symbols, it only replaces the current signal return trampoline page. A later commit will add user implementations of gettimeofday()/clock_gettime(). To support both new toolchains and old ones which don't generate ABI flags section, we define its content manually and then use a tool (genvdso) to patch up the section to have the correct name and type. genvdso also extracts symbol offsets ({,rt_}sigreturn) needed by the kernel, and generates a C file containing a "struct mips_vdso_image" containing both the VDSO data and these offsets. This C file is compiled into the kernel. On 64-bit kernels we require a different VDSO for each supported ABI, so we may build up to 3 different VDSOs. The VDSO to use is selected by the mips_abi structure. A kernel/user shared data page is created and mapped below the VDSO image. This is currently empty, but will be used by the user time function implementations which are added later. [markos.chandras@imgtec.com: - Add more comments - Move abi detection in genvdso.h since it's the get_symbol function that needs it. - Add an R6 specific way to calculate the base address of VDSO in order to avoid the branch instruction which affects performance. - Do not patch .gnu.attributes since it's not needed for dynamic linking. - Simplify Makefile a little bit. - checkpatch fixes - Restrict VDSO support for binutils < 2.25 for pre-R6 - Include atomic64.h for O32 variant on MIPS64] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: Matthew Fortune <matthew.fortune@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11337/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:54:38 -06:00
/**
* struct mips_vdso_image - Details of a VDSO image.
* @data: Pointer to VDSO image data (page-aligned).
* @size: Size of the VDSO image data (page-aligned).
* @off_sigreturn: Offset of the sigreturn() trampoline.
* @off_rt_sigreturn: Offset of the rt_sigreturn() trampoline.
* @mapping: Special mapping structure.
*
* This structure contains details of a VDSO image, including the image data
* and offsets of certain symbols required by the kernel. It is generated as
* part of the VDSO build process, aside from the mapping page array, which is
* populated at runtime.
*/
struct mips_vdso_image {
void *data;
unsigned long size;
MIPS: Initial implementation of a VDSO Add an initial implementation of a proper (i.e. an ELF shared library) VDSO. With this commit it does not export any symbols, it only replaces the current signal return trampoline page. A later commit will add user implementations of gettimeofday()/clock_gettime(). To support both new toolchains and old ones which don't generate ABI flags section, we define its content manually and then use a tool (genvdso) to patch up the section to have the correct name and type. genvdso also extracts symbol offsets ({,rt_}sigreturn) needed by the kernel, and generates a C file containing a "struct mips_vdso_image" containing both the VDSO data and these offsets. This C file is compiled into the kernel. On 64-bit kernels we require a different VDSO for each supported ABI, so we may build up to 3 different VDSOs. The VDSO to use is selected by the mips_abi structure. A kernel/user shared data page is created and mapped below the VDSO image. This is currently empty, but will be used by the user time function implementations which are added later. [markos.chandras@imgtec.com: - Add more comments - Move abi detection in genvdso.h since it's the get_symbol function that needs it. - Add an R6 specific way to calculate the base address of VDSO in order to avoid the branch instruction which affects performance. - Do not patch .gnu.attributes since it's not needed for dynamic linking. - Simplify Makefile a little bit. - checkpatch fixes - Restrict VDSO support for binutils < 2.25 for pre-R6 - Include atomic64.h for O32 variant on MIPS64] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: Matthew Fortune <matthew.fortune@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11337/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:54:38 -06:00
unsigned long off_sigreturn;
unsigned long off_rt_sigreturn;
struct vm_special_mapping mapping;
};
MIPS: Initial implementation of a VDSO Add an initial implementation of a proper (i.e. an ELF shared library) VDSO. With this commit it does not export any symbols, it only replaces the current signal return trampoline page. A later commit will add user implementations of gettimeofday()/clock_gettime(). To support both new toolchains and old ones which don't generate ABI flags section, we define its content manually and then use a tool (genvdso) to patch up the section to have the correct name and type. genvdso also extracts symbol offsets ({,rt_}sigreturn) needed by the kernel, and generates a C file containing a "struct mips_vdso_image" containing both the VDSO data and these offsets. This C file is compiled into the kernel. On 64-bit kernels we require a different VDSO for each supported ABI, so we may build up to 3 different VDSOs. The VDSO to use is selected by the mips_abi structure. A kernel/user shared data page is created and mapped below the VDSO image. This is currently empty, but will be used by the user time function implementations which are added later. [markos.chandras@imgtec.com: - Add more comments - Move abi detection in genvdso.h since it's the get_symbol function that needs it. - Add an R6 specific way to calculate the base address of VDSO in order to avoid the branch instruction which affects performance. - Do not patch .gnu.attributes since it's not needed for dynamic linking. - Simplify Makefile a little bit. - checkpatch fixes - Restrict VDSO support for binutils < 2.25 for pre-R6 - Include atomic64.h for O32 variant on MIPS64] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: Matthew Fortune <matthew.fortune@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11337/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:54:38 -06:00
/*
* The following structures are auto-generated as part of the build for each
* ABI by genvdso, see arch/mips/vdso/Makefile.
*/
extern struct mips_vdso_image vdso_image;
#ifdef CONFIG_MIPS32_O32
extern struct mips_vdso_image vdso_image_o32;
#endif
#ifdef CONFIG_MIPS32_N32
extern struct mips_vdso_image vdso_image_n32;
#endif
/**
* union mips_vdso_data - Data provided by the kernel for the VDSO.
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime() Add user-mode implementations of gettimeofday() and clock_gettime() to the VDSO. This is currently usable with 2 clocksources: the CP0 count register, which is accessible to user-mode via RDHWR on R2 and later cores, or the MIPS Global Interrupt Controller (GIC) timer, which provides a "user-mode visible" section containing a mirror of its counter registers. This section must be mapped into user memory, which is done below the VDSO data page. When a supported clocksource is not in use, the VDSO functions will return -ENOSYS, which causes libc to fall back on the standard syscall path. When support for neither of these clocksources is compiled into the kernel at all, the VDSO still provides clock_gettime(), as the coarse realtime/monotonic clocks can still be implemented. However, gettimeofday() is not provided in this case as nothing can be done without a suitable clocksource. This causes the symbol lookup to fail in libc and it will then always use the standard syscall path. This patch includes a workaround for a bug in QEMU which results in RDHWR on the CP0 count register always returning a constant (incorrect) value. A fix for this has been submitted, and the workaround can be removed after the fix has been in stable releases for a reasonable amount of time. A simple performance test which calls gettimeofday() 1000 times in a loop and calculates the average execution time gives the following results on a Malta + I6400 (running at 20MHz): - Syscall: ~31000 ns - VDSO (GIC): ~15000 ns - VDSO (CP0): ~9500 ns [markos.chandras@imgtec.com: - Minor code re-arrangements in order for mappings to be made in the order they appear to the process' address space. - Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef - Use gic_get_usm_range so we can do the GIC mapping in the arch/mips/kernel/vdso instead of the GIC irqchip driver] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-kernel@vger.kernel.org Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11338/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:57:44 -06:00
* @xtime_sec: Current real time (seconds part).
* @xtime_nsec: Current real time (nanoseconds part, shifted).
* @wall_to_mono_sec: Wall-to-monotonic offset (seconds part).
* @wall_to_mono_nsec: Wall-to-monotonic offset (nanoseconds part).
* @seq_count: Counter to synchronise updates (odd = updating).
* @cs_shift: Clocksource shift value.
* @clock_mode: Clocksource to use for time functions.
* @cs_mult: Clocksource multiplier value.
* @cs_cycle_last: Clock cycle value at last update.
* @cs_mask: Clocksource mask value.
* @tz_minuteswest: Minutes west of Greenwich (from timezone).
* @tz_dsttime: Type of DST correction (from timezone).
MIPS: Initial implementation of a VDSO Add an initial implementation of a proper (i.e. an ELF shared library) VDSO. With this commit it does not export any symbols, it only replaces the current signal return trampoline page. A later commit will add user implementations of gettimeofday()/clock_gettime(). To support both new toolchains and old ones which don't generate ABI flags section, we define its content manually and then use a tool (genvdso) to patch up the section to have the correct name and type. genvdso also extracts symbol offsets ({,rt_}sigreturn) needed by the kernel, and generates a C file containing a "struct mips_vdso_image" containing both the VDSO data and these offsets. This C file is compiled into the kernel. On 64-bit kernels we require a different VDSO for each supported ABI, so we may build up to 3 different VDSOs. The VDSO to use is selected by the mips_abi structure. A kernel/user shared data page is created and mapped below the VDSO image. This is currently empty, but will be used by the user time function implementations which are added later. [markos.chandras@imgtec.com: - Add more comments - Move abi detection in genvdso.h since it's the get_symbol function that needs it. - Add an R6 specific way to calculate the base address of VDSO in order to avoid the branch instruction which affects performance. - Do not patch .gnu.attributes since it's not needed for dynamic linking. - Simplify Makefile a little bit. - checkpatch fixes - Restrict VDSO support for binutils < 2.25 for pre-R6 - Include atomic64.h for O32 variant on MIPS64] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: Matthew Fortune <matthew.fortune@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11337/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:54:38 -06:00
*
* This structure contains data needed by functions within the VDSO. It is
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime() Add user-mode implementations of gettimeofday() and clock_gettime() to the VDSO. This is currently usable with 2 clocksources: the CP0 count register, which is accessible to user-mode via RDHWR on R2 and later cores, or the MIPS Global Interrupt Controller (GIC) timer, which provides a "user-mode visible" section containing a mirror of its counter registers. This section must be mapped into user memory, which is done below the VDSO data page. When a supported clocksource is not in use, the VDSO functions will return -ENOSYS, which causes libc to fall back on the standard syscall path. When support for neither of these clocksources is compiled into the kernel at all, the VDSO still provides clock_gettime(), as the coarse realtime/monotonic clocks can still be implemented. However, gettimeofday() is not provided in this case as nothing can be done without a suitable clocksource. This causes the symbol lookup to fail in libc and it will then always use the standard syscall path. This patch includes a workaround for a bug in QEMU which results in RDHWR on the CP0 count register always returning a constant (incorrect) value. A fix for this has been submitted, and the workaround can be removed after the fix has been in stable releases for a reasonable amount of time. A simple performance test which calls gettimeofday() 1000 times in a loop and calculates the average execution time gives the following results on a Malta + I6400 (running at 20MHz): - Syscall: ~31000 ns - VDSO (GIC): ~15000 ns - VDSO (CP0): ~9500 ns [markos.chandras@imgtec.com: - Minor code re-arrangements in order for mappings to be made in the order they appear to the process' address space. - Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef - Use gic_get_usm_range so we can do the GIC mapping in the arch/mips/kernel/vdso instead of the GIC irqchip driver] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-kernel@vger.kernel.org Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11338/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:57:44 -06:00
* populated by the kernel and mapped read-only into user memory. The time
* fields are mirrors of internal data from the timekeeping infrastructure.
MIPS: Initial implementation of a VDSO Add an initial implementation of a proper (i.e. an ELF shared library) VDSO. With this commit it does not export any symbols, it only replaces the current signal return trampoline page. A later commit will add user implementations of gettimeofday()/clock_gettime(). To support both new toolchains and old ones which don't generate ABI flags section, we define its content manually and then use a tool (genvdso) to patch up the section to have the correct name and type. genvdso also extracts symbol offsets ({,rt_}sigreturn) needed by the kernel, and generates a C file containing a "struct mips_vdso_image" containing both the VDSO data and these offsets. This C file is compiled into the kernel. On 64-bit kernels we require a different VDSO for each supported ABI, so we may build up to 3 different VDSOs. The VDSO to use is selected by the mips_abi structure. A kernel/user shared data page is created and mapped below the VDSO image. This is currently empty, but will be used by the user time function implementations which are added later. [markos.chandras@imgtec.com: - Add more comments - Move abi detection in genvdso.h since it's the get_symbol function that needs it. - Add an R6 specific way to calculate the base address of VDSO in order to avoid the branch instruction which affects performance. - Do not patch .gnu.attributes since it's not needed for dynamic linking. - Simplify Makefile a little bit. - checkpatch fixes - Restrict VDSO support for binutils < 2.25 for pre-R6 - Include atomic64.h for O32 variant on MIPS64] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: Matthew Fortune <matthew.fortune@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11337/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:54:38 -06:00
*
* Note: Care should be taken when modifying as the layout must remain the same
* for both 64- and 32-bit (for 32-bit userland on 64-bit kernel).
*/
union mips_vdso_data {
struct {
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime() Add user-mode implementations of gettimeofday() and clock_gettime() to the VDSO. This is currently usable with 2 clocksources: the CP0 count register, which is accessible to user-mode via RDHWR on R2 and later cores, or the MIPS Global Interrupt Controller (GIC) timer, which provides a "user-mode visible" section containing a mirror of its counter registers. This section must be mapped into user memory, which is done below the VDSO data page. When a supported clocksource is not in use, the VDSO functions will return -ENOSYS, which causes libc to fall back on the standard syscall path. When support for neither of these clocksources is compiled into the kernel at all, the VDSO still provides clock_gettime(), as the coarse realtime/monotonic clocks can still be implemented. However, gettimeofday() is not provided in this case as nothing can be done without a suitable clocksource. This causes the symbol lookup to fail in libc and it will then always use the standard syscall path. This patch includes a workaround for a bug in QEMU which results in RDHWR on the CP0 count register always returning a constant (incorrect) value. A fix for this has been submitted, and the workaround can be removed after the fix has been in stable releases for a reasonable amount of time. A simple performance test which calls gettimeofday() 1000 times in a loop and calculates the average execution time gives the following results on a Malta + I6400 (running at 20MHz): - Syscall: ~31000 ns - VDSO (GIC): ~15000 ns - VDSO (CP0): ~9500 ns [markos.chandras@imgtec.com: - Minor code re-arrangements in order for mappings to be made in the order they appear to the process' address space. - Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef - Use gic_get_usm_range so we can do the GIC mapping in the arch/mips/kernel/vdso instead of the GIC irqchip driver] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-kernel@vger.kernel.org Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11338/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:57:44 -06:00
u64 xtime_sec;
u64 xtime_nsec;
MIPS: VDSO: Fix conversions in do_monotonic()/do_monotonic_coarse() Fix incorrect calculation in do_monotonic() and do_monotonic_coarse() function that in turn caused incorrect values returned by the vdso version of system call clock_gettime() on mips64 if its system clock ID parameter was CLOCK_MONOTONIC or CLOCK_MONOTONIC_COARSE. Consider these variables and their types on mips32 and mips64: tk->wall_to_monotonic.tv_sec s64, s64 (kernel/vdso.c) vdso_data.wall_to_mono_sec u32, u32 (kernel/vdso.c) to_mono_sec u32, u32 (vdso/gettimeofday.c) ts->tv_sec s32, s64 (vdso/gettimeofday.c) For mips64 case, u32 vdso_data.wall_to_mono_sec variable is updated from the 64-bit signed variable tk->wall_to_monotonic.tv_sec (kernel/vdso.c:76) which is a negative number holding the time passed from 1970-01-01 to the time boot started. This 64-bit signed value is currently around 47+ years, in seconds. For instance, let this value be: -1489757461 or 11111111111111111111111111111111 10100111001101000001101011101011 By updating 32-bit vdso_data.wall_to_mono_sec variable, we lose upper 32 bits (signed 1's). to_mono_sec variable is a parameter of do_monotonic() and do_monotonic_coarse() functions which holds vdso_data.wall_to_mono_sec value. Its value needs to be added (or subtracted considering it holds negative value from the tk->wall_to_monotonic.tv_sec) to the current time passed from 1970-01-01 (ts->tv_sec), which is again something like 47+ years, but increased by the time passed from the boot to the current time. ts->tv_sec is 32-bit long in case of 32-bit architecture and 64-bit long in case of 64-bit architecture. Consider the update of ts->tv_sec (vdso/gettimeofday.c:55 & 167): ts->tv_sec += to_mono_sec; mips32 case: This update will be performed correctly, since both ts->tv_sec and to_mono_sec are 32-bit long and the sign in to_mono_sec is preserved. Implicit conversion from u32 to s32 will be done correctly. mips64 case: This update will be wrong, since the implicit conversion will not be done correctly. The reason is that the conversion will be from u32 to s64. This is because to_mono_sec is 32-bit long for both mips32 and mips64 cases and s64..33 bits of converted to_mono_sec variable will be zeros. So, in order to make MIPS64 implementation work properly for MONOTONIC and MONOTONIC_COARSE clock ids on mips64, the size of wall_to_mono_sec variable in mips_vdso_data union and respective parameters in do_monotonic() and do_monotonic_coarse() functions should be changed from u32 to u64. Because of consistency, this size change from u32 and u64 is also done for wall_to_mono_nsec variable and corresponding function parameters. As far as similar situations for other architectures are concerned, let's take a look at arm. Arm has two distinct vdso_data structures for 32-bit & 64-bit cases, and arm's wall_to_mono_sec and wall_to_mono_nsec are u32 for 32-bit and u64 for 64-bit cases. On the other hand, MIPS has only one structure (mips_vdso_data), hence the need for changing the size of above mentioned parameters. Signed-off-by: Goran Ferenc <goran.ferenc@imgtec.com> Signed-off-by: Miodrag Dinic <miodrag.dinic@imgtec.com> Signed-off-by: Aleksandar Markovic <aleksandar.markovic@imgtec.com> Cc: Douglas Leung <douglas.leung@imgtec.com> Cc: James Hogan <james.hogan@imgtec.com> Cc: Paul Burton <paul.burton@imgtec.com> Cc: Petar Jovanovic <petar.jovanovic@imgtec.com> Cc: Raghu Gandham <raghu.gandham@imgtec.com> Cc: linux-mips@linux-mips.org Cc: linux-kernel@vger.kernel.org Patchwork: https://patchwork.linux-mips.org/patch/16638/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2017-06-28 09:55:28 -06:00
u64 wall_to_mono_sec;
u64 wall_to_mono_nsec;
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime() Add user-mode implementations of gettimeofday() and clock_gettime() to the VDSO. This is currently usable with 2 clocksources: the CP0 count register, which is accessible to user-mode via RDHWR on R2 and later cores, or the MIPS Global Interrupt Controller (GIC) timer, which provides a "user-mode visible" section containing a mirror of its counter registers. This section must be mapped into user memory, which is done below the VDSO data page. When a supported clocksource is not in use, the VDSO functions will return -ENOSYS, which causes libc to fall back on the standard syscall path. When support for neither of these clocksources is compiled into the kernel at all, the VDSO still provides clock_gettime(), as the coarse realtime/monotonic clocks can still be implemented. However, gettimeofday() is not provided in this case as nothing can be done without a suitable clocksource. This causes the symbol lookup to fail in libc and it will then always use the standard syscall path. This patch includes a workaround for a bug in QEMU which results in RDHWR on the CP0 count register always returning a constant (incorrect) value. A fix for this has been submitted, and the workaround can be removed after the fix has been in stable releases for a reasonable amount of time. A simple performance test which calls gettimeofday() 1000 times in a loop and calculates the average execution time gives the following results on a Malta + I6400 (running at 20MHz): - Syscall: ~31000 ns - VDSO (GIC): ~15000 ns - VDSO (CP0): ~9500 ns [markos.chandras@imgtec.com: - Minor code re-arrangements in order for mappings to be made in the order they appear to the process' address space. - Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef - Use gic_get_usm_range so we can do the GIC mapping in the arch/mips/kernel/vdso instead of the GIC irqchip driver] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-kernel@vger.kernel.org Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11338/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:57:44 -06:00
u32 seq_count;
u32 cs_shift;
u8 clock_mode;
u32 cs_mult;
u64 cs_cycle_last;
u64 cs_mask;
s32 tz_minuteswest;
s32 tz_dsttime;
MIPS: Initial implementation of a VDSO Add an initial implementation of a proper (i.e. an ELF shared library) VDSO. With this commit it does not export any symbols, it only replaces the current signal return trampoline page. A later commit will add user implementations of gettimeofday()/clock_gettime(). To support both new toolchains and old ones which don't generate ABI flags section, we define its content manually and then use a tool (genvdso) to patch up the section to have the correct name and type. genvdso also extracts symbol offsets ({,rt_}sigreturn) needed by the kernel, and generates a C file containing a "struct mips_vdso_image" containing both the VDSO data and these offsets. This C file is compiled into the kernel. On 64-bit kernels we require a different VDSO for each supported ABI, so we may build up to 3 different VDSOs. The VDSO to use is selected by the mips_abi structure. A kernel/user shared data page is created and mapped below the VDSO image. This is currently empty, but will be used by the user time function implementations which are added later. [markos.chandras@imgtec.com: - Add more comments - Move abi detection in genvdso.h since it's the get_symbol function that needs it. - Add an R6 specific way to calculate the base address of VDSO in order to avoid the branch instruction which affects performance. - Do not patch .gnu.attributes since it's not needed for dynamic linking. - Simplify Makefile a little bit. - checkpatch fixes - Restrict VDSO support for binutils < 2.25 for pre-R6 - Include atomic64.h for O32 variant on MIPS64] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: Matthew Fortune <matthew.fortune@imgtec.com> Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11337/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:54:38 -06:00
};
u8 page[PAGE_SIZE];
};
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime() Add user-mode implementations of gettimeofday() and clock_gettime() to the VDSO. This is currently usable with 2 clocksources: the CP0 count register, which is accessible to user-mode via RDHWR on R2 and later cores, or the MIPS Global Interrupt Controller (GIC) timer, which provides a "user-mode visible" section containing a mirror of its counter registers. This section must be mapped into user memory, which is done below the VDSO data page. When a supported clocksource is not in use, the VDSO functions will return -ENOSYS, which causes libc to fall back on the standard syscall path. When support for neither of these clocksources is compiled into the kernel at all, the VDSO still provides clock_gettime(), as the coarse realtime/monotonic clocks can still be implemented. However, gettimeofday() is not provided in this case as nothing can be done without a suitable clocksource. This causes the symbol lookup to fail in libc and it will then always use the standard syscall path. This patch includes a workaround for a bug in QEMU which results in RDHWR on the CP0 count register always returning a constant (incorrect) value. A fix for this has been submitted, and the workaround can be removed after the fix has been in stable releases for a reasonable amount of time. A simple performance test which calls gettimeofday() 1000 times in a loop and calculates the average execution time gives the following results on a Malta + I6400 (running at 20MHz): - Syscall: ~31000 ns - VDSO (GIC): ~15000 ns - VDSO (CP0): ~9500 ns [markos.chandras@imgtec.com: - Minor code re-arrangements in order for mappings to be made in the order they appear to the process' address space. - Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef - Use gic_get_usm_range so we can do the GIC mapping in the arch/mips/kernel/vdso instead of the GIC irqchip driver] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-kernel@vger.kernel.org Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11338/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:57:44 -06:00
static inline u32 vdso_data_read_begin(const union mips_vdso_data *data)
{
u32 seq;
while (true) {
locking/atomics: COCCINELLE/treewide: Convert trivial ACCESS_ONCE() patterns to READ_ONCE()/WRITE_ONCE() Please do not apply this to mainline directly, instead please re-run the coccinelle script shown below and apply its output. For several reasons, it is desirable to use {READ,WRITE}_ONCE() in preference to ACCESS_ONCE(), and new code is expected to use one of the former. So far, there's been no reason to change most existing uses of ACCESS_ONCE(), as these aren't harmful, and changing them results in churn. However, for some features, the read/write distinction is critical to correct operation. To distinguish these cases, separate read/write accessors must be used. This patch migrates (most) remaining ACCESS_ONCE() instances to {READ,WRITE}_ONCE(), using the following coccinelle script: ---- // Convert trivial ACCESS_ONCE() uses to equivalent READ_ONCE() and // WRITE_ONCE() // $ make coccicheck COCCI=/home/mark/once.cocci SPFLAGS="--include-headers" MODE=patch virtual patch @ depends on patch @ expression E1, E2; @@ - ACCESS_ONCE(E1) = E2 + WRITE_ONCE(E1, E2) @ depends on patch @ expression E; @@ - ACCESS_ONCE(E) + READ_ONCE(E) ---- Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Peter Zijlstra <peterz@infradead.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: davem@davemloft.net Cc: linux-arch@vger.kernel.org Cc: mpe@ellerman.id.au Cc: shuah@kernel.org Cc: snitzer@redhat.com Cc: thor.thayer@linux.intel.com Cc: tj@kernel.org Cc: viro@zeniv.linux.org.uk Cc: will.deacon@arm.com Link: http://lkml.kernel.org/r/1508792849-3115-19-git-send-email-paulmck@linux.vnet.ibm.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
2017-10-23 15:07:29 -06:00
seq = READ_ONCE(data->seq_count);
MIPS: VDSO: Add implementations of gettimeofday() and clock_gettime() Add user-mode implementations of gettimeofday() and clock_gettime() to the VDSO. This is currently usable with 2 clocksources: the CP0 count register, which is accessible to user-mode via RDHWR on R2 and later cores, or the MIPS Global Interrupt Controller (GIC) timer, which provides a "user-mode visible" section containing a mirror of its counter registers. This section must be mapped into user memory, which is done below the VDSO data page. When a supported clocksource is not in use, the VDSO functions will return -ENOSYS, which causes libc to fall back on the standard syscall path. When support for neither of these clocksources is compiled into the kernel at all, the VDSO still provides clock_gettime(), as the coarse realtime/monotonic clocks can still be implemented. However, gettimeofday() is not provided in this case as nothing can be done without a suitable clocksource. This causes the symbol lookup to fail in libc and it will then always use the standard syscall path. This patch includes a workaround for a bug in QEMU which results in RDHWR on the CP0 count register always returning a constant (incorrect) value. A fix for this has been submitted, and the workaround can be removed after the fix has been in stable releases for a reasonable amount of time. A simple performance test which calls gettimeofday() 1000 times in a loop and calculates the average execution time gives the following results on a Malta + I6400 (running at 20MHz): - Syscall: ~31000 ns - VDSO (GIC): ~15000 ns - VDSO (CP0): ~9500 ns [markos.chandras@imgtec.com: - Minor code re-arrangements in order for mappings to be made in the order they appear to the process' address space. - Move do_{monotonic, realtime} outside of the MIPS_CLOCK_VSYSCALL ifdef - Use gic_get_usm_range so we can do the GIC mapping in the arch/mips/kernel/vdso instead of the GIC irqchip driver] Signed-off-by: Alex Smith <alex.smith@imgtec.com> Signed-off-by: Markos Chandras <markos.chandras@imgtec.com> Cc: linux-kernel@vger.kernel.org Cc: linux-mips@linux-mips.org Patchwork: https://patchwork.linux-mips.org/patch/11338/ Signed-off-by: Ralf Baechle <ralf@linux-mips.org>
2015-10-21 02:57:44 -06:00
if (likely(!(seq & 1))) {
/* Paired with smp_wmb() in vdso_data_write_*(). */
smp_rmb();
return seq;
}
cpu_relax();
}
}
static inline bool vdso_data_read_retry(const union mips_vdso_data *data,
u32 start_seq)
{
/* Paired with smp_wmb() in vdso_data_write_*(). */
smp_rmb();
return unlikely(data->seq_count != start_seq);
}
static inline void vdso_data_write_begin(union mips_vdso_data *data)
{
++data->seq_count;
/* Ensure sequence update is written before other data page values. */
smp_wmb();
}
static inline void vdso_data_write_end(union mips_vdso_data *data)
{
/* Ensure data values are written before updating sequence again. */
smp_wmb();
++data->seq_count;
}
#endif /* __ASM_VDSO_H */