1
0
Fork 0
alistair23-linux/drivers/acpi
Joerg Roedel 763802b53a x86/mm: split vmalloc_sync_all()
Commit 3f8fd02b1b ("mm/vmalloc: Sync unmappings in
__purge_vmap_area_lazy()") introduced a call to vmalloc_sync_all() in
the vunmap() code-path.  While this change was necessary to maintain
correctness on x86-32-pae kernels, it also adds additional cycles for
architectures that don't need it.

Specifically on x86-64 with CONFIG_VMAP_STACK=y some people reported
severe performance regressions in micro-benchmarks because it now also
calls the x86-64 implementation of vmalloc_sync_all() on vunmap().  But
the vmalloc_sync_all() implementation on x86-64 is only needed for newly
created mappings.

To avoid the unnecessary work on x86-64 and to gain the performance
back, split up vmalloc_sync_all() into two functions:

	* vmalloc_sync_mappings(), and
	* vmalloc_sync_unmappings()

Most call-sites to vmalloc_sync_all() only care about new mappings being
synchronized.  The only exception is the new call-site added in the
above mentioned commit.

Shile Zhang directed us to a report of an 80% regression in reaim
throughput.

Fixes: 3f8fd02b1b ("mm/vmalloc: Sync unmappings in __purge_vmap_area_lazy()")
Reported-by: kernel test robot <oliver.sang@intel.com>
Reported-by: Shile Zhang <shile.zhang@linux.alibaba.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Tested-by: Borislav Petkov <bp@suse.de>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>	[GHES]
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: <stable@vger.kernel.org>
Link: http://lkml.kernel.org/r/20191009124418.8286-1-joro@8bytes.org
Link: https://lists.01.org/hyperkitty/list/lkp@lists.01.org/thread/4D3JPPHBNOSPFK2KEPC6KGKS6J25AIDB/
Link: http://lkml.kernel.org/r/20191113095530.228959-1-shile.zhang@linux.alibaba.com
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-03-21 18:56:06 -07:00
..
acpica ACPI: PM: s2idle: Check fixed wakeup events in acpi_s2idle_wake() 2020-02-21 10:01:25 -08:00
apei x86/mm: split vmalloc_sync_all() 2020-03-21 18:56:06 -07:00
arm64 IOMMU Updates for Linux v5.6 2020-02-05 17:49:54 +00:00
dptf ACPI: DPTF: Add Tiger Lake ACPI device IDs 2019-12-19 22:51:55 +01:00
nfit libnvdimm: Move nvdimm_bus_attribute_group to device_type 2019-11-19 09:52:12 -08:00
numa ACPI: HMAT: use %u instead of %d to print u32 values 2019-11-12 10:05:57 +01:00
pmic ACPI / PMIC: Add Cherry Trail Crystal Cove PMIC OpRegion driver 2019-10-25 11:43:08 +02:00
x86 x86/intel: Aggregate big core mobile naming 2019-08-28 11:29:31 +02:00
Kconfig ACPI: processor: Make ACPI_PROCESSOR_CSTATE depend on ACPI_PROCESSOR 2019-12-27 11:01:45 +01:00
Makefile Merge branch 'acpi-mm' 2019-11-26 10:31:02 +01:00
ac.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157 2019-05-30 11:26:37 -07:00
acpi_adxl.c ACPI/ADXL: Add address translation interface using an ACPI DSM 2018-10-16 10:03:00 +02:00
acpi_amba.c Merge 5.2-rc6 into char-misc-next 2019-06-23 09:23:33 +02:00
acpi_apd.c ACPI / APD: Add clock frequency for Hisilicon Hip08-Lite I2C controller 2020-02-03 12:03:18 +01:00
acpi_cmos_rtc.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
acpi_configfs.c ACPICA: make acpi_load_table() return table index 2019-10-28 22:12:46 +01:00
acpi_dbg.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
acpi_extlog.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 437 2019-06-05 17:37:17 +02:00
acpi_ipmi.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157 2019-05-30 11:26:37 -07:00
acpi_lpat.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 174 2019-05-30 11:26:41 -07:00
acpi_lpit.c remove ioremap_nocache and devm_ioremap_nocache 2020-01-06 09:45:59 +01:00
acpi_lpss.c ACPI / LPSS: Rename pwm_backlight pwm-lookup to pwm_soc_backlight 2019-12-17 11:24:47 +01:00
acpi_memhotplug.c mm/memory_hotplug: rename walk_memory_range() and pass start+size instead of pfns 2019-07-18 17:08:06 -07:00
acpi_pad.c ACPI, x86: Add Zhaoxin processors support for NONSTOP TSC 2019-06-22 11:45:57 +02:00
acpi_platform.c ACPI: platform: Unregister stale platform devices 2019-10-17 23:48:39 +02:00
acpi_pnp.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
acpi_processor.c ACPI: processor: Export acpi_processor_evaluate_cst() 2019-12-27 11:02:07 +01:00
acpi_tad.c ACPI: TAD: Add low-level support for real time capability 2018-10-18 09:11:53 +02:00
acpi_video.c ACPI: video: fix typo in comment 2019-12-16 11:19:58 +01:00
acpi_watchdog.c ACPI: watchdog: Fix gas->access_width usage 2020-02-13 23:51:30 +01:00
battery.c proc: convert everything to "struct proc_ops" 2020-02-04 03:05:26 +00:00
bgrt.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
blacklist.c ACPI: blacklist: fix clang warning for unused DMI table 2019-07-11 22:45:00 +02:00
bus.c ACPI: bus: Fix NULL pointer check in acpi_bus_get_private_data() 2019-11-29 11:42:10 +01:00
button.c ACPI: button: Add DMI quirk for Razer Blade Stealth 13 late 2019 lid switch 2020-01-07 11:51:41 +01:00
cm_sbs.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157 2019-05-30 11:26:37 -07:00
container.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157 2019-05-30 11:26:37 -07:00
cppc_acpi.c ACPI: CPPC: Set pcc_data[pcc_ss_id] to NULL in acpi_cppc_processor_exit() 2019-10-18 10:36:37 +02:00
custom_method.c Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security 2019-09-28 08:14:15 -07:00
debugfs.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00
device_pm.c ACPI: fan: Add Tiger Lake ACPI device ID 2019-12-19 22:51:55 +01:00
device_sysfs.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 174 2019-05-30 11:26:41 -07:00
dock.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157 2019-05-30 11:26:37 -07:00
ec.c ACPI: EC: Fix flushing of pending work 2020-02-11 10:07:43 +01:00
ec_sys.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 482 2019-06-19 17:09:52 +02:00
event.c ACPI: event: replace strcpy() by strscpy() 2019-04-23 10:54:26 +02:00
evged.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 284 2019-06-05 17:36:37 +02:00
fan.c Merge branches 'acpi-battery', 'acpi-video', 'acpi-fan' and 'acpi-drivers' 2020-01-27 10:57:09 +01:00
glue.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 428 2019-06-05 17:37:16 +02:00
hed.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 174 2019-05-30 11:26:41 -07:00
internal.h ACPI: EC: add support for hardware-reduced systems 2019-10-28 16:00:09 +01:00
ioapic.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
irq.c Merge branch 'irq-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-07-08 11:01:13 -07:00
nvs.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 428 2019-06-05 17:37:16 +02:00
osi.c ACPI: OSI: Shoot duplicate word 2019-11-13 11:12:08 +01:00
osl.c ACPI: OSL: only free map once in osl.c 2019-11-29 10:31:12 +01:00
pci_irq.c ACPI / PCI: fix acpi_pci_irq_enable() memory leak 2019-09-03 09:41:25 +02:00
pci_link.c ACPI/PCI: Remove surplus parentheses from a return statement 2019-08-21 00:05:20 +02:00
pci_mcfg.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 459 2019-06-19 17:09:09 +02:00
pci_root.c PCI: Move ASPM declarations to linux/pci.h 2019-08-28 08:28:39 -05:00
pci_slot.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 491 2019-06-19 17:09:52 +02:00
power.c ACPI / PM: Introduce concept of a _PR0 dependent device 2019-06-27 12:31:57 +02:00
pptt.c ACPI: PPTT: Consistently use unsigned int as parameter type 2020-01-07 11:46:36 +01:00
proc.c proc: convert everything to "struct proc_ops" 2020-02-04 03:05:26 +00:00
processor_core.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00
processor_driver.c cpufreq: Use per-policy frequency QoS 2019-10-21 02:05:21 +02:00
processor_idle.c ACPI: processor: Export acpi_processor_evaluate_cst() 2019-12-27 11:02:07 +01:00
processor_pdc.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
processor_perflib.c ACPI: processor: Add QoS requests for all CPUs 2019-10-25 11:33:48 +02:00
processor_thermal.c ACPI: processor: Add QoS requests for all CPUs 2019-10-25 11:33:48 +02:00
processor_throttling.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157 2019-05-30 11:26:37 -07:00
property.c device property: Add a function to obtain a node's prefix 2019-10-11 11:26:55 +02:00
reboot.c ACPI: Allow CONFIG_PCI to be unset for reboot 2018-12-20 10:19:49 +01:00
resource.c acpi: Use pr_warn instead of pr_warning 2019-10-18 15:00:19 +02:00
sbs.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 157 2019-05-30 11:26:37 -07:00
sbshc.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 372 2019-06-05 17:37:10 +02:00
sbshc.h ACPI: SBS: remove unused const variable 'SMBUS_PEC' 2019-08-23 12:55:09 +02:00
scan.c treewide: remove redundant IS_ERR() before error code check 2020-02-04 03:05:27 +00:00
sleep.c ACPI: PM: s2idle: Check fixed wakeup events in acpi_s2idle_wake() 2020-02-21 10:01:25 -08:00
sleep.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
spcr.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
sysfs.c ACPI: sysfs: Change ACPI_MASKABLE_GPE_MAX to 0x100 2019-11-19 09:40:16 +01:00
tables.c Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security 2019-09-28 08:14:15 -07:00
thermal.c ACPI: thermal: switch to use <linux/units.h> helpers 2020-01-31 10:30:39 -08:00
utils.c ACPI / utils: Introduce acpi_dev_hid_uid_match() helper 2019-10-15 12:06:23 +02:00
video_detect.c ACPI: video: Do not export a non working backlight interface on MSI MS-7721 boards 2019-12-19 23:01:52 +01:00
wakeup.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00