Tim Chen 7e496299d4 tmpfs: make tmpfs scalable with percpu_counter for used blocks ... The current implementation of tmpfs is not scalable. We found that stat_lock is contended by multiple threads when we need to get a new page, leading to useless spinning inside this spin lock. This patch makes use of the percpu_counter library to maintain local count of used blocks to speed up getting and returning of pages. So the acquisition of stat_lock is unnecessary for getting and returning blocks, improving the performance of tmpfs on system with large number of cpus. On a 4 socket 32 core NHM-EX system, we saw improvement of 270%. The implementation below has a slight chance of race between threads causing a slight overshoot of the maximum configured blocks. However, any overshoot is small, and is bounded by the number of cpus. This happens when the number of used blocks is slightly below the maximum configured blocks when a thread checks the used block count, and another thread allocates the last block before the current thread does. This should not be a problem for tmpfs, as the overshoot is most likely to be a few blocks and bounded. If a strict limit is really desired, then configured the max blocks to be the limit less the number of cpus in system. Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com> Cc: Hugh Dickins <hughd@google.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>		2010-08-09 20:44:58 -07:00
..
backing-dev.c	Merge branch 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip	2010-08-06 09:23:07 -07:00
bootmem.c	x86,nobootmem: make alloc_bootmem_node fall back to other node when 32bit numa is used	2010-07-20 16:25:40 -07:00
bounce.c	include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h	2010-03-30 22:02:32 +09:00
compaction.c	mm: compaction: add a tunable that decides when memory should be compacted and when it should be reclaimed	2010-05-25 08:06:59 -07:00
debug-pagealloc.c	generic debug pagealloc	2009-04-01 08:59:13 -07:00
dmapool.c	dmapools: protect page_list walk in show_pools()	2009-06-30 18:56:00 -07:00
fadvise.c	readahead: introduce FMODE_RANDOM for POSIX_FADV_RANDOM	2010-03-06 11:26:25 -08:00
failslab.c	include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h	2010-03-30 22:02:32 +09:00
filemap.c	gcc-4.6: mm: fix unused but set warnings	2010-08-09 20:44:58 -07:00
filemap_xip.c	include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h	2010-03-30 22:02:32 +09:00
fremap.c	mm: clean up mm_counter	2010-03-06 11:26:23 -08:00
highmem.c	mm,kdb,kgdb: Add a debug reference for the kdb kmap usage	2010-08-05 09:22:24 -05:00
hugetlb.c	hugetlb: call mmu notifiers on hugepage cow	2010-08-09 20:44:54 -07:00
hwpoison-inject.c	HWPOISON: Don't do early filtering if filter is disabled	2009-12-16 12:20:01 +01:00
init-mm.c	mm: provide init_mm mm_context initializer	2010-08-09 20:44:54 -07:00
internal.h	HWPOISON: add an interface to switch off/on all the page filters	2009-12-16 12:19:59 +01:00
Kconfig	lmb: rename to memblock	2010-07-14 17:14:00 +10:00
Kconfig.debug	trivial: improve help text for mm debug config options	2009-09-21 15:14:57 +02:00
kmemcheck.c	kmemcheck: Fix build errors due to missing slab.h	2010-03-30 22:02:32 +09:00
kmemleak-test.c	percpu: clean up percpu variable definitions	2009-06-24 15:13:48 +09:00
kmemleak.c	include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h	2010-03-30 22:02:32 +09:00
ksm.c	mm: extend KSM refcounts to the anon_vma root	2010-08-09 20:44:55 -07:00
maccess.c	maccess,probe_kernel: Allow arch specific override probe_kernel_(read|write)	2010-01-07 11:58:36 -06:00
madvise.c	HWPOISON: Add a madvise() injector for soft page offlining	2009-12-16 12:20:00 +01:00
Makefile	lmb: rename to memblock	2010-07-14 17:14:00 +10:00
memblock.c	lmb: rename to memblock	2010-07-14 17:14:00 +10:00
memcontrol.c	memcg: fix wake up in oom wait queue	2010-06-29 15:29:30 -07:00
memory-failure.c	KVM: Fix a race condition for usage of is_hwpoison_address()	2010-08-01 10:47:11 +03:00
memory.c	gcc-4.6: mm: fix unused but set warnings	2010-08-09 20:44:58 -07:00
memory_hotplug.c	mem-hotplug: fix potential race while building zonelist for new populated zone	2010-05-25 08:07:02 -07:00
mempolicy.c	mempolicy: reduce stack size of migrate_pages()	2010-08-09 20:44:58 -07:00
mempool.c	mm: remove broken 'kzalloc' mempool	2009-09-22 07:17:35 -07:00
migrate.c	mm: extend KSM refcounts to the anon_vma root	2010-08-09 20:44:55 -07:00
mincore.c	mincore: do nested page table walks	2010-05-25 08:06:58 -07:00
mlock.c	x86, perf, bts, mm: Delete the never used BTS-ptrace code	2010-03-26 11:33:55 +01:00
mm_init.c
mmap.c	mm: always lock the root (oldest) anon_vma	2010-08-09 20:44:55 -07:00
mmu_context.c	exit: fix oops in sync_mm_rss	2010-03-24 16:31:21 -07:00
mmu_notifier.c	include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h	2010-03-30 22:02:32 +09:00
mmzone.c	[ARM] Double check memmap is actually valid with a memmap has unexpected holes V2	2009-05-18 11:22:24 +01:00
mprotect.c	include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h	2010-03-30 22:02:32 +09:00
mremap.c	include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h	2010-03-30 22:02:32 +09:00
msync.c	sanitize vfs_fsync calling conventions	2010-05-21 18:31:21 -04:00
nommu.c	nommu: allow private mappings of read-only devices	2010-05-26 08:19:23 -07:00
oom_kill.c	oom: fold __out_of_memory into out_of_memory	2010-08-09 20:44:57 -07:00
page-writeback.c	writeback: remove writeback_inodes_wbc	2010-07-06 08:54:03 +02:00
page_alloc.c	mm: rename try_set_zone_oom() to try_set_zonelist_oom()	2010-08-09 20:44:57 -07:00
page_cgroup.c	kmemleak: Annotate false positive in init_section_page_cgroup()	2010-07-19 11:54:14 +01:00
page_io.c	include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h	2010-03-30 22:02:32 +09:00
page_isolation.c
pagewalk.c	pagemap: fix pfn calculation for hugepage	2010-04-07 08:38:04 -07:00
percpu-km.c	percpu: implement kernel memory based chunk allocation	2010-05-01 08:30:50 +02:00
percpu-vm.c	percpu: move vmalloc based chunk management into percpu-vm.c	2010-05-01 08:30:50 +02:00
percpu.c	percpu: allow limited allocation before slab is online	2010-06-27 18:50:00 +02:00
percpu_up.c	percpu: don't implicitly include slab.h from percpu.h	2010-03-30 22:02:32 +09:00
prio_tree.c
quicklist.c	include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h	2010-03-30 22:02:32 +09:00
readahead.c	readahead.c: fix comment	2010-05-25 08:07:00 -07:00
rmap.c	mm: extend KSM refcounts to the anon_vma root	2010-08-09 20:44:55 -07:00
shmem.c	tmpfs: make tmpfs scalable with percpu_counter for used blocks	2010-08-09 20:44:58 -07:00
slab.c	gcc-4.6: mm: fix unused but set warnings	2010-08-09 20:44:58 -07:00
slob.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/penberg/slab-2.6	2010-08-06 11:44:08 -07:00
slub.c	Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/penberg/slab-2.6	2010-08-06 11:44:08 -07:00
sparse-vmemmap.c	include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h	2010-03-30 22:02:32 +09:00
sparse.c	sparsemem: on no vmemmap path put mem_map on node high too	2010-05-25 08:06:56 -07:00
swap.c	mm: export lru_cache_add_*() to modules	2010-05-25 15:06:06 +02:00
swap_state.c	include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h	2010-03-30 22:02:32 +09:00
swapfile.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging-2.6	2010-05-21 15:26:46 -07:00
thrash.c	mm: pass mm to grab_swap_token	2009-06-23 12:50:05 -07:00
truncate.c	fs: introduce new truncate sequence	2010-05-27 22:15:33 -04:00
util.c	mm: use memdup_user	2010-08-09 20:44:54 -07:00
vmalloc.c	mm: use ERR_CAST	2010-08-09 20:44:54 -07:00
vmscan.c	mm/vmscan.c: fix mapping use after free	2010-07-20 16:25:40 -07:00
vmstat.c	mm: use for_each_online_cpu() in vmstat	2010-08-09 20:44:57 -07:00