mm: correct page->pfmemalloc to fix deactivate_slab regression
Commit cfd19c5a9e
("mm: only set page->pfmemalloc when
ALLOC_NO_WATERMARKS was used") tried to narrow down page->pfmemalloc
setting, but it missed some places the pfmemalloc should be set.
So, in __slab_alloc, the unalignment pfmemalloc and ALLOC_NO_WATERMARKS
cause incorrect deactivate_slab() on our core2 server:
64.73% fio [kernel.kallsyms] [k] _raw_spin_lock
|
--- _raw_spin_lock
|
|---0.34%-- deactivate_slab
| __slab_alloc
| kmem_cache_alloc
| |
That causes our fio sync write performance to have a 40% regression.
Move the checking in get_page_from_freelist() which resolves this issue.
Signed-off-by: Alex Shi <alex.shi@intel.com>
Acked-by: Mel Gorman <mgorman@suse.de>
Cc: David Miller <davem@davemloft.net
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Tested-by: Eric Dumazet <eric.dumazet@gmail.com>
Tested-by: Sage Weil <sage@inktank.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This commit is contained in:
parent
5ed12f1282
commit
b121186ab1
|
@ -1928,6 +1928,17 @@ this_zone_full:
|
|||
zlc_active = 0;
|
||||
goto zonelist_scan;
|
||||
}
|
||||
|
||||
if (page)
|
||||
/*
|
||||
* page->pfmemalloc is set when ALLOC_NO_WATERMARKS was
|
||||
* necessary to allocate the page. The expectation is
|
||||
* that the caller is taking steps that will free more
|
||||
* memory. The caller should avoid the page being used
|
||||
* for !PFMEMALLOC purposes.
|
||||
*/
|
||||
page->pfmemalloc = !!(alloc_flags & ALLOC_NO_WATERMARKS);
|
||||
|
||||
return page;
|
||||
}
|
||||
|
||||
|
@ -2389,14 +2400,6 @@ rebalance:
|
|||
zonelist, high_zoneidx, nodemask,
|
||||
preferred_zone, migratetype);
|
||||
if (page) {
|
||||
/*
|
||||
* page->pfmemalloc is set when ALLOC_NO_WATERMARKS was
|
||||
* necessary to allocate the page. The expectation is
|
||||
* that the caller is taking steps that will free more
|
||||
* memory. The caller should avoid the page being used
|
||||
* for !PFMEMALLOC purposes.
|
||||
*/
|
||||
page->pfmemalloc = true;
|
||||
goto got_pg;
|
||||
}
|
||||
}
|
||||
|
@ -2569,8 +2572,6 @@ retry_cpuset:
|
|||
page = __alloc_pages_slowpath(gfp_mask, order,
|
||||
zonelist, high_zoneidx, nodemask,
|
||||
preferred_zone, migratetype);
|
||||
else
|
||||
page->pfmemalloc = false;
|
||||
|
||||
trace_mm_page_alloc(page, order, gfp_mask, migratetype);
|
||||
|
||||
|
|
Loading…
Reference in a new issue