redonkable
/
alistair23-linux
1
0
Fork 0
Code Releases Activity

slub: Move gfpflag masking out of the hotpath 

Move the gfpflags masking into the hooks for checkers and into the slowpaths.
gfpflag masking requires access to a global variable and thus adds an
additional cacheline reference to the hotpaths.

If no hooks are active then the gfpflag masking will result in
code that the compiler can toss out.

Acked-by: David Rientjes <rientjes@google.com>
Signed-off-by: Christoph Lameter <cl@linux-foundation.org>
Signed-off-by: Pekka Enberg <penberg@kernel.org>
hifive-unleashed-5.1
Christoph Lameter 2010-08-20 12:37:17 -05:00 committed by Pekka Enberg
parent c016b0bdee
commit c1d508365e
1 changed files with 3 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
mm/slub.c
View File

@ -796,6 +796,7 @@ static void trace(struct kmem_cache *s, struct page *page, void *object,
*/
static inline int slab_pre_alloc_hook(struct kmem_cache *s, gfp_t flags)
{
flags &= gfp_allowed_mask;
lockdep_trace_alloc(flags);
might_sleep_if(flags & __GFP_WAIT);
@ -804,6 +805,7 @@ static inline int slab_pre_alloc_hook(struct kmem_cache *s, gfp_t flags)
static inline void slab_post_alloc_hook(struct kmem_cache *s, gfp_t flags, void *object)
{
flags &= gfp_allowed_mask;
kmemcheck_slab_alloc(s, flags, object, s->objsize);
kmemleak_alloc_recursive(object, s->objsize, 1, s->flags, flags);
}
@ -1677,6 +1679,7 @@ new_slab:
goto load_freelist;
}
gfpflags &= gfp_allowed_mask;
if (gfpflags & __GFP_WAIT)
local_irq_enable();
@ -1725,8 +1728,6 @@ static __always_inline void *slab_alloc(struct kmem_cache *s,
struct kmem_cache_cpu *c;
unsigned long flags;
gfpflags &= gfp_allowed_mask;
if (slab_pre_alloc_hook(s, gfpflags))
return NULL;