Johannes Weiner 14b468791f mm: workingset: move shadow entry tracking to radix tree exceptional tracking ... Currently, we track the shadow entries in the page cache in the upper bits of the radix_tree_node->count, behind the back of the radix tree implementation. Because the radix tree code has no awareness of them, we rely on random subtleties throughout the implementation (such as the node->count != 1 check in the shrinking code, which is meant to exclude multi-entry nodes but also happens to skip nodes with only one shadow entry, as that's accounted in the upper bits). This is error prone and has, in fact, caused the bug fixed in d3798ae8c6 ("mm: filemap: don't plant shadow entries without radix tree node"). To remove these subtleties, this patch moves shadow entry tracking from the upper bits of node->count to the existing counter for exceptional entries. node->count goes back to being a simple counter of valid entries in the tree node and can be shrunk to a single byte. This vastly simplifies the page cache code. All accounting happens natively inside the radix tree implementation, and maintaining the LRU linkage of shadow nodes is consolidated into a single function in the workingset code that is called for leaf nodes affected by a change in the page cache tree. This also removes the last user of the __radix_delete_node() return value. Eliminate it. Link: http://lkml.kernel.org/r/20161117193211.GE23430@cmpxchg.org Signed-off-by: Johannes Weiner <hannes@cmpxchg.org> Reviewed-by: Jan Kara <jack@suse.cz> Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Cc: Hugh Dickins <hughd@google.com> Cc: Matthew Wilcox <mawilcox@linuxonhyperv.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>		2016-12-12 18:55:08 -08:00
..
kasan	kasan: support use-after-scope detection	2016-11-30 16:32:52 -08:00
Kconfig	of/fdt: mark hotpluggable memory	2016-12-12 18:55:07 -08:00
Kconfig.debug	PM / Hibernate: allow hibernation with PAGE_POISONING_ZERO	2016-09-13 02:35:27 +02:00
Makefile	Disable the __builtin_return_address() warning globally after all	2016-10-12 10:23:41 -07:00
backing-dev.c	block: fix bdi vs gendisk lifetime mismatch	2016-08-04 14:19:16 -06:00
balloon_compaction.c	mm: balloon: use general non-lru movable page feature	2016-07-26 16:19:19 -07:00
bootmem.c	mm: kmemleak: avoid using __va() on addresses that don't have a lowmem mapping	2016-10-11 15:06:33 -07:00
cleancache.c	cleancache: constify cleancache_ops structure	2016-01-27 09:09:57 -05:00
cma.c	mm/cma.c: check the max limit for cma allocation	2016-11-11 08:12:37 -08:00
cma.h	mm: cma: mark cma_bitmap_maxno() inline in header	2015-08-14 15:56:32 -07:00
cma_debug.c	mm/cma_debug: correct size input to bitmap function	2015-07-17 16:39:54 -07:00
compaction.c	mm, compaction: fix NR_ISOLATED_* stats for pfn based migration	2016-12-12 18:55:07 -08:00
debug.c	mm: clarify why we avoid page_mapcount() for slab pages in dump_page()	2016-10-07 18:46:29 -07:00
debug_page_ref.c	mm/page_ref: add tracepoint to track down page reference manipulation	2016-03-17 15:09:34 -07:00
dmapool.c	mm: convert printk(KERN_<LEVEL> to pr_<level>	2016-03-17 15:09:34 -07:00
early_ioremap.c	mm/early_ioremap: use offset_in_page macro	2015-11-05 19:34:48 -08:00
fadvise.c	mm/fadvise.c: do not discard partial pages with POSIX_FADV_DONTNEED	2016-06-09 14:23:11 -07:00
failslab.c	mm: fault-inject take over bootstrap kmem_cache check	2016-03-15 16:55:16 -07:00
filemap.c	mm: workingset: move shadow entry tracking to radix tree exceptional tracking	2016-12-12 18:55:08 -08:00
frame_vector.c	mm: replace get_vaddr_frames() write/force parameters with gup_flags	2016-10-19 08:11:24 -07:00
frontswap.c	mm, frontswap: convert frontswap_enabled to static key	2016-07-26 16:19:19 -07:00
gup.c	mm: fix up get_user_pages* comments	2016-12-12 18:55:07 -08:00
highmem.c	mm/highmem: make nr_free_highpages() handles all highmem zones by itself	2016-05-19 19:12:14 -07:00
huge_memory.c	mm: add tlb_remove_check_page_size_change to track page size change	2016-12-12 18:55:07 -08:00
hugetlb.c	mm: add tlb_remove_check_page_size_change to track page size change	2016-12-12 18:55:07 -08:00
hugetlb_cgroup.c	mm, hugetlb_cgroup: round limit_in_bytes down to hugepage size	2016-05-20 17:58:30 -07:00
hwpoison-inject.c	hwpoison: use page_cgroup_ino for filtering by memcg	2015-09-10 13:29:01 -07:00
init-mm.c	…
internal.h	mm, compaction: make full priority ignore pageblock suitability	2016-10-07 18:46:29 -07:00
interval_tree.c	mm: replace vma->sharead.linear with vma->shared	2015-02-10 14:30:31 -08:00
khugepaged.c	lib: radix-tree: check accounting of existing slot replacement users	2016-12-12 18:55:08 -08:00
kmemcheck.c	mm: convert printk(KERN_<LEVEL> to pr_<level>	2016-03-17 15:09:34 -07:00
kmemleak-test.c	mm: convert printk(KERN_<LEVEL> to pr_<level>	2016-03-17 15:09:34 -07:00
kmemleak.c	kmemleak: fix reference to Documentation	2016-12-12 18:55:07 -08:00
ksm.c	mm,ksm: add __GFP_HIGH to the allocation in alloc_stable_node()	2016-10-07 18:46:29 -07:00
list_lru.c	mm/list_lru.c: avoid error-path NULL pointer deref	2016-10-27 18:43:42 -07:00
maccess.c	x86: remove more uaccess_32.h complexity	2016-05-22 17:21:27 -07:00
madvise.c	mm: add tlb_remove_check_page_size_change to track page size change	2016-12-12 18:55:07 -08:00
memblock.c	mm: kmemleak: avoid using __va() on addresses that don't have a lowmem mapping	2016-10-11 15:06:33 -07:00
memcontrol.c	mm: memcontrol: use special workqueue for creating per-memcg caches	2016-12-12 18:55:06 -08:00
memory-failure.c	mm: hwpoison: fix thp split handling in memory_failure()	2016-11-11 08:12:37 -08:00
memory.c	mm: remove the page size change check in tlb_remove_page	2016-12-12 18:55:07 -08:00
memory_hotplug.c	mm: remove x86-only restriction of movable_node	2016-12-12 18:55:07 -08:00
mempolicy.c	mm/mempolicy.c: forbid static or relative flags for local NUMA mode	2016-12-12 18:55:07 -08:00
mempool.c	Revert "mm, mempool: only set __GFP_NOMEMALLOC if there are free elements"	2016-07-28 16:07:41 -07:00
memtest.c	memtest: remove unused header files	2015-09-08 15:35:28 -07:00
migrate.c	lib: radix-tree: check accounting of existing slot replacement users	2016-12-12 18:55:08 -08:00
mincore.c	mm, swap: use offset of swap entry as key of swap cache	2016-10-07 18:46:28 -07:00
mlock.c	thp: fix corner case of munlock() of PTE-mapped THPs	2016-11-30 16:32:52 -08:00
mm_init.c	mm: convert printk(KERN_<LEVEL> to pr_<level>	2016-03-17 15:09:34 -07:00
mmap.c	mm: vma_merge: correct false positive from __vma_unlink->validate_mm_rb	2016-10-07 18:46:29 -07:00
mmu_context.c	mm/mmu_context, sched/core: Fix mmu_context.h assumption	2016-04-28 11:44:19 +02:00
mmu_notifier.c	fix Christoph's email addresses	2016-03-17 15:09:34 -07:00
mmzone.c	mm, page_alloc: inline the fast path of the zonelist iterator	2016-05-19 19:12:14 -07:00
mprotect.c	mm/pkeys: generate pkey system call code only if ARCH_HAS_PKEYS is selected	2016-12-12 18:55:07 -08:00
mremap.c	mremap: move_ptes: check pte dirty after its removal	2016-11-29 08:20:24 -08:00
msync.c	mm/msync: use offset_in_page macro	2015-11-05 19:34:48 -08:00
nobootmem.c	mm: kmemleak: avoid using __va() on addresses that don't have a lowmem mapping	2016-10-11 15:06:33 -07:00
nommu.c	mm: unexport __get_user_pages()	2016-10-24 19:13:20 -07:00
oom_kill.c	oom: print nodemask in the oom report	2016-10-07 18:46:29 -07:00
page-writeback.c	mm: don't use radix tree writeback tags for pages in swap cache	2016-10-07 18:46:28 -07:00
page_alloc.c	mm: make unreserve highatomic functions reliable	2016-12-12 18:55:07 -08:00
page_counter.c	mm: page_counter: let page_counter_try_charge() return bool	2015-11-05 19:34:48 -08:00
page_ext.c	mm/page_ext: support extra space allocation by page_ext user	2016-10-07 18:46:27 -07:00
page_idle.c	mm, vmscan: move lru_lock to the node	2016-07-28 16:07:41 -07:00
page_io.c	mm/page_io.c: replace some BUG_ON()s with VM_BUG_ON_PAGE()	2016-10-07 18:46:29 -07:00
page_isolation.c	mm/page_isolation: fix typo: "paes" -> "pages"	2016-10-07 18:46:29 -07:00
page_owner.c	mm/page_owner: don't define fields on struct page_ext by hard-coding	2016-10-07 18:46:27 -07:00
page_poison.c	mm: check the return value of lookup_page_ext for all call sites	2016-06-03 15:06:22 -07:00
pagewalk.c	thp: rename split_huge_page_pmd() to split_huge_pmd()	2016-01-15 17:56:32 -08:00
percpu-km.c	mm: percpu: use pr_fmt to prefix output	2016-03-17 15:09:34 -07:00
percpu-vm.c	percpu: move region iterations out of pcpu_[de]populate_chunk()	2014-09-02 14:46:02 -04:00
percpu.c	mm/percpu.c: fix potential memory leakage for pcpu_embed_first_chunk()	2016-10-05 11:52:55 -04:00
pgtable-generic.c	mm/thp/migration: switch from flush_tlb_range to flush_pmd_tlb_range	2016-03-17 15:09:34 -07:00
process_vm_access.c	mm: remove write/force parameters from __get_user_pages_unlocked()	2016-10-18 14:13:37 -07:00
quicklist.c	fix Christoph's email addresses	2016-03-17 15:09:34 -07:00
readahead.c	mm: don't cap request size based on read-ahead setting	2016-12-12 18:55:08 -08:00
rmap.c	rmap: fix compound check logic in page_remove_file_rmap	2016-08-10 16:40:56 -07:00
shmem.c	lib: radix-tree: update callback for changing leaf nodes	2016-12-12 18:55:08 -08:00
slab.c	mm, slab: maintain total slab count instead of active count	2016-12-12 18:55:07 -08:00
slab.h	mm, slab: maintain total slab count instead of active count	2016-12-12 18:55:07 -08:00
slab_common.c	mm/slab_common.c: check kmem_create_cache flags are common	2016-12-12 18:55:06 -08:00
slob.c	slub: move synchronize_sched out of slab_mutex on shrink	2016-12-12 18:55:06 -08:00
slub.c	slub: avoid false-postive warning	2016-12-12 18:55:06 -08:00
sparse-vmemmap.c	treewide: replace obsolete _refok by __ref	2016-08-02 17:31:41 -04:00
sparse.c	treewide: replace obsolete _refok by __ref	2016-08-02 17:31:41 -04:00
swap.c	thp: reduce usage of huge zero page's atomic counter	2016-10-07 18:46:28 -07:00
swap_cgroup.c	mm: convert printk(KERN_<LEVEL> to pr_<level>	2016-03-17 15:09:34 -07:00
swap_state.c	mm, swap: use offset of swap entry as key of swap cache	2016-10-07 18:46:28 -07:00
swapfile.c	swapfile: fix memory corruption via malformed swapfile	2016-11-11 08:12:37 -08:00
truncate.c	mm: workingset: move shadow entry tracking to radix tree exceptional tracking	2016-12-12 18:55:08 -08:00
usercopy.c	mm: usercopy: Check for module addresses	2016-09-20 16:07:39 -07:00
userfaultfd.c	mm, fs: get rid of PAGE_CACHE_* and page_cache_{get,release} macros	2016-04-04 10:41:08 -07:00
util.c	Merge branch 'mm-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip	2016-10-22 09:39:10 -07:00
vmacache.c	mm: unrig VMA cache hit ratio	2016-10-07 18:46:27 -07:00
vmalloc.c	mm/vmalloc.c: simplify /proc/vmallocinfo implementation	2016-12-12 18:55:07 -08:00
vmpressure.c	mm/vmpressure.c: fix subtree pressure detection	2016-02-03 08:28:43 -08:00
vmscan.c	mm/vmscan.c: set correct defer count for shrinker	2016-12-12 18:55:07 -08:00
vmstat.c	seq/proc: modify seq_put_decimal_[u]ll to take a const char *, not char	2016-10-07 18:46:30 -07:00
workingset.c	mm: workingset: move shadow entry tracking to radix tree exceptional tracking	2016-12-12 18:55:08 -08:00
z3fold.c	mm/z3fold.c: avoid modifying HEADLESS page and minor cleanup	2016-06-03 16:02:55 -07:00
zbud.c	mm/zbud.c: use list_last_entry() instead of list_tail_entry()	2016-01-15 11:40:52 -08:00
zpool.c	mm: zsmalloc: constify struct zs_pool name	2015-11-06 17:50:42 -08:00
zsmalloc.c	zsmalloc: Delete an unnecessary check before the function call "iput"	2016-07-28 16:07:41 -07:00
zswap.c	mm/zswap: use workqueue to destroy pool	2016-05-20 17:58:30 -07:00