redonkable
/
alistair23-linux
1
0
Fork 0
Code Releases Activity
alistair23-linux/fs
History
Roman Gushchin f1782c9bc5 dcache: account external names as indirectly reclaimable memory 
I received a report about suspicious growth of unreclaimable slabs on
some machines.  I've found that it happens on machines with low memory
pressure, and these unreclaimable slabs are external names attached to
dentries.

External names are allocated using generic kmalloc() function, so they
are accounted as unreclaimable.  But they are held by dentries, which
are reclaimable, and they will be reclaimed under the memory pressure.

In particular, this breaks MemAvailable calculation, as it doesn't take
unreclaimable slabs into account.  This leads to a silly situation, when
a machine is almost idle, has no memory pressure and therefore has a big
dentry cache.  And the resulting MemAvailable is too low to start a new
workload.

To address the issue, the NR_INDIRECTLY_RECLAIMABLE_BYTES counter is
used to track the amount of memory, consumed by external names.  The
counter is increased in the dentry allocation path, if an external name
structure is allocated; and it's decreased in the dentry freeing path.

To reproduce the problem I've used the following Python script:

  import os

  for iter in range (0, 10000000):
      try:
          name = ("/some_long_name_%d" % iter) + "_" * 220
          os.stat(name)
      except Exception:
          pass

Without this patch:
  $ cat /proc/meminfo | grep MemAvailable
  MemAvailable:    7811688 kB
  $ python indirect.py
  $ cat /proc/meminfo | grep MemAvailable
  MemAvailable:    2753052 kB

With the patch:
  $ cat /proc/meminfo | grep MemAvailable
  MemAvailable:    7809516 kB
  $ python indirect.py
  $ cat /proc/meminfo | grep MemAvailable
  MemAvailable:    7749144 kB

[guro@fb.com: fix indirectly reclaimable memory accounting for CONFIG_SLOB]
  Link: http://lkml.kernel.org/r/20180312194140.19517-1-guro@fb.com
[guro@fb.com: fix indirectly reclaimable memory accounting]
  Link: http://lkml.kernel.org/r/20180313125701.7955-1-guro@fb.com
Link: http://lkml.kernel.org/r/20180305133743.12746-5-guro@fb.com
Signed-off-by: Roman Gushchin <guro@fb.com>
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
 		2018-04-11 10:28:29 -07:00
..
9p fscache development 2018-04-07 09:08:24 -07:00
adfs Rename superblock flags (MS_xyz -> SB_xyz) 2017-11-27 13:05:09 -08:00
affs iversion: Rename make inode_cmp_iversion{+raw} to inode_eq_iversion{+raw} 2018-02-01 08:15:25 -05:00
afs fscache: Pass object size in rather than calling back for it 2018-04-06 14:05:14 +01:00
autofs4 fs: add ksys_close() wrapper; remove in-kernel calls to sys_close() 2018-04-02 20:16:00 +02:00
befs befs: Define usercopy region in befs_inode_cache slab cache 2018-01-15 12:07:54 -08:00
bfs License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
btrfs for-4.17-tag 2018-04-04 13:03:38 -07:00
cachefiles fscache: Pass object size in rather than calling back for it 2018-04-06 14:05:14 +01:00
ceph fscache development 2018-04-07 09:08:24 -07:00
cifs fscache development 2018-04-07 09:08:24 -07:00
coda vfs: do bulk POLL* -> EPOLL* replacement 2018-02-11 14:34:03 -08:00
configfs configfs: make ci_type field, some pointers and function arguments const 2017-10-19 16:15:16 +02:00
cramfs cramfs: better MTD dependency expression 2018-02-08 11:37:31 -08:00
crypto fscrypt: fix build with pre-4.6 gcc versions 2018-02-01 10:51:18 -05:00
debugfs debugfs_lookup(): switch to lookup_one_len_unlocked() 2018-03-29 15:07:47 -04:00
devpts devpts: comment devpts_mntget() 2018-03-14 13:31:23 +01:00
dlm net: make getname() functions return length rather than use int* parameter 2018-02-12 14:15:04 -05:00
ecryptfs vfs: do bulk POLL* -> EPOLL* replacement 2018-02-11 14:34:03 -08:00
efivarfs efivarfs: Limit the rate for non-root to read files 2018-02-22 10:21:02 -08:00
efs Rename superblock flags (MS_xyz -> SB_xyz) 2017-11-27 13:05:09 -08:00
exofs iversion.h related cleanup for v4.16 2018-02-07 14:25:22 -08:00
exportfs
…
ext2 libnvdimm for 4.17 2018-04-10 10:25:57 -07:00
ext4 libnvdimm for 4.17 2018-04-10 10:25:57 -07:00
f2fs f2fs: remain written times to update inode during fsync 2018-04-03 18:52:47 -07:00
fat iversion: Rename make inode_cmp_iversion{+raw} to inode_eq_iversion{+raw} 2018-02-01 08:15:25 -05:00
freevxfs vxfs: Define usercopy region in vxfs_inode slab cache 2018-01-15 12:07:57 -08:00
fscache fscache: Maintain a catalogue of allocated cookies 2018-04-06 14:05:14 +01:00
fuse fuse: define the filesystem as untrusted 2018-03-23 06:31:37 -04:00
gfs2 Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2018-04-06 11:07:08 -07:00
hfs Rename superblock flags (MS_xyz -> SB_xyz) 2017-11-27 13:05:09 -08:00
hfsplus hfsplus: honor setgid flag on directories 2018-02-06 18:32:45 -08:00
hostfs hostfs: rename do_rmdir() to hostfs_do_rmdir() 2018-04-02 20:15:53 +02:00
hpfs hpfs: don't bother with the i_version counter or f_version 2017-12-10 12:58:18 -08:00
hugetlbfs hugetlbfs: fix bug in pgoff overflow checking 2018-04-05 21:36:21 -07:00
isofs Rename superblock flags (MS_xyz -> SB_xyz) 2017-11-27 13:05:09 -08:00
jbd2 jbd2: if the journal is aborted then don't allow update of the log tail 2018-02-19 12:22:53 -05:00
jffs2 mtd: Unconditionally update ->fail_addr and ->addr in part_erase() 2018-03-15 18:22:26 +01:00
jfs Currently, hardened usercopy performs dynamic bounds checking on slab 2018-02-03 16:25:42 -08:00
kernfs vfs: do bulk POLL* -> EPOLL* replacement 2018-02-11 14:34:03 -08:00
lockd net: Drop pernet_operations::async 2018-03-27 13:18:09 -04:00
minix treewide: simplify Kconfig dependencies for removed archs 2018-03-26 15:55:57 +02:00
nfs fscache development 2018-04-07 09:08:24 -07:00
nfs_common net: Drop pernet_operations::async 2018-03-27 13:18:09 -04:00
nfsd nfsd: fix incorrect umasks 2018-04-03 16:27:08 -04:00
nilfs2 nilfs2: use time64_t internally 2018-02-06 18:32:45 -08:00
nls License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
notify Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs 2018-04-05 19:17:50 -07:00
ntfs ntfs: fix bogus __mark_inode_dirty(I_DIRTY_SYNC | I_DIRTY_DATASYNC) call 2018-03-28 01:39:02 -04:00
ocfs2 Merge branch 'akpm' (patches from Andrew) 2018-04-06 14:19:26 -07:00
omfs License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
openpromfs Rename superblock flags (MS_xyz -> SB_xyz) 2017-11-27 13:05:09 -08:00
orangefs orangefs: fixes and cleanups 2018-04-09 12:45:47 -07:00
overlayfs ovl: update Kconfig texts 2018-03-07 11:47:15 +01:00
proc net: Drop pernet_operations::async 2018-03-27 13:18:09 -04:00
pstore pstore: fix crypto dependencies without compression 2018-04-06 15:45:33 -07:00
qnx4 Rename superblock flags (MS_xyz -> SB_xyz) 2017-11-27 13:05:09 -08:00
qnx6 Rename superblock flags (MS_xyz -> SB_xyz) 2017-11-27 13:05:09 -08:00
quota fs/quota: use COMPAT_SYSCALL_DEFINE for sys32_quotactl() 2018-04-02 20:15:47 +02:00
ramfs mm: make pagevec_lookup() update index 2017-09-06 17:27:26 -07:00
reiserfs reiserfs: Remove VLA from fs/reiserfs/reiserfs.h 2018-03-09 13:04:20 +01:00
romfs Rename superblock flags (MS_xyz -> SB_xyz) 2017-11-27 13:05:09 -08:00
squashfs Rename superblock flags (MS_xyz -> SB_xyz) 2017-11-27 13:05:09 -08:00
sysfs sysfs: symlink: export sysfs_create_link_nowarn() 2018-03-19 21:14:26 -04:00
sysv Rename superblock flags (MS_xyz -> SB_xyz) 2017-11-27 13:05:09 -08:00
tracefs VFS: Don't use save/replace_mount_options if not using generic_show_options 2017-07-06 03:31:46 -04:00
ubifs ubifs: fix bogus __mark_inode_dirty(I_DIRTY_SYNC | I_DIRTY_DATASYNC) call 2018-03-28 01:39:02 -04:00
udf udf: fix potential refcnt problem of nls module 2018-03-02 14:23:12 +01:00
ufs iversion.h related cleanup for v4.16 2018-02-07 14:25:22 -08:00
xfs libnvdimm for 4.17 2018-04-10 10:25:57 -07:00
Kconfig libnvdimm for 4.16 2018-02-06 10:41:33 -08:00
Kconfig.binfmt treewide: simplify Kconfig dependencies for removed archs 2018-03-26 15:55:57 +02:00
Makefile split d_path() and friends into a separate file 2018-03-29 15:07:46 -04:00
aio.c fs/aio: Use rcu_work instead of explicit rcu and work item 2018-03-19 10:12:03 -07:00
anon_inodes.c
…
attr.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
bad_inode.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
binfmt_aout.c fs: fix kernel_read prototype 2017-09-04 19:05:15 -04:00
binfmt_elf.c elf: fix NT_FILE integer overflow 2018-02-06 18:32:45 -08:00
binfmt_elf_fdpic.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-11-17 12:54:01 -08:00
binfmt_em86.c
…
binfmt_flat.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
binfmt_misc.c fs: add ksys_close() wrapper; remove in-kernel calls to sys_close() 2018-04-02 20:16:00 +02:00
binfmt_script.c exec: load_script: kill the onstack interp[BINPRM_BUF_SIZE] array 2017-10-03 17:54:25 -07:00
block_dev.c libnvdimm for 4.17 2018-04-10 10:25:57 -07:00
buffer.c block_invalidatepage(): only release page if the full page was invalidated 2018-04-05 21:36:27 -07:00
char_dev.c block, char_dev: Use correct format specifier for unsigned ints 2018-03-15 17:59:24 +01:00
compat.c
…
compat_binfmt_elf.c
…
compat_ioctl.c fs: compat_ioctl: add new DVB demux ioctls 2017-12-28 11:17:29 -05:00
coredump.c Merge branch 'misc.compat' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-11-17 11:54:55 -08:00
d_path.c split d_path() and friends into a separate file 2018-03-29 15:07:46 -04:00
dax.c fs, dax: use page->mapping to warn if truncate collides with a busy page 2018-04-03 05:41:19 -07:00
dcache.c dcache: account external names as indirectly reclaimable memory 2018-04-11 10:28:29 -07:00
dcookies.c fs: add do_lookup_dcookie() helper; remove in-kernel call to syscall 2018-04-02 20:15:39 +02:00
direct-io.c Merge branch 'akpm' (patches from Andrew) 2018-04-06 14:19:26 -07:00
drop_caches.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
eventfd.c fs: add do_eventfd() helper; remove internal call to sys_eventfd() 2018-04-02 20:15:39 +02:00
eventpoll.c fs: add do_epoll_*() helpers; remove internal calls to sys_epoll_*() 2018-04-02 20:15:37 +02:00
exec.c exec: Set file unwritable before LSM check 2018-03-19 15:49:32 +11:00
fcntl.c fs: add do_compat_fcntl64() helper; remove in-kernel call to compat syscall 2018-04-02 20:15:42 +02:00
fhandle.c vfs: Copy struct mount.mnt_id to userspace using put_user() 2018-01-15 12:07:51 -08:00
file.c fs: add ksys_close() wrapper; remove in-kernel calls to sys_close() 2018-04-02 20:16:00 +02:00
file_table.c vfs: remove unused hardirq.h 2017-12-07 14:23:30 -05:00
filesystems.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
fs-writeback.c fs: move I_DIRTY_INODE to fs.h 2018-03-28 01:39:02 -04:00
fs_pin.c Merge branch 'linus' into locking/core, to resolve conflicts 2017-11-07 10:32:44 +01:00
fs_struct.c
…
inode.c inode: don't memset the inode address space twice 2018-03-11 20:27:56 -07:00
internal.h Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2018-04-06 11:07:08 -07:00
ioctl.c fs: add ksys_ioctl() helper; remove in-kernel calls to sys_ioctl() 2018-04-02 20:16:03 +02:00
iomap.c iomap: warn on zero-length mappings 2018-01-29 07:27:24 -08:00
libfs.c fs, dax: prepare for dax-specific address_space_operations 2018-03-30 11:34:55 -07:00
locks.c treewide: Align function definition open/close braces 2018-03-26 11:13:09 +02:00
mbcache.c mbcache: make sure c_entry_count is not decremented past zero 2018-01-09 23:57:52 -05:00
mount.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
mpage.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
namei.c Merge branch 'work.namei' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2018-04-09 12:48:05 -07:00
namespace.c fs: add ksys_umount() helper; remove in-kernel call to sys_umount() 2018-04-02 20:15:48 +02:00
no-block.c
…
nsfs.c net: Export open_related_ns() 2018-02-15 15:34:42 -05:00
open.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2018-04-06 11:07:08 -07:00
pipe.c fs: add do_pipe2() helper; remove internal call to sys_pipe2() 2018-04-02 20:15:35 +02:00
pnode.c
…
pnode.h
…
posix_acl.c posix_acl: convert posix_acl.a_refcount from atomic_t to refcount_t 2018-01-02 19:27:28 -08:00
proc_namespace.c vfs: do bulk POLL* -> EPOLL* replacement 2018-02-11 14:34:03 -08:00
read_write.c fs: add ksys_p{read,write}64() helpers; remove in-kernel calls to syscalls 2018-04-02 20:16:09 +02:00
readdir.c fs: add ksys_getdents64() helper; remove in-kernel calls to sys_getdents64() 2018-04-02 20:16:02 +02:00
select.c fs: add do_compat_select() helper; remove in-kernel call to compat syscall 2018-04-02 20:15:42 +02:00
seq_file.c seq_file: fix incomplete reset on read from zero offset 2018-01-20 02:31:15 -05:00
signalfd.c fs: add do_compat_signalfd4() helper; remove in-kernel call to compat syscall 2018-04-02 20:15:43 +02:00
splice.c fs: add do_vmsplice() helper; remove in-kernel call to syscall 2018-04-02 20:15:40 +02:00
stack.c
…
stat.c fs: add do_readlinkat() helper; remove internal call to sys_readlinkat() 2018-04-02 20:15:34 +02:00
statfs.c Rename superblock flags (MS_xyz -> SB_xyz) 2017-11-27 13:05:09 -08:00
super.c Merge branch 'work.misc' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2018-01-31 09:25:20 -08:00
sync.c Changes for this release: 2018-04-04 12:44:02 -07:00
timerfd.c vfs: do bulk POLL* -> EPOLL* replacement 2018-02-11 14:34:03 -08:00
userfaultfd.c vfs: do bulk POLL* -> EPOLL* replacement 2018-02-11 14:34:03 -08:00
utimes.c fs: add do_compat_futimesat() helper; remove in-kernel call to compat syscall 2018-04-02 20:15:44 +02:00
xattr.c lsm: fix smack_inode_removexattr and xattr_getsecurity memleak 2017-10-04 18:03:15 +11:00