redonkable
/
remarkable-linux
1
0
Fork 0
Code Releases Activity
remarkable-linux/fs
History
Roman Gushchin 6d7942377c dcache: account external names as indirectly reclaimable memory 
commit f1782c9bc5 upstream.

I received a report about suspicious growth of unreclaimable slabs on
some machines.  I've found that it happens on machines with low memory
pressure, and these unreclaimable slabs are external names attached to
dentries.

External names are allocated using generic kmalloc() function, so they
are accounted as unreclaimable.  But they are held by dentries, which
are reclaimable, and they will be reclaimed under the memory pressure.

In particular, this breaks MemAvailable calculation, as it doesn't take
unreclaimable slabs into account.  This leads to a silly situation, when
a machine is almost idle, has no memory pressure and therefore has a big
dentry cache.  And the resulting MemAvailable is too low to start a new
workload.

To address the issue, the NR_INDIRECTLY_RECLAIMABLE_BYTES counter is
used to track the amount of memory, consumed by external names.  The
counter is increased in the dentry allocation path, if an external name
structure is allocated; and it's decreased in the dentry freeing path.

To reproduce the problem I've used the following Python script:

  import os

  for iter in range (0, 10000000):
      try:
          name = ("/some_long_name_%d" % iter) + "_" * 220
          os.stat(name)
      except Exception:
          pass

Without this patch:
  $ cat /proc/meminfo | grep MemAvailable
  MemAvailable:    7811688 kB
  $ python indirect.py
  $ cat /proc/meminfo | grep MemAvailable
  MemAvailable:    2753052 kB

With the patch:
  $ cat /proc/meminfo | grep MemAvailable
  MemAvailable:    7809516 kB
  $ python indirect.py
  $ cat /proc/meminfo | grep MemAvailable
  MemAvailable:    7749144 kB

[guro@fb.com: fix indirectly reclaimable memory accounting for CONFIG_SLOB]
  Link: http://lkml.kernel.org/r/20180312194140.19517-1-guro@fb.com
[guro@fb.com: fix indirectly reclaimable memory accounting]
  Link: http://lkml.kernel.org/r/20180313125701.7955-1-guro@fb.com
Link: http://lkml.kernel.org/r/20180305133743.12746-5-guro@fb.com
Signed-off-by: Roman Gushchin <guro@fb.com>
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Mel Gorman <mgorman@techsingularity.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
 		2018-10-18 09:16:25 +02:00
..
9p fs/9p/xattr.c: catch the error of p9_client_clunk when setting xattr failed 2018-09-09 19:55:55 +02:00
adfs License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
affs affs_lookup(): close a race with affs_remove_link() 2018-05-30 07:51:47 +02:00
afs afs: Fix directory permissions check 2018-07-08 15:30:51 +02:00
autofs4 autofs: fix autofs_sbi() does not check super block type 2018-09-19 22:43:48 +02:00
befs License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
bfs License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
btrfs btrfs: btrfs_shrink_device should call commit transaction at the end 2018-10-10 08:54:19 +02:00
cachefiles cachefiles: Wait rather than BUG'ing on "Unexpected object collision" 2018-09-05 09:26:29 +02:00
ceph ceph: fix dentry leak in splice_dentry() 2018-08-24 13:09:07 +02:00
cifs smb2: fix missing files in root share directory listing 2018-10-10 08:54:27 +02:00
coda coda: fix 'kernel memory exposure attempt' in fsync 2017-11-24 08:37:05 +01:00
configfs configfs: fix registered group removal 2018-09-26 08:38:10 +02:00
cramfs License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
crypto fscrypt: use unbound workqueue for decryption 2018-08-03 07:50:32 +02:00
debugfs Merge branch 'work.mount' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-07-15 12:00:42 -07:00
devpts devpts: resolve devpts bind-mounts 2018-07-17 11:39:26 +02:00
dlm License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ecryptfs eCryptfs: don't pass up plaintext names when using filename encryption 2018-06-21 04:02:42 +09:00
efivarfs VFS: Kill off s_options and helpers 2017-07-11 06:09:21 -04:00
efs License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
exofs Merge branch 'for-4.14/block' of git://git.kernel.dk/linux-block 2017-09-07 11:59:42 -07:00
exportfs Merge branch 'rebased-statx' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-03-03 11:38:56 -08:00
ext2 dax: change bdev_dax_supported() to support boolean returns 2018-07-11 16:29:22 +02:00
ext4 ext4: show test_dummy_encryption mount option in /proc/mounts 2018-09-29 03:06:06 -07:00
f2fs f2fs: fix invalid memory access 2018-10-13 09:27:29 +02:00
fat fat: validate ->i_start before using 2018-09-15 09:45:27 +02:00
freevxfs freevxfs: update Kconfig information 2016-06-13 10:20:39 +02:00
fscache fscache: Allow cancelled operations to be enqueued 2018-09-05 09:26:29 +02:00
fuse fuse: Add missed unlock_page() to fuse_readpages_fill() 2018-09-05 09:26:38 +02:00
gfs2 gfs2: Don't reject a supposedly full bitmap if we have blocks reserved 2018-09-26 08:38:01 +02:00
hfs hfs: prevent crash on exit from failed search 2018-09-15 09:45:26 +02:00
hfsplus hfsplus: fix NULL dereference in hfsplus_lookup() 2018-09-15 09:45:27 +02:00
hostfs License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hpfs License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hugetlbfs hugetlbfs: fix bug in pgoff overflow checking 2018-04-19 08:56:21 +02:00
isofs isofs: reject hardware sector size > 2048 bytes 2018-10-03 17:00:57 -07:00
jbd2 jbd2: don't mark block as modified if the handle is out of credits 2018-07-11 16:29:16 +02:00
jffs2 do d_instantiate/unlock_new_inode combinations safely 2018-05-30 07:51:47 +02:00
jfs jfs: Fix inconsistency between memory allocation and ea_buf->max_size 2018-08-09 12:16:39 +02:00
kernfs kernfs: fix regression in kernfs_fop_write caused by wrong type 2018-02-16 20:22:59 +01:00
lockd race of lockd inetaddr notifiers vs nlmsvc_rqst change 2018-02-03 17:39:08 +01:00
minix License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ncpfs staging: ncpfs: memory corruption in ncp_read_kernel() 2018-03-28 18:24:43 +02:00
nfs NFSv4.1 fix infinite loop on I/O. 2018-09-26 08:38:09 +02:00
nfs_common lockd: fix "list_add double add" caused by legacy signal interface 2018-02-03 17:39:08 +01:00
nfsd nfsd: fix corrupted reply to badly ordered compound 2018-10-03 17:00:53 -07:00
nilfs2 do d_instantiate/unlock_new_inode combinations safely 2018-05-30 07:51:47 +02:00
nls License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
notify fsnotify: fix ignore mask logic in send_to_group() 2018-06-21 04:02:41 +09:00
ntfs License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
ocfs2 ocfs2: fix locking for res->tracking and dlm->tracking_list 2018-10-10 08:54:28 +02:00
omfs License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
openpromfs Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
orangefs orangefs: report attributes_mask and attributes for statx 2018-06-26 08:06:33 +08:00
overlayfs ovl: fix format of setxattr debug 2018-10-10 08:54:27 +02:00
proc proc: restrict kernel stack dumps to root 2018-10-10 08:54:28 +02:00
pstore pstore: Fix incorrect persistent ram buffer mapping 2018-09-26 08:38:05 +02:00
qnx4 License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
qnx6 License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
quota fs/quota: Fix spectre gadget in do_quotactl 2018-09-09 19:56:02 +02:00
ramfs mm: make pagevec_lookup() update index 2017-09-06 17:27:26 -07:00
reiserfs reiserfs: change j_timestamp type to time64_t 2018-09-15 09:45:27 +02:00
romfs License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
squashfs Squashfs: Compute expected length from inode size rather than block length 2018-09-05 09:26:32 +02:00
sysfs scsi: sysfs: Introduce sysfs_{un,}break_active_protection() 2018-09-05 09:26:41 +02:00
sysv License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
tracefs VFS: Don't use save/replace_mount_options if not using generic_show_options 2017-07-06 03:31:46 -04:00
ubifs ubifs: Check for name being NULL while mounting 2018-10-13 09:27:29 +02:00
udf udf: Detect incorrect directory size 2018-07-03 11:25:03 +02:00
ufs do d_instantiate/unlock_new_inode combinations safely 2018-05-30 07:51:47 +02:00
xfs xfs: don't call xfs_da_shrink_inode with NULL bp 2018-08-09 12:16:39 +02:00
Kconfig fs/Kconfig: kill CONFIG_PERCPU_RWSEM some more 2017-07-12 16:26:00 -07:00
Kconfig.binfmt docs: fix locations of several documents that got moved 2016-10-24 08:12:35 -02:00
Makefile License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
aio.c fix io_destroy()/aio_complete() race 2018-06-05 11:41:54 +02:00
anon_inodes.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
attr.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
bad_inode.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
binfmt_aout.c fs: fix kernel_read prototype 2017-09-04 19:05:15 -04:00
binfmt_elf.c binfmt_elf: Respect error return from `regset->active' 2018-09-26 08:38:09 +02:00
binfmt_elf_fdpic.c Merge branch 'work.set_fs' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-09-14 18:13:32 -07:00
binfmt_em86.c fs/binfmt_em86.c: fix incompatible pointer type 2016-08-02 19:35:15 -04:00
binfmt_flat.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
binfmt_misc.c fs/binfmt_misc.c: do not allow offset overflow 2018-06-26 08:06:33 +08:00
binfmt_script.c exec: load_script: kill the onstack interp[BINPRM_BUF_SIZE] array 2017-10-03 17:54:25 -07:00
block_dev.c blkdev: __blkdev_direct_IO_simple: fix leak in error case 2018-08-03 07:50:42 +02:00
buffer.c fs: guard_bio_eod() needs to consider partitions 2017-11-30 08:40:45 +00:00
char_dev.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
compat.c fs/compat.c: trim unused includes 2017-04-17 12:52:27 -04:00
compat_binfmt_elf.c fs/binfmt: Convert obsolete cputime type to nsecs 2017-02-01 09:13:51 +01:00
compat_ioctl.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
coredump.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
dax.c fs/dax.c: release PMD lock even when there is no PMD support in DAX 2018-04-26 11:02:14 +02:00
dcache.c dcache: account external names as indirectly reclaimable memory 2018-10-18 09:16:25 +02:00
dcookies.c Replace <asm/uaccess.h> with <linux/uaccess.h> globally 2016-12-24 11:46:01 -08:00
direct-io.c direct-io: Fix sleep in atomic due to sync AIO 2018-03-08 22:41:06 -08:00
drop_caches.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
eventfd.c There has been a fair amount of activity in the docs tree this time 2017-07-03 21:13:25 -07:00
eventpoll.c fs/epoll: use faster rb_first_cached() 2017-09-08 18:26:49 -07:00
exec.c exec: avoid gcc-8 warning for get_task_comm 2018-03-03 10:24:21 +01:00
fcntl.c fcntl: don't cap l_start and l_end values for F_GETLK64 in compat syscall 2017-12-17 15:07:59 +01:00
fhandle.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
file.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
file_table.c fput: Don't reinvent the wheel but use existing llist API 2017-08-28 00:50:23 -04:00
filesystems.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
fs-writeback.c bdi: Fix oops in wb_workfn() 2018-05-16 10:10:25 +02:00
fs_pin.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
fs_struct.c sched/headers: Prepare for new header dependencies before moving code to <linux/sched/task.h> 2017-03-02 08:42:35 +01:00
inode.c Fix up non-directory creation in SGID directories 2018-07-17 11:39:27 +02:00
internal.h Merge branch 'overlayfs-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs 2017-09-13 09:11:44 -07:00
ioctl.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
iomap.c iomap: complete partial direct I/O writes synchronously 2018-10-03 17:00:53 -07:00
libfs.c fs: convert __generic_file_fsync to use errseq_t based reporting 2017-07-06 07:02:29 -04:00
locks.c fs/lock: skip lock owner pid translation in case we are in init_pid_ns 2018-10-03 17:00:53 -07:00
mbcache.c mbcache: initialize entry->e_referenced in mb_cache_entry_create() 2018-02-22 15:42:25 +01:00
mount.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
mpage.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
namei.c getname_kernel() needs to make sure that ->name != ->iname in long case 2018-04-19 08:56:19 +02:00
namespace.c vfs: fix freeze protection in mnt_want_write_file() for overlayfs 2018-09-26 08:38:09 +02:00
no-block.c
…
nsfs.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
open.c ovl: don't allow writing ioctl on lower layer 2017-09-05 12:53:12 +02:00
pipe.c pipe: fix off-by-one error when checking buffer limits 2018-02-16 20:23:05 +01:00
pnode.c mnt: Make propagate_umount less slow for overlapping mount propagation trees 2017-05-23 08:41:17 -05:00
pnode.h mnt: Tuck mounts under others instead of creating shadow/side mounts. 2017-02-04 00:01:06 +13:00
posix_acl.c sched/headers: Prepare to remove <linux/cred.h> inclusion from <linux/sched.h> 2017-03-02 08:42:31 +01:00
proc_namespace.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
read_write.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
readdir.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
select.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
seq_file.c seq_file: fix incomplete reset on read from zero offset 2018-02-22 15:42:28 +01:00
signalfd.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
splice.c fs: move kernel_write to fs/read_write.c 2017-09-04 19:05:15 -04:00
stack.c
…
stat.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
statfs.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
super.c fs: don't scan the inode cache before SB_BORN is set 2018-05-30 07:51:47 +02:00
sync.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
timerfd.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
userfaultfd.c userfaultfd: remove uffd flags from vma->vm_flags if UFFD_EVENT_FORK fails 2018-08-06 16:20:49 +02:00
utimes.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xattr.c sysfs: Do not return POSIX ACL xattrs via listxattr 2018-10-10 08:54:27 +02:00