2019-05-27 00:55:01 -06:00
|
|
|
// SPDX-License-Identifier: GPL-2.0-or-later
|
2007-04-26 16:55:03 -06:00
|
|
|
/* AFS filesystem file handling
|
2005-04-16 16:20:36 -06:00
|
|
|
*
|
2007-04-26 16:55:03 -06:00
|
|
|
* Copyright (C) 2002, 2007 Red Hat, Inc. All Rights Reserved.
|
2005-04-16 16:20:36 -06:00
|
|
|
* Written by David Howells (dhowells@redhat.com)
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include <linux/kernel.h>
|
|
|
|
#include <linux/module.h>
|
|
|
|
#include <linux/init.h>
|
|
|
|
#include <linux/fs.h>
|
|
|
|
#include <linux/pagemap.h>
|
AFS: implement basic file write support
Implement support for writing to regular AFS files, including:
(1) write
(2) truncate
(3) fsync, fdatasync
(4) chmod, chown, chgrp, utime.
AFS writeback attempts to batch writes into as chunks as large as it can manage
up to the point that it writes back 65535 pages in one chunk or it meets a
locked page.
Furthermore, if a page has been written to using a particular key, then should
another write to that page use some other key, the first write will be flushed
before the second is allowed to take place. If the first write fails due to a
security error, then the page will be scrapped and reread before the second
write takes place.
If a page is dirty and the callback on it is broken by the server, then the
dirty data is not discarded (same behaviour as NFS).
Shared-writable mappings are not supported by this patch.
[akpm@linux-foundation.org: fix a bunch of warnings]
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-05-09 03:33:46 -06:00
|
|
|
#include <linux/writeback.h>
|
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 02:04:11 -06:00
|
|
|
#include <linux/gfp.h>
|
2017-01-05 03:38:35 -07:00
|
|
|
#include <linux/task_io_accounting_ops.h>
|
2019-01-03 16:29:02 -07:00
|
|
|
#include <linux/mm.h>
|
2005-04-16 16:20:36 -06:00
|
|
|
#include "internal.h"
|
|
|
|
|
2017-11-02 09:27:52 -06:00
|
|
|
static int afs_file_mmap(struct file *file, struct vm_area_struct *vma);
|
2007-05-09 03:33:45 -06:00
|
|
|
static int afs_readpage(struct file *file, struct page *page);
|
2013-05-21 21:17:23 -06:00
|
|
|
static void afs_invalidatepage(struct page *page, unsigned int offset,
|
|
|
|
unsigned int length);
|
2007-05-09 03:33:45 -06:00
|
|
|
static int afs_releasepage(struct page *page, gfp_t gfp_flags);
|
2005-04-16 16:20:36 -06:00
|
|
|
|
2009-04-03 09:42:41 -06:00
|
|
|
static int afs_readpages(struct file *filp, struct address_space *mapping,
|
|
|
|
struct list_head *pages, unsigned nr_pages);
|
|
|
|
|
2007-04-26 16:57:07 -06:00
|
|
|
const struct file_operations afs_file_operations = {
|
|
|
|
.open = afs_open,
|
|
|
|
.release = afs_release,
|
|
|
|
.llseek = generic_file_llseek,
|
2014-04-02 12:33:16 -06:00
|
|
|
.read_iter = generic_file_read_iter,
|
2014-04-03 12:13:46 -06:00
|
|
|
.write_iter = afs_file_write,
|
2017-11-02 09:27:52 -06:00
|
|
|
.mmap = afs_file_mmap,
|
2007-06-01 03:49:19 -06:00
|
|
|
.splice_read = generic_file_splice_read,
|
AFS: implement basic file write support
Implement support for writing to regular AFS files, including:
(1) write
(2) truncate
(3) fsync, fdatasync
(4) chmod, chown, chgrp, utime.
AFS writeback attempts to batch writes into as chunks as large as it can manage
up to the point that it writes back 65535 pages in one chunk or it meets a
locked page.
Furthermore, if a page has been written to using a particular key, then should
another write to that page use some other key, the first write will be flushed
before the second is allowed to take place. If the first write fails due to a
security error, then the page will be scrapped and reread before the second
write takes place.
If a page is dirty and the callback on it is broken by the server, then the
dirty data is not discarded (same behaviour as NFS).
Shared-writable mappings are not supported by this patch.
[akpm@linux-foundation.org: fix a bunch of warnings]
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-05-09 03:33:46 -06:00
|
|
|
.fsync = afs_fsync,
|
2007-07-16 00:40:12 -06:00
|
|
|
.lock = afs_lock,
|
|
|
|
.flock = afs_flock,
|
2007-04-26 16:57:07 -06:00
|
|
|
};
|
|
|
|
|
2007-02-12 01:55:38 -07:00
|
|
|
const struct inode_operations afs_file_inode_operations = {
|
2007-05-09 03:33:45 -06:00
|
|
|
.getattr = afs_getattr,
|
AFS: implement basic file write support
Implement support for writing to regular AFS files, including:
(1) write
(2) truncate
(3) fsync, fdatasync
(4) chmod, chown, chgrp, utime.
AFS writeback attempts to batch writes into as chunks as large as it can manage
up to the point that it writes back 65535 pages in one chunk or it meets a
locked page.
Furthermore, if a page has been written to using a particular key, then should
another write to that page use some other key, the first write will be flushed
before the second is allowed to take place. If the first write fails due to a
security error, then the page will be scrapped and reread before the second
write takes place.
If a page is dirty and the callback on it is broken by the server, then the
dirty data is not discarded (same behaviour as NFS).
Shared-writable mappings are not supported by this patch.
[akpm@linux-foundation.org: fix a bunch of warnings]
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-05-09 03:33:46 -06:00
|
|
|
.setattr = afs_setattr,
|
2007-04-26 16:57:07 -06:00
|
|
|
.permission = afs_permission,
|
2017-07-06 08:50:27 -06:00
|
|
|
.listxattr = afs_listxattr,
|
2005-04-16 16:20:36 -06:00
|
|
|
};
|
|
|
|
|
2006-06-28 05:26:44 -06:00
|
|
|
const struct address_space_operations afs_fs_aops = {
|
2007-05-09 03:33:45 -06:00
|
|
|
.readpage = afs_readpage,
|
2009-04-03 09:42:41 -06:00
|
|
|
.readpages = afs_readpages,
|
AFS: implement basic file write support
Implement support for writing to regular AFS files, including:
(1) write
(2) truncate
(3) fsync, fdatasync
(4) chmod, chown, chgrp, utime.
AFS writeback attempts to batch writes into as chunks as large as it can manage
up to the point that it writes back 65535 pages in one chunk or it meets a
locked page.
Furthermore, if a page has been written to using a particular key, then should
another write to that page use some other key, the first write will be flushed
before the second is allowed to take place. If the first write fails due to a
security error, then the page will be scrapped and reread before the second
write takes place.
If a page is dirty and the callback on it is broken by the server, then the
dirty data is not discarded (same behaviour as NFS).
Shared-writable mappings are not supported by this patch.
[akpm@linux-foundation.org: fix a bunch of warnings]
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-05-09 03:33:46 -06:00
|
|
|
.set_page_dirty = afs_set_page_dirty,
|
|
|
|
.launder_page = afs_launder_page,
|
2007-05-09 03:33:45 -06:00
|
|
|
.releasepage = afs_releasepage,
|
|
|
|
.invalidatepage = afs_invalidatepage,
|
2008-10-15 23:04:32 -06:00
|
|
|
.write_begin = afs_write_begin,
|
|
|
|
.write_end = afs_write_end,
|
AFS: implement basic file write support
Implement support for writing to regular AFS files, including:
(1) write
(2) truncate
(3) fsync, fdatasync
(4) chmod, chown, chgrp, utime.
AFS writeback attempts to batch writes into as chunks as large as it can manage
up to the point that it writes back 65535 pages in one chunk or it meets a
locked page.
Furthermore, if a page has been written to using a particular key, then should
another write to that page use some other key, the first write will be flushed
before the second is allowed to take place. If the first write fails due to a
security error, then the page will be scrapped and reread before the second
write takes place.
If a page is dirty and the callback on it is broken by the server, then the
dirty data is not discarded (same behaviour as NFS).
Shared-writable mappings are not supported by this patch.
[akpm@linux-foundation.org: fix a bunch of warnings]
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2007-05-09 03:33:46 -06:00
|
|
|
.writepage = afs_writepage,
|
|
|
|
.writepages = afs_writepages,
|
2005-04-16 16:20:36 -06:00
|
|
|
};
|
|
|
|
|
2017-11-02 09:27:52 -06:00
|
|
|
static const struct vm_operations_struct afs_vm_ops = {
|
|
|
|
.fault = filemap_fault,
|
|
|
|
.map_pages = filemap_map_pages,
|
|
|
|
.page_mkwrite = afs_page_mkwrite,
|
|
|
|
};
|
|
|
|
|
2017-11-02 09:27:52 -06:00
|
|
|
/*
|
|
|
|
* Discard a pin on a writeback key.
|
|
|
|
*/
|
|
|
|
void afs_put_wb_key(struct afs_wb_key *wbk)
|
|
|
|
{
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 13:51:51 -06:00
|
|
|
if (wbk && refcount_dec_and_test(&wbk->usage)) {
|
2017-11-02 09:27:52 -06:00
|
|
|
key_put(wbk->key);
|
|
|
|
kfree(wbk);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Cache key for writeback.
|
|
|
|
*/
|
|
|
|
int afs_cache_wb_key(struct afs_vnode *vnode, struct afs_file *af)
|
|
|
|
{
|
|
|
|
struct afs_wb_key *wbk, *p;
|
|
|
|
|
|
|
|
wbk = kzalloc(sizeof(struct afs_wb_key), GFP_KERNEL);
|
|
|
|
if (!wbk)
|
|
|
|
return -ENOMEM;
|
|
|
|
refcount_set(&wbk->usage, 2);
|
|
|
|
wbk->key = af->key;
|
|
|
|
|
|
|
|
spin_lock(&vnode->wb_lock);
|
|
|
|
list_for_each_entry(p, &vnode->wb_keys, vnode_link) {
|
|
|
|
if (p->key == wbk->key)
|
|
|
|
goto found;
|
|
|
|
}
|
|
|
|
|
|
|
|
key_get(wbk->key);
|
|
|
|
list_add_tail(&wbk->vnode_link, &vnode->wb_keys);
|
|
|
|
spin_unlock(&vnode->wb_lock);
|
|
|
|
af->wb = wbk;
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
found:
|
|
|
|
refcount_inc(&p->usage);
|
|
|
|
spin_unlock(&vnode->wb_lock);
|
|
|
|
af->wb = p;
|
|
|
|
kfree(wbk);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
2007-04-26 16:57:07 -06:00
|
|
|
/*
|
|
|
|
* open an AFS file or directory and attach a key to it
|
|
|
|
*/
|
|
|
|
int afs_open(struct inode *inode, struct file *file)
|
|
|
|
{
|
|
|
|
struct afs_vnode *vnode = AFS_FS_I(inode);
|
2017-11-02 09:27:52 -06:00
|
|
|
struct afs_file *af;
|
2007-04-26 16:57:07 -06:00
|
|
|
struct key *key;
|
2007-04-26 16:59:35 -06:00
|
|
|
int ret;
|
2007-04-26 16:57:07 -06:00
|
|
|
|
2018-10-19 17:57:57 -06:00
|
|
|
_enter("{%llx:%llu},", vnode->fid.vid, vnode->fid.vnode);
|
2007-04-26 16:57:07 -06:00
|
|
|
|
|
|
|
key = afs_request_key(vnode->volume->cell);
|
|
|
|
if (IS_ERR(key)) {
|
2017-11-02 09:27:52 -06:00
|
|
|
ret = PTR_ERR(key);
|
|
|
|
goto error;
|
2007-04-26 16:57:07 -06:00
|
|
|
}
|
|
|
|
|
2017-11-02 09:27:52 -06:00
|
|
|
af = kzalloc(sizeof(*af), GFP_KERNEL);
|
|
|
|
if (!af) {
|
|
|
|
ret = -ENOMEM;
|
|
|
|
goto error_key;
|
2007-04-26 16:59:35 -06:00
|
|
|
}
|
2017-11-02 09:27:52 -06:00
|
|
|
af->key = key;
|
2007-04-26 16:59:35 -06:00
|
|
|
|
2017-11-02 09:27:52 -06:00
|
|
|
ret = afs_validate(vnode, key);
|
|
|
|
if (ret < 0)
|
|
|
|
goto error_af;
|
|
|
|
|
2017-11-02 09:27:52 -06:00
|
|
|
if (file->f_mode & FMODE_WRITE) {
|
|
|
|
ret = afs_cache_wb_key(vnode, af);
|
|
|
|
if (ret < 0)
|
|
|
|
goto error_af;
|
|
|
|
}
|
2018-04-06 07:17:26 -06:00
|
|
|
|
|
|
|
if (file->f_flags & O_TRUNC)
|
|
|
|
set_bit(AFS_VNODE_NEW_CONTENT, &vnode->flags);
|
2017-11-02 09:27:52 -06:00
|
|
|
|
2017-11-02 09:27:52 -06:00
|
|
|
file->private_data = af;
|
2007-04-26 16:57:07 -06:00
|
|
|
_leave(" = 0");
|
|
|
|
return 0;
|
2017-11-02 09:27:52 -06:00
|
|
|
|
|
|
|
error_af:
|
|
|
|
kfree(af);
|
|
|
|
error_key:
|
|
|
|
key_put(key);
|
|
|
|
error:
|
|
|
|
_leave(" = %d", ret);
|
|
|
|
return ret;
|
2007-04-26 16:57:07 -06:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* release an AFS file or directory and discard its key
|
|
|
|
*/
|
|
|
|
int afs_release(struct inode *inode, struct file *file)
|
|
|
|
{
|
|
|
|
struct afs_vnode *vnode = AFS_FS_I(inode);
|
2017-11-02 09:27:52 -06:00
|
|
|
struct afs_file *af = file->private_data;
|
2019-05-15 05:09:17 -06:00
|
|
|
int ret = 0;
|
2007-04-26 16:57:07 -06:00
|
|
|
|
2018-10-19 17:57:57 -06:00
|
|
|
_enter("{%llx:%llu},", vnode->fid.vid, vnode->fid.vnode);
|
2007-04-26 16:57:07 -06:00
|
|
|
|
2018-04-06 07:17:26 -06:00
|
|
|
if ((file->f_mode & FMODE_WRITE))
|
2019-05-15 05:09:17 -06:00
|
|
|
ret = vfs_fsync(file, 0);
|
2018-04-06 07:17:26 -06:00
|
|
|
|
2017-11-02 09:27:52 -06:00
|
|
|
file->private_data = NULL;
|
2017-11-02 09:27:52 -06:00
|
|
|
if (af->wb)
|
|
|
|
afs_put_wb_key(af->wb);
|
2017-11-02 09:27:52 -06:00
|
|
|
key_put(af->key);
|
|
|
|
kfree(af);
|
2017-11-02 09:27:52 -06:00
|
|
|
afs_prune_wb_keys(vnode);
|
2019-05-15 05:09:17 -06:00
|
|
|
_leave(" = %d", ret);
|
|
|
|
return ret;
|
2007-04-26 16:57:07 -06:00
|
|
|
}
|
|
|
|
|
2017-01-05 03:38:34 -07:00
|
|
|
/*
|
|
|
|
* Dispose of a ref to a read record.
|
|
|
|
*/
|
|
|
|
void afs_put_read(struct afs_read *req)
|
|
|
|
{
|
|
|
|
int i;
|
|
|
|
|
afs: Fix directory handling
AFS directories are structured blobs that are downloaded just like files
and then parsed by the lookup and readdir code and, as such, are currently
handled in the pagecache like any other file, with the entire directory
content being thrown away each time the directory changes.
However, since the blob is a known structure and since the data version
counter on a directory increases by exactly one for each change committed
to that directory, we can actually edit the directory locally rather than
fetching it from the server after each locally-induced change.
What we can't do, though, is mix data from the server and data from the
client since the server is technically at liberty to rearrange or compress
a directory if it sees fit, provided it updates the data version number
when it does so and breaks the callback (ie. sends a notification).
Further, lookup with lookup-ahead, readdir and, when it arrives, local
editing are likely want to scan the whole of a directory.
So directory handling needs to be improved to maintain the coherency of the
directory blob prior to permitting local directory editing.
To this end:
(1) If any directory page gets discarded, invalidate and reread the entire
directory.
(2) If readpage notes that if when it fetches a single page that the
version number has changed, the entire directory is flagged for
invalidation.
(3) Read as much of the directory in one go as we can.
Note that this removes local caching of directories in fscache for the
moment as we can't pass the pages to fscache_read_or_alloc_pages() since
page->lru is in use by the LRU.
Signed-off-by: David Howells <dhowells@redhat.com>
2018-04-06 07:17:25 -06:00
|
|
|
if (refcount_dec_and_test(&req->usage)) {
|
2019-07-30 07:38:51 -06:00
|
|
|
if (req->pages) {
|
|
|
|
for (i = 0; i < req->nr_pages; i++)
|
|
|
|
if (req->pages[i])
|
|
|
|
put_page(req->pages[i]);
|
|
|
|
if (req->pages != req->array)
|
|
|
|
kfree(req->pages);
|
|
|
|
}
|
2017-01-05 03:38:34 -07:00
|
|
|
kfree(req);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2009-04-17 05:56:38 -06:00
|
|
|
#ifdef CONFIG_AFS_FSCACHE
|
2005-04-16 16:20:36 -06:00
|
|
|
/*
|
|
|
|
* deal with notification that a page was read from the cache
|
|
|
|
*/
|
2009-04-03 09:42:41 -06:00
|
|
|
static void afs_file_readpage_read_complete(struct page *page,
|
|
|
|
void *data,
|
|
|
|
int error)
|
2005-04-16 16:20:36 -06:00
|
|
|
{
|
2009-04-03 09:42:41 -06:00
|
|
|
_enter("%p,%p,%d", page, data, error);
|
2005-04-16 16:20:36 -06:00
|
|
|
|
2009-04-03 09:42:41 -06:00
|
|
|
/* if the read completes with an error, we just unlock the page and let
|
|
|
|
* the VM reissue the readpage */
|
|
|
|
if (!error)
|
2005-04-16 16:20:36 -06:00
|
|
|
SetPageUptodate(page);
|
|
|
|
unlock_page(page);
|
2007-04-26 16:49:28 -06:00
|
|
|
}
|
2009-04-17 05:56:38 -06:00
|
|
|
#endif
|
2005-04-16 16:20:36 -06:00
|
|
|
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 13:51:51 -06:00
|
|
|
static void afs_fetch_data_success(struct afs_operation *op)
|
|
|
|
{
|
|
|
|
struct afs_vnode *vnode = op->file[0].vnode;
|
|
|
|
|
|
|
|
_enter("op=%08x", op->debug_id);
|
|
|
|
afs_vnode_commit_status(op, &op->file[0]);
|
|
|
|
afs_stat_v(vnode, n_fetches);
|
|
|
|
atomic_long_add(op->fetch.req->actual_len, &op->net->n_fetch_bytes);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void afs_fetch_data_put(struct afs_operation *op)
|
|
|
|
{
|
|
|
|
afs_put_read(op->fetch.req);
|
|
|
|
}
|
|
|
|
|
|
|
|
static const struct afs_operation_ops afs_fetch_data_operation = {
|
|
|
|
.issue_afs_rpc = afs_fs_fetch_data,
|
|
|
|
.issue_yfs_rpc = yfs_fs_fetch_data,
|
|
|
|
.success = afs_fetch_data_success,
|
2020-06-15 17:34:09 -06:00
|
|
|
.aborted = afs_check_for_remote_deletion,
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 13:51:51 -06:00
|
|
|
.put = afs_fetch_data_put,
|
|
|
|
};
|
|
|
|
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 09:27:50 -06:00
|
|
|
/*
|
|
|
|
* Fetch file data from the volume.
|
|
|
|
*/
|
2019-11-21 02:12:17 -07:00
|
|
|
int afs_fetch_data(struct afs_vnode *vnode, struct key *key, struct afs_read *req)
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 09:27:50 -06:00
|
|
|
{
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 13:51:51 -06:00
|
|
|
struct afs_operation *op;
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 09:27:50 -06:00
|
|
|
|
2018-10-19 17:57:57 -06:00
|
|
|
_enter("%s{%llx:%llu.%u},%x,,,",
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 09:27:50 -06:00
|
|
|
vnode->volume->name,
|
|
|
|
vnode->fid.vid,
|
|
|
|
vnode->fid.vnode,
|
|
|
|
vnode->fid.unique,
|
|
|
|
key_serial(key));
|
|
|
|
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 13:51:51 -06:00
|
|
|
op = afs_alloc_operation(key, vnode->volume);
|
|
|
|
if (IS_ERR(op))
|
|
|
|
return PTR_ERR(op);
|
2019-05-09 08:16:10 -06:00
|
|
|
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 13:51:51 -06:00
|
|
|
afs_op_set_vnode(op, 0, vnode);
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 09:27:50 -06:00
|
|
|
|
afs: Build an abstraction around an "operation" concept
Turn the afs_operation struct into the main way that most fileserver
operations are managed. Various things are added to the struct, including
the following:
(1) All the parameters and results of the relevant operations are moved
into it, removing corresponding fields from the afs_call struct.
afs_call gets a pointer to the op.
(2) The target volume is made the main focus of the operation, rather than
the target vnode(s), and a bunch of op->vnode->volume are made
op->volume instead.
(3) Two vnode records are defined (op->file[]) for the vnode(s) involved
in most operations. The vnode record (struct afs_vnode_param)
contains:
- The vnode pointer.
- The fid of the vnode to be included in the parameters or that was
returned in the reply (eg. FS.MakeDir).
- The status and callback information that may be returned in the
reply about the vnode.
- Callback break and data version tracking for detecting
simultaneous third-parth changes.
(4) Pointers to dentries to be updated with new inodes.
(5) An operations table pointer. The table includes pointers to functions
for issuing AFS and YFS-variant RPCs, handling the success and abort
of an operation and handling post-I/O-lock local editing of a
directory.
To make this work, the following function restructuring is made:
(A) The rotation loop that issues calls to fileservers that can be found
in each function that wants to issue an RPC (such as afs_mkdir()) is
extracted out into common code, in a new file called fs_operation.c.
(B) The rotation loops, such as the one in afs_mkdir(), are replaced with
a much smaller piece of code that allocates an operation, sets the
parameters and then calls out to the common code to do the actual
work.
(C) The code for handling the success and failure of an operation are
moved into operation functions (as (5) above) and these are called
from the core code at appropriate times.
(D) The pseudo inode getting stuff used by the dynamic root code is moved
over into dynroot.c.
(E) struct afs_iget_data is absorbed into the operation struct and
afs_iget() expects to be given an op pointer and a vnode record.
(F) Point (E) doesn't work for the root dir of a volume, but we know the
FID in advance (it's always vnode 1, unique 1), so a separate inode
getter, afs_root_iget(), is provided to special-case that.
(G) The inode status init/update functions now also take an op and a vnode
record.
(H) The RPC marshalling functions now, for the most part, just take an
afs_operation struct as their only argument. All the data they need
is held there. The result delivery functions write their answers
there as well.
(I) The call is attached to the operation and then the operation core does
the waiting.
And then the new operation code is, for the moment, made to just initialise
the operation, get the appropriate vnode I/O locks and do the same rotation
loop as before.
This lays the foundation for the following changes in the future:
(*) Overhauling the rotation (again).
(*) Support for asynchronous I/O, where the fileserver rotation must be
done asynchronously also.
Signed-off-by: David Howells <dhowells@redhat.com>
2020-04-10 13:51:51 -06:00
|
|
|
op->fetch.req = afs_get_read(req);
|
|
|
|
op->ops = &afs_fetch_data_operation;
|
|
|
|
return afs_do_sync_operation(op);
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 09:27:50 -06:00
|
|
|
}
|
|
|
|
|
2005-04-16 16:20:36 -06:00
|
|
|
/*
|
2010-05-21 08:27:09 -06:00
|
|
|
* read page from file, directory or symlink, given a key to use
|
2005-04-16 16:20:36 -06:00
|
|
|
*/
|
2010-05-21 08:27:09 -06:00
|
|
|
int afs_page_filler(void *data, struct page *page)
|
2005-04-16 16:20:36 -06:00
|
|
|
{
|
2010-05-21 08:27:09 -06:00
|
|
|
struct inode *inode = page->mapping->host;
|
|
|
|
struct afs_vnode *vnode = AFS_FS_I(inode);
|
2017-01-05 03:38:34 -07:00
|
|
|
struct afs_read *req;
|
2010-05-21 08:27:09 -06:00
|
|
|
struct key *key = data;
|
2005-04-16 16:20:36 -06:00
|
|
|
int ret;
|
|
|
|
|
2007-04-26 16:57:07 -06:00
|
|
|
_enter("{%x},{%lu},{%lu}", key_serial(key), inode->i_ino, page->index);
|
2005-04-16 16:20:36 -06:00
|
|
|
|
2005-05-01 09:59:01 -06:00
|
|
|
BUG_ON(!PageLocked(page));
|
2005-04-16 16:20:36 -06:00
|
|
|
|
|
|
|
ret = -ESTALE;
|
2007-04-26 16:55:03 -06:00
|
|
|
if (test_bit(AFS_VNODE_DELETED, &vnode->flags))
|
2005-04-16 16:20:36 -06:00
|
|
|
goto error;
|
|
|
|
|
|
|
|
/* is it cached? */
|
2009-04-03 09:42:41 -06:00
|
|
|
#ifdef CONFIG_AFS_FSCACHE
|
|
|
|
ret = fscache_read_or_alloc_page(vnode->cache,
|
2005-04-16 16:20:36 -06:00
|
|
|
page,
|
|
|
|
afs_file_readpage_read_complete,
|
|
|
|
NULL,
|
|
|
|
GFP_KERNEL);
|
|
|
|
#else
|
|
|
|
ret = -ENOBUFS;
|
|
|
|
#endif
|
|
|
|
switch (ret) {
|
|
|
|
/* read BIO submitted (page in cache) */
|
|
|
|
case 0:
|
|
|
|
break;
|
|
|
|
|
2009-04-03 09:42:41 -06:00
|
|
|
/* page not yet cached */
|
2005-04-16 16:20:36 -06:00
|
|
|
case -ENODATA:
|
2009-04-03 09:42:41 -06:00
|
|
|
_debug("cache said ENODATA");
|
|
|
|
goto go_on;
|
|
|
|
|
|
|
|
/* page will not be cached */
|
|
|
|
case -ENOBUFS:
|
|
|
|
_debug("cache said ENOBUFS");
|
2019-01-10 14:52:25 -07:00
|
|
|
|
|
|
|
/* fall through */
|
2005-04-16 16:20:36 -06:00
|
|
|
default:
|
2009-04-03 09:42:41 -06:00
|
|
|
go_on:
|
2019-06-20 11:12:17 -06:00
|
|
|
req = kzalloc(struct_size(req, array, 1), GFP_KERNEL);
|
2017-01-05 03:38:34 -07:00
|
|
|
if (!req)
|
|
|
|
goto enomem;
|
|
|
|
|
2017-03-16 10:27:44 -06:00
|
|
|
/* We request a full page. If the page is a partial one at the
|
|
|
|
* end of the file, the server will return a short read and the
|
|
|
|
* unmarshalling code will clear the unfilled space.
|
|
|
|
*/
|
afs: Fix directory handling
AFS directories are structured blobs that are downloaded just like files
and then parsed by the lookup and readdir code and, as such, are currently
handled in the pagecache like any other file, with the entire directory
content being thrown away each time the directory changes.
However, since the blob is a known structure and since the data version
counter on a directory increases by exactly one for each change committed
to that directory, we can actually edit the directory locally rather than
fetching it from the server after each locally-induced change.
What we can't do, though, is mix data from the server and data from the
client since the server is technically at liberty to rearrange or compress
a directory if it sees fit, provided it updates the data version number
when it does so and breaks the callback (ie. sends a notification).
Further, lookup with lookup-ahead, readdir and, when it arrives, local
editing are likely want to scan the whole of a directory.
So directory handling needs to be improved to maintain the coherency of the
directory blob prior to permitting local directory editing.
To this end:
(1) If any directory page gets discarded, invalidate and reread the entire
directory.
(2) If readpage notes that if when it fetches a single page that the
version number has changed, the entire directory is flagged for
invalidation.
(3) Read as much of the directory in one go as we can.
Note that this removes local caching of directories in fscache for the
moment as we can't pass the pages to fscache_read_or_alloc_pages() since
page->lru is in use by the LRU.
Signed-off-by: David Howells <dhowells@redhat.com>
2018-04-06 07:17:25 -06:00
|
|
|
refcount_set(&req->usage, 1);
|
2017-01-05 03:38:34 -07:00
|
|
|
req->pos = (loff_t)page->index << PAGE_SHIFT;
|
2017-03-16 10:27:44 -06:00
|
|
|
req->len = PAGE_SIZE;
|
2017-01-05 03:38:34 -07:00
|
|
|
req->nr_pages = 1;
|
afs: Fix directory handling
AFS directories are structured blobs that are downloaded just like files
and then parsed by the lookup and readdir code and, as such, are currently
handled in the pagecache like any other file, with the entire directory
content being thrown away each time the directory changes.
However, since the blob is a known structure and since the data version
counter on a directory increases by exactly one for each change committed
to that directory, we can actually edit the directory locally rather than
fetching it from the server after each locally-induced change.
What we can't do, though, is mix data from the server and data from the
client since the server is technically at liberty to rearrange or compress
a directory if it sees fit, provided it updates the data version number
when it does so and breaks the callback (ie. sends a notification).
Further, lookup with lookup-ahead, readdir and, when it arrives, local
editing are likely want to scan the whole of a directory.
So directory handling needs to be improved to maintain the coherency of the
directory blob prior to permitting local directory editing.
To this end:
(1) If any directory page gets discarded, invalidate and reread the entire
directory.
(2) If readpage notes that if when it fetches a single page that the
version number has changed, the entire directory is flagged for
invalidation.
(3) Read as much of the directory in one go as we can.
Note that this removes local caching of directories in fscache for the
moment as we can't pass the pages to fscache_read_or_alloc_pages() since
page->lru is in use by the LRU.
Signed-off-by: David Howells <dhowells@redhat.com>
2018-04-06 07:17:25 -06:00
|
|
|
req->pages = req->array;
|
2017-01-05 03:38:34 -07:00
|
|
|
req->pages[0] = page;
|
|
|
|
get_page(page);
|
2005-04-16 16:20:36 -06:00
|
|
|
|
|
|
|
/* read the contents of the file from the server into the
|
|
|
|
* page */
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 09:27:50 -06:00
|
|
|
ret = afs_fetch_data(vnode, key, req);
|
2017-01-05 03:38:34 -07:00
|
|
|
afs_put_read(req);
|
2017-11-02 09:27:52 -06:00
|
|
|
|
2005-04-16 16:20:36 -06:00
|
|
|
if (ret < 0) {
|
2007-04-26 16:55:03 -06:00
|
|
|
if (ret == -ENOENT) {
|
2005-04-16 16:20:36 -06:00
|
|
|
_debug("got NOENT from server"
|
|
|
|
" - marking file deleted and stale");
|
2007-04-26 16:55:03 -06:00
|
|
|
set_bit(AFS_VNODE_DELETED, &vnode->flags);
|
2005-04-16 16:20:36 -06:00
|
|
|
ret = -ESTALE;
|
|
|
|
}
|
2009-04-03 09:42:41 -06:00
|
|
|
|
|
|
|
#ifdef CONFIG_AFS_FSCACHE
|
|
|
|
fscache_uncache_page(vnode->cache, page);
|
2005-04-16 16:20:36 -06:00
|
|
|
#endif
|
2009-04-03 09:42:41 -06:00
|
|
|
BUG_ON(PageFsCache(page));
|
2017-03-16 10:27:48 -06:00
|
|
|
|
|
|
|
if (ret == -EINTR ||
|
|
|
|
ret == -ENOMEM ||
|
|
|
|
ret == -ERESTARTSYS ||
|
|
|
|
ret == -EAGAIN)
|
|
|
|
goto error;
|
|
|
|
goto io_error;
|
2005-04-16 16:20:36 -06:00
|
|
|
}
|
|
|
|
|
|
|
|
SetPageUptodate(page);
|
|
|
|
|
2009-04-03 09:42:41 -06:00
|
|
|
/* send the page to the cache */
|
|
|
|
#ifdef CONFIG_AFS_FSCACHE
|
|
|
|
if (PageFsCache(page) &&
|
2018-04-04 06:41:28 -06:00
|
|
|
fscache_write_page(vnode->cache, page, vnode->status.size,
|
|
|
|
GFP_KERNEL) != 0) {
|
2009-04-03 09:42:41 -06:00
|
|
|
fscache_uncache_page(vnode->cache, page);
|
|
|
|
BUG_ON(PageFsCache(page));
|
2005-04-16 16:20:36 -06:00
|
|
|
}
|
|
|
|
#endif
|
2009-04-03 09:42:41 -06:00
|
|
|
unlock_page(page);
|
2005-04-16 16:20:36 -06:00
|
|
|
}
|
|
|
|
|
|
|
|
_leave(" = 0");
|
|
|
|
return 0;
|
|
|
|
|
2017-03-16 10:27:48 -06:00
|
|
|
io_error:
|
|
|
|
SetPageError(page);
|
|
|
|
goto error;
|
2017-01-05 03:38:34 -07:00
|
|
|
enomem:
|
|
|
|
ret = -ENOMEM;
|
2007-04-26 16:55:03 -06:00
|
|
|
error:
|
2005-04-16 16:20:36 -06:00
|
|
|
unlock_page(page);
|
|
|
|
_leave(" = %d", ret);
|
|
|
|
return ret;
|
2007-04-26 16:49:28 -06:00
|
|
|
}
|
2005-04-16 16:20:36 -06:00
|
|
|
|
2010-05-21 08:27:09 -06:00
|
|
|
/*
|
|
|
|
* read page from file, directory or symlink, given a file to nominate the key
|
|
|
|
* to be used
|
|
|
|
*/
|
|
|
|
static int afs_readpage(struct file *file, struct page *page)
|
|
|
|
{
|
|
|
|
struct key *key;
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
if (file) {
|
2017-11-02 09:27:52 -06:00
|
|
|
key = afs_file_key(file);
|
2010-05-21 08:27:09 -06:00
|
|
|
ASSERT(key != NULL);
|
|
|
|
ret = afs_page_filler(key, page);
|
|
|
|
} else {
|
|
|
|
struct inode *inode = page->mapping->host;
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 09:27:50 -06:00
|
|
|
key = afs_request_key(AFS_FS_S(inode->i_sb)->cell);
|
2010-05-21 08:27:09 -06:00
|
|
|
if (IS_ERR(key)) {
|
|
|
|
ret = PTR_ERR(key);
|
|
|
|
} else {
|
|
|
|
ret = afs_page_filler(key, page);
|
|
|
|
key_put(key);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2017-01-05 03:38:35 -07:00
|
|
|
/*
|
|
|
|
* Make pages available as they're filled.
|
|
|
|
*/
|
2019-05-09 08:16:10 -06:00
|
|
|
static void afs_readpages_page_done(struct afs_read *req)
|
2017-01-05 03:38:35 -07:00
|
|
|
{
|
2017-01-13 07:46:19 -07:00
|
|
|
#ifdef CONFIG_AFS_FSCACHE
|
2019-05-09 08:16:10 -06:00
|
|
|
struct afs_vnode *vnode = req->vnode;
|
2017-01-13 07:46:19 -07:00
|
|
|
#endif
|
2017-01-05 03:38:35 -07:00
|
|
|
struct page *page = req->pages[req->index];
|
|
|
|
|
|
|
|
req->pages[req->index] = NULL;
|
|
|
|
SetPageUptodate(page);
|
|
|
|
|
|
|
|
/* send the page to the cache */
|
|
|
|
#ifdef CONFIG_AFS_FSCACHE
|
|
|
|
if (PageFsCache(page) &&
|
2018-04-04 06:41:28 -06:00
|
|
|
fscache_write_page(vnode->cache, page, vnode->status.size,
|
|
|
|
GFP_KERNEL) != 0) {
|
2017-01-05 03:38:35 -07:00
|
|
|
fscache_uncache_page(vnode->cache, page);
|
|
|
|
BUG_ON(PageFsCache(page));
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
unlock_page(page);
|
|
|
|
put_page(page);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Read a contiguous set of pages.
|
|
|
|
*/
|
|
|
|
static int afs_readpages_one(struct file *file, struct address_space *mapping,
|
|
|
|
struct list_head *pages)
|
|
|
|
{
|
|
|
|
struct afs_vnode *vnode = AFS_FS_I(mapping->host);
|
|
|
|
struct afs_read *req;
|
|
|
|
struct list_head *p;
|
|
|
|
struct page *first, *page;
|
2017-11-02 09:27:52 -06:00
|
|
|
struct key *key = afs_file_key(file);
|
2017-01-05 03:38:35 -07:00
|
|
|
pgoff_t index;
|
|
|
|
int ret, n, i;
|
|
|
|
|
|
|
|
/* Count the number of contiguous pages at the front of the list. Note
|
|
|
|
* that the list goes prev-wards rather than next-wards.
|
|
|
|
*/
|
2019-01-03 16:29:02 -07:00
|
|
|
first = lru_to_page(pages);
|
2017-01-05 03:38:35 -07:00
|
|
|
index = first->index + 1;
|
|
|
|
n = 1;
|
|
|
|
for (p = first->lru.prev; p != pages; p = p->prev) {
|
|
|
|
page = list_entry(p, struct page, lru);
|
|
|
|
if (page->index != index)
|
|
|
|
break;
|
|
|
|
index++;
|
|
|
|
n++;
|
|
|
|
}
|
|
|
|
|
2019-06-20 11:12:17 -06:00
|
|
|
req = kzalloc(struct_size(req, array, n), GFP_NOFS);
|
2017-01-05 03:38:35 -07:00
|
|
|
if (!req)
|
|
|
|
return -ENOMEM;
|
|
|
|
|
afs: Fix directory handling
AFS directories are structured blobs that are downloaded just like files
and then parsed by the lookup and readdir code and, as such, are currently
handled in the pagecache like any other file, with the entire directory
content being thrown away each time the directory changes.
However, since the blob is a known structure and since the data version
counter on a directory increases by exactly one for each change committed
to that directory, we can actually edit the directory locally rather than
fetching it from the server after each locally-induced change.
What we can't do, though, is mix data from the server and data from the
client since the server is technically at liberty to rearrange or compress
a directory if it sees fit, provided it updates the data version number
when it does so and breaks the callback (ie. sends a notification).
Further, lookup with lookup-ahead, readdir and, when it arrives, local
editing are likely want to scan the whole of a directory.
So directory handling needs to be improved to maintain the coherency of the
directory blob prior to permitting local directory editing.
To this end:
(1) If any directory page gets discarded, invalidate and reread the entire
directory.
(2) If readpage notes that if when it fetches a single page that the
version number has changed, the entire directory is flagged for
invalidation.
(3) Read as much of the directory in one go as we can.
Note that this removes local caching of directories in fscache for the
moment as we can't pass the pages to fscache_read_or_alloc_pages() since
page->lru is in use by the LRU.
Signed-off-by: David Howells <dhowells@redhat.com>
2018-04-06 07:17:25 -06:00
|
|
|
refcount_set(&req->usage, 1);
|
2019-05-09 08:16:10 -06:00
|
|
|
req->vnode = vnode;
|
2017-01-05 03:38:35 -07:00
|
|
|
req->page_done = afs_readpages_page_done;
|
|
|
|
req->pos = first->index;
|
|
|
|
req->pos <<= PAGE_SHIFT;
|
afs: Fix directory handling
AFS directories are structured blobs that are downloaded just like files
and then parsed by the lookup and readdir code and, as such, are currently
handled in the pagecache like any other file, with the entire directory
content being thrown away each time the directory changes.
However, since the blob is a known structure and since the data version
counter on a directory increases by exactly one for each change committed
to that directory, we can actually edit the directory locally rather than
fetching it from the server after each locally-induced change.
What we can't do, though, is mix data from the server and data from the
client since the server is technically at liberty to rearrange or compress
a directory if it sees fit, provided it updates the data version number
when it does so and breaks the callback (ie. sends a notification).
Further, lookup with lookup-ahead, readdir and, when it arrives, local
editing are likely want to scan the whole of a directory.
So directory handling needs to be improved to maintain the coherency of the
directory blob prior to permitting local directory editing.
To this end:
(1) If any directory page gets discarded, invalidate and reread the entire
directory.
(2) If readpage notes that if when it fetches a single page that the
version number has changed, the entire directory is flagged for
invalidation.
(3) Read as much of the directory in one go as we can.
Note that this removes local caching of directories in fscache for the
moment as we can't pass the pages to fscache_read_or_alloc_pages() since
page->lru is in use by the LRU.
Signed-off-by: David Howells <dhowells@redhat.com>
2018-04-06 07:17:25 -06:00
|
|
|
req->pages = req->array;
|
2017-01-05 03:38:35 -07:00
|
|
|
|
|
|
|
/* Transfer the pages to the request. We add them in until one fails
|
|
|
|
* to add to the LRU and then we stop (as that'll make a hole in the
|
|
|
|
* contiguous run.
|
|
|
|
*
|
|
|
|
* Note that it's possible for the file size to change whilst we're
|
|
|
|
* doing this, but we rely on the server returning less than we asked
|
|
|
|
* for if the file shrank. We also rely on this to deal with a partial
|
|
|
|
* page at the end of the file.
|
|
|
|
*/
|
|
|
|
do {
|
2019-01-03 16:29:02 -07:00
|
|
|
page = lru_to_page(pages);
|
2017-01-05 03:38:35 -07:00
|
|
|
list_del(&page->lru);
|
|
|
|
index = page->index;
|
|
|
|
if (add_to_page_cache_lru(page, mapping, index,
|
|
|
|
readahead_gfp_mask(mapping))) {
|
|
|
|
#ifdef CONFIG_AFS_FSCACHE
|
|
|
|
fscache_uncache_page(vnode->cache, page);
|
|
|
|
#endif
|
|
|
|
put_page(page);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
|
|
|
|
req->pages[req->nr_pages++] = page;
|
|
|
|
req->len += PAGE_SIZE;
|
|
|
|
} while (req->nr_pages < n);
|
|
|
|
|
|
|
|
if (req->nr_pages == 0) {
|
|
|
|
kfree(req);
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
afs: Overhaul volume and server record caching and fileserver rotation
The current code assumes that volumes and servers are per-cell and are
never shared, but this is not enforced, and, indeed, public cells do exist
that are aliases of each other. Further, an organisation can, say, set up
a public cell and a private cell with overlapping, but not identical, sets
of servers. The difference is purely in the database attached to the VL
servers.
The current code will malfunction if it sees a server in two cells as it
assumes global address -> server record mappings and that each server is in
just one cell.
Further, each server may have multiple addresses - and may have addresses
of different families (IPv4 and IPv6, say).
To this end, the following structural changes are made:
(1) Server record management is overhauled:
(a) Server records are made independent of cell. The namespace keeps
track of them, volume records have lists of them and each vnode
has a server on which its callback interest currently resides.
(b) The cell record no longer keeps a list of servers known to be in
that cell.
(c) The server records are now kept in a flat list because there's no
single address to sort on.
(d) Server records are now keyed by their UUID within the namespace.
(e) The addresses for a server are obtained with the VL.GetAddrsU
rather than with VL.GetEntryByName, using the server's UUID as a
parameter.
(f) Cached server records are garbage collected after a period of
non-use and are counted out of existence before purging is allowed
to complete. This protects the work functions against rmmod.
(g) The servers list is now in /proc/fs/afs/servers.
(2) Volume record management is overhauled:
(a) An RCU-replaceable server list is introduced. This tracks both
servers and their coresponding callback interests.
(b) The superblock is now keyed on cell record and numeric volume ID.
(c) The volume record is now tied to the superblock which mounts it,
and is activated when mounted and deactivated when unmounted.
This makes it easier to handle the cache cookie without causing a
double-use in fscache.
(d) The volume record is loaded from the VLDB using VL.GetEntryByNameU
to get the server UUID list.
(e) The volume name is updated if it is seen to have changed when the
volume is updated (the update is keyed on the volume ID).
(3) The vlocation record is got rid of and VLDB records are no longer
cached. Sufficient information is stored in the volume record, though
an update to a volume record is now no longer shared between related
volumes (volumes come in bundles of three: R/W, R/O and backup).
and the following procedural changes are made:
(1) The fileserver cursor introduced previously is now fleshed out and
used to iterate over fileservers and their addresses.
(2) Volume status is checked during iteration, and the server list is
replaced if a change is detected.
(3) Server status is checked during iteration, and the address list is
replaced if a change is detected.
(4) The abort code is saved into the address list cursor and -ECONNABORTED
returned in afs_make_call() if a remote abort happened rather than
translating the abort into an error message. This allows actions to
be taken depending on the abort code more easily.
(a) If a VMOVED abort is seen then this is handled by rechecking the
volume and restarting the iteration.
(b) If a VBUSY, VRESTARTING or VSALVAGING abort is seen then this is
handled by sleeping for a short period and retrying and/or trying
other servers that might serve that volume. A message is also
displayed once until the condition has cleared.
(c) If a VOFFLINE abort is seen, then this is handled as VBUSY for the
moment.
(d) If a VNOVOL abort is seen, the volume is rechecked in the VLDB to
see if it has been deleted; if not, the fileserver is probably
indicating that the volume couldn't be attached and needs
salvaging.
(e) If statfs() sees one of these aborts, it does not sleep, but
rather returns an error, so as not to block the umount program.
(5) The fileserver iteration functions in vnode.c are now merged into
their callers and more heavily macroised around the cursor. vnode.c
is removed.
(6) Operations on a particular vnode are serialised on that vnode because
the server will lock that vnode whilst it operates on it, so a second
op sent will just have to wait.
(7) Fileservers are probed with FS.GetCapabilities before being used.
This is where service upgrade will be done.
(8) A callback interest on a fileserver is set up before an FS operation
is performed and passed through to afs_make_call() so that it can be
set on the vnode if the operation returns a callback. The callback
interest is passed through to afs_iget() also so that it can be set
there too.
In general, record updating is done on an as-needed basis when we try to
access servers, volumes or vnodes rather than offloading it to work items
and special threads.
Notes:
(1) Pre AFS-3.4 servers are no longer supported, though this can be added
back if necessary (AFS-3.4 was released in 1998).
(2) VBUSY is retried forever for the moment at intervals of 1s.
(3) /proc/fs/afs/<cell>/servers no longer exists.
Signed-off-by: David Howells <dhowells@redhat.com>
2017-11-02 09:27:50 -06:00
|
|
|
ret = afs_fetch_data(vnode, key, req);
|
2017-01-05 03:38:35 -07:00
|
|
|
if (ret < 0)
|
|
|
|
goto error;
|
|
|
|
|
|
|
|
task_io_account_read(PAGE_SIZE * req->nr_pages);
|
|
|
|
afs_put_read(req);
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
error:
|
|
|
|
if (ret == -ENOENT) {
|
|
|
|
_debug("got NOENT from server"
|
|
|
|
" - marking file deleted and stale");
|
|
|
|
set_bit(AFS_VNODE_DELETED, &vnode->flags);
|
|
|
|
ret = -ESTALE;
|
|
|
|
}
|
|
|
|
|
|
|
|
for (i = 0; i < req->nr_pages; i++) {
|
|
|
|
page = req->pages[i];
|
|
|
|
if (page) {
|
|
|
|
#ifdef CONFIG_AFS_FSCACHE
|
|
|
|
fscache_uncache_page(vnode->cache, page);
|
|
|
|
#endif
|
|
|
|
SetPageError(page);
|
|
|
|
unlock_page(page);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
afs_put_read(req);
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2005-04-16 16:20:36 -06:00
|
|
|
/*
|
2009-04-03 09:42:41 -06:00
|
|
|
* read a set of pages
|
2005-04-16 16:20:36 -06:00
|
|
|
*/
|
2009-04-03 09:42:41 -06:00
|
|
|
static int afs_readpages(struct file *file, struct address_space *mapping,
|
|
|
|
struct list_head *pages, unsigned nr_pages)
|
2005-04-16 16:20:36 -06:00
|
|
|
{
|
2017-11-02 09:27:52 -06:00
|
|
|
struct key *key = afs_file_key(file);
|
2009-04-03 09:42:41 -06:00
|
|
|
struct afs_vnode *vnode;
|
|
|
|
int ret = 0;
|
2005-04-16 16:20:36 -06:00
|
|
|
|
2010-05-21 08:27:09 -06:00
|
|
|
_enter("{%d},{%lu},,%d",
|
|
|
|
key_serial(key), mapping->host->i_ino, nr_pages);
|
|
|
|
|
|
|
|
ASSERT(key != NULL);
|
2005-04-16 16:20:36 -06:00
|
|
|
|
2009-04-03 09:42:41 -06:00
|
|
|
vnode = AFS_FS_I(mapping->host);
|
2012-03-20 10:58:06 -06:00
|
|
|
if (test_bit(AFS_VNODE_DELETED, &vnode->flags)) {
|
2009-04-03 09:42:41 -06:00
|
|
|
_leave(" = -ESTALE");
|
|
|
|
return -ESTALE;
|
|
|
|
}
|
2005-04-16 16:20:36 -06:00
|
|
|
|
2009-04-03 09:42:41 -06:00
|
|
|
/* attempt to read as many of the pages as possible */
|
|
|
|
#ifdef CONFIG_AFS_FSCACHE
|
|
|
|
ret = fscache_read_or_alloc_pages(vnode->cache,
|
|
|
|
mapping,
|
|
|
|
pages,
|
|
|
|
&nr_pages,
|
|
|
|
afs_file_readpage_read_complete,
|
|
|
|
NULL,
|
|
|
|
mapping_gfp_mask(mapping));
|
|
|
|
#else
|
|
|
|
ret = -ENOBUFS;
|
|
|
|
#endif
|
|
|
|
|
|
|
|
switch (ret) {
|
|
|
|
/* all pages are being read from the cache */
|
|
|
|
case 0:
|
|
|
|
BUG_ON(!list_empty(pages));
|
|
|
|
BUG_ON(nr_pages != 0);
|
|
|
|
_leave(" = 0 [reading all]");
|
|
|
|
return 0;
|
|
|
|
|
|
|
|
/* there were pages that couldn't be read from the cache */
|
|
|
|
case -ENODATA:
|
|
|
|
case -ENOBUFS:
|
|
|
|
break;
|
|
|
|
|
|
|
|
/* other error */
|
|
|
|
default:
|
|
|
|
_leave(" = %d", ret);
|
|
|
|
return ret;
|
2005-04-16 16:20:36 -06:00
|
|
|
}
|
|
|
|
|
2017-01-05 03:38:35 -07:00
|
|
|
while (!list_empty(pages)) {
|
|
|
|
ret = afs_readpages_one(file, mapping, pages);
|
|
|
|
if (ret < 0)
|
|
|
|
break;
|
|
|
|
}
|
2009-04-03 09:42:41 -06:00
|
|
|
|
|
|
|
_leave(" = %d [netting]", ret);
|
|
|
|
return ret;
|
2007-04-26 16:49:28 -06:00
|
|
|
}
|
2005-04-16 16:20:36 -06:00
|
|
|
|
|
|
|
/*
|
2009-04-03 09:42:41 -06:00
|
|
|
* invalidate part or all of a page
|
|
|
|
* - release a page and clean up its private data if offset is 0 (indicating
|
|
|
|
* the entire page)
|
|
|
|
*/
|
2013-05-21 21:17:23 -06:00
|
|
|
static void afs_invalidatepage(struct page *page, unsigned int offset,
|
|
|
|
unsigned int length)
|
2009-04-03 09:42:41 -06:00
|
|
|
{
|
2017-11-02 09:27:53 -06:00
|
|
|
struct afs_vnode *vnode = AFS_FS_I(page->mapping->host);
|
|
|
|
unsigned long priv;
|
|
|
|
|
2013-05-21 21:17:23 -06:00
|
|
|
_enter("{%lu},%u,%u", page->index, offset, length);
|
2009-04-03 09:42:41 -06:00
|
|
|
|
|
|
|
BUG_ON(!PageLocked(page));
|
|
|
|
|
|
|
|
/* we clean up only if the entire page is being invalidated */
|
mm, fs: get rid of PAGE_CACHE_* and page_cache_{get,release} macros
PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} macros were introduced *long* time
ago with promise that one day it will be possible to implement page
cache with bigger chunks than PAGE_SIZE.
This promise never materialized. And unlikely will.
We have many places where PAGE_CACHE_SIZE assumed to be equal to
PAGE_SIZE. And it's constant source of confusion on whether
PAGE_CACHE_* or PAGE_* constant should be used in a particular case,
especially on the border between fs and mm.
Global switching to PAGE_CACHE_SIZE != PAGE_SIZE would cause to much
breakage to be doable.
Let's stop pretending that pages in page cache are special. They are
not.
The changes are pretty straight-forward:
- <foo> << (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
- <foo> >> (PAGE_CACHE_SHIFT - PAGE_SHIFT) -> <foo>;
- PAGE_CACHE_{SIZE,SHIFT,MASK,ALIGN} -> PAGE_{SIZE,SHIFT,MASK,ALIGN};
- page_cache_get() -> get_page();
- page_cache_release() -> put_page();
This patch contains automated changes generated with coccinelle using
script below. For some reason, coccinelle doesn't patch header files.
I've called spatch for them manually.
The only adjustment after coccinelle is revert of changes to
PAGE_CAHCE_ALIGN definition: we are going to drop it later.
There are few places in the code where coccinelle didn't reach. I'll
fix them manually in a separate patch. Comments and documentation also
will be addressed with the separate patch.
virtual patch
@@
expression E;
@@
- E << (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E
@@
expression E;
@@
- E >> (PAGE_CACHE_SHIFT - PAGE_SHIFT)
+ E
@@
@@
- PAGE_CACHE_SHIFT
+ PAGE_SHIFT
@@
@@
- PAGE_CACHE_SIZE
+ PAGE_SIZE
@@
@@
- PAGE_CACHE_MASK
+ PAGE_MASK
@@
expression E;
@@
- PAGE_CACHE_ALIGN(E)
+ PAGE_ALIGN(E)
@@
expression E;
@@
- page_cache_get(E)
+ get_page(E)
@@
expression E;
@@
- page_cache_release(E)
+ put_page(E)
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Acked-by: Michal Hocko <mhocko@suse.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-04-01 06:29:47 -06:00
|
|
|
if (offset == 0 && length == PAGE_SIZE) {
|
2009-04-03 09:42:41 -06:00
|
|
|
#ifdef CONFIG_AFS_FSCACHE
|
|
|
|
if (PageFsCache(page)) {
|
|
|
|
struct afs_vnode *vnode = AFS_FS_I(page->mapping->host);
|
|
|
|
fscache_wait_on_page_write(vnode->cache, page);
|
|
|
|
fscache_uncache_page(vnode->cache, page);
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
|
|
|
if (PagePrivate(page)) {
|
2017-11-02 09:27:53 -06:00
|
|
|
priv = page_private(page);
|
|
|
|
trace_afs_page_dirty(vnode, tracepoint_string("inval"),
|
|
|
|
page->index, priv);
|
2017-11-02 09:27:52 -06:00
|
|
|
set_page_private(page, 0);
|
|
|
|
ClearPagePrivate(page);
|
2009-04-03 09:42:41 -06:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
_leave("");
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* release a page and clean up its private state if it's not busy
|
|
|
|
* - return true if the page can now be released, false if not
|
2005-04-16 16:20:36 -06:00
|
|
|
*/
|
2007-05-09 03:33:45 -06:00
|
|
|
static int afs_releasepage(struct page *page, gfp_t gfp_flags)
|
2005-04-16 16:20:36 -06:00
|
|
|
{
|
2007-05-09 03:33:45 -06:00
|
|
|
struct afs_vnode *vnode = AFS_FS_I(page->mapping->host);
|
2017-11-02 09:27:53 -06:00
|
|
|
unsigned long priv;
|
2005-04-16 16:20:36 -06:00
|
|
|
|
2018-10-19 17:57:57 -06:00
|
|
|
_enter("{{%llx:%llu}[%lu],%lx},%x",
|
2007-05-09 03:33:45 -06:00
|
|
|
vnode->fid.vid, vnode->fid.vnode, page->index, page->flags,
|
|
|
|
gfp_flags);
|
2005-04-16 16:20:36 -06:00
|
|
|
|
2009-04-03 09:42:41 -06:00
|
|
|
/* deny if page is being written to the cache and the caller hasn't
|
|
|
|
* elected to wait */
|
|
|
|
#ifdef CONFIG_AFS_FSCACHE
|
FS-Cache: Handle pages pending storage that get evicted under OOM conditions
Handle netfs pages that the vmscan algorithm wants to evict from the pagecache
under OOM conditions, but that are waiting for write to the cache. Under these
conditions, vmscan calls the releasepage() function of the netfs, asking if a
page can be discarded.
The problem is typified by the following trace of a stuck process:
kslowd005 D 0000000000000000 0 4253 2 0x00000080
ffff88001b14f370 0000000000000046 ffff880020d0d000 0000000000000007
0000000000000006 0000000000000001 ffff88001b14ffd8 ffff880020d0d2a8
000000000000ddf0 00000000000118c0 00000000000118c0 ffff880020d0d2a8
Call Trace:
[<ffffffffa00782d8>] __fscache_wait_on_page_write+0x8b/0xa7 [fscache]
[<ffffffff8104c0f1>] ? autoremove_wake_function+0x0/0x34
[<ffffffffa0078240>] ? __fscache_check_page_write+0x63/0x70 [fscache]
[<ffffffffa00b671d>] nfs_fscache_release_page+0x4e/0xc4 [nfs]
[<ffffffffa00927f0>] nfs_release_page+0x3c/0x41 [nfs]
[<ffffffff810885d3>] try_to_release_page+0x32/0x3b
[<ffffffff81093203>] shrink_page_list+0x316/0x4ac
[<ffffffff8109372b>] shrink_inactive_list+0x392/0x67c
[<ffffffff813532fa>] ? __mutex_unlock_slowpath+0x100/0x10b
[<ffffffff81058df0>] ? trace_hardirqs_on_caller+0x10c/0x130
[<ffffffff8135330e>] ? mutex_unlock+0x9/0xb
[<ffffffff81093aa2>] shrink_list+0x8d/0x8f
[<ffffffff81093d1c>] shrink_zone+0x278/0x33c
[<ffffffff81052d6c>] ? ktime_get_ts+0xad/0xba
[<ffffffff81094b13>] try_to_free_pages+0x22e/0x392
[<ffffffff81091e24>] ? isolate_pages_global+0x0/0x212
[<ffffffff8108e743>] __alloc_pages_nodemask+0x3dc/0x5cf
[<ffffffff81089529>] grab_cache_page_write_begin+0x65/0xaa
[<ffffffff8110f8c0>] ext3_write_begin+0x78/0x1eb
[<ffffffff81089ec5>] generic_file_buffered_write+0x109/0x28c
[<ffffffff8103cb69>] ? current_fs_time+0x22/0x29
[<ffffffff8108a509>] __generic_file_aio_write+0x350/0x385
[<ffffffff8108a588>] ? generic_file_aio_write+0x4a/0xae
[<ffffffff8108a59e>] generic_file_aio_write+0x60/0xae
[<ffffffff810b2e82>] do_sync_write+0xe3/0x120
[<ffffffff8104c0f1>] ? autoremove_wake_function+0x0/0x34
[<ffffffff810b18e1>] ? __dentry_open+0x1a5/0x2b8
[<ffffffff810b1a76>] ? dentry_open+0x82/0x89
[<ffffffffa00e693c>] cachefiles_write_page+0x298/0x335 [cachefiles]
[<ffffffffa0077147>] fscache_write_op+0x178/0x2c2 [fscache]
[<ffffffffa0075656>] fscache_op_execute+0x7a/0xd1 [fscache]
[<ffffffff81082093>] slow_work_execute+0x18f/0x2d1
[<ffffffff8108239a>] slow_work_thread+0x1c5/0x308
[<ffffffff8104c0f1>] ? autoremove_wake_function+0x0/0x34
[<ffffffff810821d5>] ? slow_work_thread+0x0/0x308
[<ffffffff8104be91>] kthread+0x7a/0x82
[<ffffffff8100beda>] child_rip+0xa/0x20
[<ffffffff8100b87c>] ? restore_args+0x0/0x30
[<ffffffff8102ef83>] ? tg_shares_up+0x171/0x227
[<ffffffff8104be17>] ? kthread+0x0/0x82
[<ffffffff8100bed0>] ? child_rip+0x0/0x20
In the above backtrace, the following is happening:
(1) A page storage operation is being executed by a slow-work thread
(fscache_write_op()).
(2) FS-Cache farms the operation out to the cache to perform
(cachefiles_write_page()).
(3) CacheFiles is then calling Ext3 to perform the actual write, using Ext3's
standard write (do_sync_write()) under KERNEL_DS directly from the netfs
page.
(4) However, for Ext3 to perform the write, it must allocate some memory, in
particular, it must allocate at least one page cache page into which it
can copy the data from the netfs page.
(5) Under OOM conditions, the memory allocator can't immediately come up with
a page, so it uses vmscan to find something to discard
(try_to_free_pages()).
(6) vmscan finds a clean netfs page it might be able to discard (possibly the
one it's trying to write out).
(7) The netfs is called to throw the page away (nfs_release_page()) - but it's
called with __GFP_WAIT, so the netfs decides to wait for the store to
complete (__fscache_wait_on_page_write()).
(8) This blocks a slow-work processing thread - possibly against itself.
The system ends up stuck because it can't write out any netfs pages to the
cache without allocating more memory.
To avoid this, we make FS-Cache cancel some writes that aren't in the middle of
actually being performed. This means that some data won't make it into the
cache this time. To support this, a new FS-Cache function is added
fscache_maybe_release_page() that replaces what the netfs releasepage()
functions used to do with respect to the cache.
The decisions fscache_maybe_release_page() makes are counted and displayed
through /proc/fs/fscache/stats on a line labelled "VmScan". There are four
counters provided: "nos=N" - pages that weren't pending storage; "gon=N" -
pages that were pending storage when we first looked, but weren't by the time
we got the object lock; "bsy=N" - pages that we ignored as they were actively
being written when we looked; and "can=N" - pages that we cancelled the storage
of.
What I'd really like to do is alter the behaviour of the cancellation
heuristics, depending on how necessary it is to expel pages. If there are
plenty of other pages that aren't waiting to be written to the cache that
could be ejected first, then it would be nice to hold up on immediate
cancellation of cache writes - but I don't see a way of doing that.
Signed-off-by: David Howells <dhowells@redhat.com>
2009-11-19 11:11:35 -07:00
|
|
|
if (!fscache_maybe_release_page(vnode->cache, page, gfp_flags)) {
|
|
|
|
_leave(" = F [cache busy]");
|
|
|
|
return 0;
|
2009-04-03 09:42:41 -06:00
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
2005-04-16 16:20:36 -06:00
|
|
|
if (PagePrivate(page)) {
|
2017-11-02 09:27:53 -06:00
|
|
|
priv = page_private(page);
|
|
|
|
trace_afs_page_dirty(vnode, tracepoint_string("rel"),
|
|
|
|
page->index, priv);
|
2017-11-02 09:27:52 -06:00
|
|
|
set_page_private(page, 0);
|
2005-04-16 16:20:36 -06:00
|
|
|
ClearPagePrivate(page);
|
|
|
|
}
|
|
|
|
|
2009-04-03 09:42:41 -06:00
|
|
|
/* indicate that the page can be released */
|
|
|
|
_leave(" = T");
|
|
|
|
return 1;
|
2007-04-26 16:49:28 -06:00
|
|
|
}
|
2017-11-02 09:27:52 -06:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Handle setting up a memory mapping on an AFS file.
|
|
|
|
*/
|
|
|
|
static int afs_file_mmap(struct file *file, struct vm_area_struct *vma)
|
|
|
|
{
|
|
|
|
int ret;
|
|
|
|
|
|
|
|
ret = generic_file_mmap(file, vma);
|
|
|
|
if (ret == 0)
|
|
|
|
vma->vm_ops = &afs_vm_ops;
|
|
|
|
return ret;
|
|
|
|
}
|