a051661ca6
Currently, request_queue has one request_list to allocate requests
from regardless of blkcg of the IO being issued. When the unified
request pool is used up, cfq proportional IO limits become meaningless
- whoever grabs the next request being freed wins the race regardless
of the configured weights.
This can be easily demonstrated by creating a blkio cgroup w/ very low
weight, put a program which can issue a lot of random direct IOs there
and running a sequential IO from a different cgroup. As soon as the
request pool is used up, the sequential IO bandwidth crashes.
This patch implements per-blkg request_list. Each blkg has its own
request_list and any IO allocates its request from the matching blkg
making blkcgs completely isolated in terms of request allocation.
* Root blkcg uses the request_list embedded in each request_queue,
which was renamed to @q->root_rl from @q->rq. While making blkcg rl
handling a bit harier, this enables avoiding most overhead for root
blkcg.
* Queue fullness is properly per request_list but bdi isn't blkcg
aware yet, so congestion state currently just follows the root
blkcg. As writeback isn't aware of blkcg yet, this works okay for
async congestion but readahead may get the wrong signals. It's
better than blkcg completely collapsing with shared request_list but
needs to be improved with future changes.
* After this change, each block cgroup gets a full request pool making
resource consumption of each cgroup higher. This makes allowing
non-root users to create cgroups less desirable; however, note that
allowing non-root users to directly manage cgroups is already
severely broken regardless of this patch - each block cgroup
consumes kernel memory and skews IO weight (IO weights are not
hierarchical).
v2: queue-sysfs.txt updated and patch description udpated as suggested
by Vivek.
v3: blk_get_rl() wasn't checking error return from
blkg_lookup_create() and may cause oops on lookup failure. Fix it
by falling back to root_rl on blkg lookup failures. This problem
was spotted by Rakesh Iyer <rni@google.com>.
v4: Updated to accomodate 458f27a982 "block: Avoid missed wakeup in
request waitqueue". blk_drain_queue() now wakes up waiters on all
blkg->rl on the target queue.
Signed-off-by: Tejun Heo <tj@kernel.org>
Acked-by: Vivek Goyal <vgoyal@redhat.com>
Cc: Wu Fengguang <fengguang.wu@intel.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
75 lines
2.8 KiB
Plaintext
75 lines
2.8 KiB
Plaintext
Queue sysfs files
|
|
=================
|
|
|
|
This text file will detail the queue files that are located in the sysfs tree
|
|
for each block device. Note that stacked devices typically do not export
|
|
any settings, since their queue merely functions are a remapping target.
|
|
These files are the ones found in the /sys/block/xxx/queue/ directory.
|
|
|
|
Files denoted with a RO postfix are readonly and the RW postfix means
|
|
read-write.
|
|
|
|
hw_sector_size (RO)
|
|
-------------------
|
|
This is the hardware sector size of the device, in bytes.
|
|
|
|
max_hw_sectors_kb (RO)
|
|
----------------------
|
|
This is the maximum number of kilobytes supported in a single data transfer.
|
|
|
|
max_sectors_kb (RW)
|
|
-------------------
|
|
This is the maximum number of kilobytes that the block layer will allow
|
|
for a filesystem request. Must be smaller than or equal to the maximum
|
|
size allowed by the hardware.
|
|
|
|
nomerges (RW)
|
|
-------------
|
|
This enables the user to disable the lookup logic involved with IO
|
|
merging requests in the block layer. By default (0) all merges are
|
|
enabled. When set to 1 only simple one-hit merges will be tried. When
|
|
set to 2 no merge algorithms will be tried (including one-hit or more
|
|
complex tree/hash lookups).
|
|
|
|
nr_requests (RW)
|
|
----------------
|
|
This controls how many requests may be allocated in the block layer for
|
|
read or write requests. Note that the total allocated number may be twice
|
|
this amount, since it applies only to reads or writes (not the accumulated
|
|
sum).
|
|
|
|
To avoid priority inversion through request starvation, a request
|
|
queue maintains a separate request pool per each cgroup when
|
|
CONFIG_BLK_CGROUP is enabled, and this parameter applies to each such
|
|
per-block-cgroup request pool. IOW, if there are N block cgroups,
|
|
each request queue may have upto N request pools, each independently
|
|
regulated by nr_requests.
|
|
|
|
read_ahead_kb (RW)
|
|
------------------
|
|
Maximum number of kilobytes to read-ahead for filesystems on this block
|
|
device.
|
|
|
|
rq_affinity (RW)
|
|
----------------
|
|
If this option is '1', the block layer will migrate request completions to the
|
|
cpu "group" that originally submitted the request. For some workloads this
|
|
provides a significant reduction in CPU cycles due to caching effects.
|
|
|
|
For storage configurations that need to maximize distribution of completion
|
|
processing setting this option to '2' forces the completion to run on the
|
|
requesting cpu (bypassing the "group" aggregation logic).
|
|
|
|
scheduler (RW)
|
|
--------------
|
|
When read, this file will display the current and available IO schedulers
|
|
for this block device. The currently active IO scheduler will be enclosed
|
|
in [] brackets. Writing an IO scheduler name to this file will switch
|
|
control of this block device to that new IO scheduler. Note that writing
|
|
an IO scheduler name to this file will attempt to load that IO scheduler
|
|
module, if it isn't already present in the system.
|
|
|
|
|
|
|
|
Jens Axboe <jens.axboe@oracle.com>, February 2009
|