Jens Axboe c4cf5261f8 bio: skip atomic inc/dec of ->bi_remaining for non-chains ... Struct bio has an atomic ref count for chained bio's, and we use this to know when to end IO on the bio. However, most bio's are not chained, so we don't need to always introduce this atomic operation as part of ending IO. Add a helper to elevate the bi_remaining count, and flag the bio as now actually needing the decrement at end_io time. Rename the field to __bi_remaining to catch any current users of this doing the incrementing manually. For high IOPS workloads, this reduces the overhead of bio_endio() substantially. Tested-by: Robert Elliott <elliott@hp.com> Acked-by: Kent Overstreet <kent.overstreet@gmail.com> Reviewed-by: Jan Kara <jack@suse.cz> Signed-off-by: Jens Axboe <axboe@fb.com>		2015-05-05 13:32:47 -06:00
..
partitions	Merge branch 'for-3.20/core' of git://git.kernel.dk/linux-block	2015-02-12 14:13:23 -08:00
Kconfig	block: Add T10 Protection Information functions	2014-09-27 09:14:59 -06:00
Kconfig.iosched	blkcg: make CONFIG_BLK_CGROUP bool	2012-03-06 21:27:21 +01:00
Makefile	block: Add T10 Protection Information functions	2014-09-27 09:14:59 -06:00
bio-integrity.c	block: fix regression where bio_integrity_process uses wrong bio_vec iterator	2014-12-02 08:15:21 -07:00
bio.c	bio: skip atomic inc/dec of ->bi_remaining for non-chains	2015-05-05 13:32:47 -06:00
blk-cgroup.c	blkcg: remove blkcg->id	2014-09-08 09:55:37 -06:00
blk-cgroup.h	blkcg: remove blkcg->id	2014-09-08 09:55:37 -06:00
blk-core.c	block: allocate request memory local to request queue	2015-03-24 20:00:07 -06:00
blk-exec.c	blk-mq: avoid infinite recursion with the FUA flag	2014-09-22 11:55:19 -06:00
blk-flush.c	blk-mq: support per-distpatch_queue flush machinery	2014-09-25 15:22:45 -06:00
blk-integrity.c	block: Don't merge requests if integrity flags differ	2014-09-27 09:14:57 -06:00
blk-ioc.c	block: Substitute rcu_access_pointer() for rcu_dereference_raw()	2014-02-18 12:21:26 -08:00
blk-iopoll.c	Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip into next	2014-06-03 12:57:53 -07:00
blk-lib.c	block: Quiesce zeroout wrapper	2015-02-05 10:14:54 -07:00
blk-map.c	blk_rq_map_user(): use import_single_range()	2015-04-11 22:27:13 -04:00
blk-merge.c	Fix bug in blk_rq_merge_ok	2015-03-20 08:50:41 -06:00
blk-mq-cpu.c	blk-mq: add file comments and update copyright notices	2014-05-28 10:15:41 -06:00
blk-mq-cpumap.c	blk-mq: Use all available hardware queues	2014-12-09 09:08:21 -07:00
blk-mq-sysfs.c	blk-mq: add blk_mq_init_allocated_queue and export blk_mq_register_disk	2015-03-13 08:26:53 -06:00
blk-mq-tag.c	blkmq: Fix NULL pointer deref when all reserved tags in	2015-03-18 17:06:18 -06:00
blk-mq-tag.h	blk-mq: add tag allocation policy	2015-01-23 14:18:00 -07:00
blk-mq.c	blk-mq: fix iteration of busy bitmap	2015-04-17 08:31:12 -06:00
blk-mq.h	blk-mq: release mq's kobjects in blk_release_queue()	2015-01-29 08:30:51 -08:00
blk-settings.c	block: fix blk_stack_limits() regression due to lcm() change	2015-03-31 09:45:50 -06:00
blk-softirq.c	block: fix regression with block enabled tagging	2014-04-09 21:54:06 -06:00
blk-sysfs.c	blk-mq: release mq's kobjects in blk_release_queue()	2015-01-29 08:30:51 -08:00
blk-tag.c	block: support different tag allocation policy	2015-01-23 14:15:46 -07:00
blk-throttle.c	blk-throttle: check stats_cpu before reading it from sysfs	2015-02-20 22:11:58 -08:00
blk-timeout.c	blk-mq: Allow requests to never expire	2015-01-08 08:59:01 -07:00
blk.h	blk-mq: support per-distpatch_queue flush machinery	2014-09-25 15:22:45 -06:00
bounce.c	mm: convert some level-less printks to pr_*	2014-06-06 16:08:18 -07:00
bsg-lib.c	bsg: Remove unused function bsg_goose_queue()	2012-12-06 14:33:02 +01:00
bsg.c	block: Simplify bsg complete all	2015-02-04 09:57:52 -07:00
cfq-iosched.c	cfq-iosched: handle failure of cfq group allocation	2015-02-09 10:22:39 -07:00
cmdline-parser.c	block: remove unrelated header files and export symbol	2014-01-21 20:18:26 -08:00
compat_ioctl.c	block, bdi: an active gendisk always has a request_queue associated with it	2014-09-08 10:00:35 -06:00
deadline-iosched.c	block: Stop abusing csd.list for fifo_time	2014-02-24 14:46:32 -08:00
elevator.c	block / PM: Replace CONFIG_PM_RUNTIME with CONFIG_PM	2014-12-04 01:00:23 +01:00
genhd.c	genhd: check for int overflow in disk_expand_part_tbl()	2014-11-19 13:09:07 -07:00
ioctl.c	block: Add discard flag to blkdev_issue_zeroout() function	2015-01-21 10:41:46 -07:00
ioprio.c	block: Fix computation of merged request priority	2014-10-31 08:30:43 -06:00
noop-iosched.c	elevator: Fix a race in elevator switching	2013-07-03 13:25:24 +02:00
partition-generic.c	block: Fix dev_t minor allocation lifetime	2014-09-03 15:01:02 -06:00
scsi_ioctl.c	sg_io(): use import_iovec()	2015-04-11 22:27:13 -04:00
t10-pi.c	block: Add T10 Protection Information functions	2014-09-27 09:14:59 -06:00