redonkable
/
alistair23-linux
1
0
Fork 0
Code Releases Activity
alistair23-linux/net/sched
History
Eric Dumazet 79640a4ca6 net: add additional lock to qdisc to increase throughput 
When many cpus compete for sending frames on a given qdisc, the qdisc
spinlock suffers from very high contention.

The cpu owning __QDISC_STATE_RUNNING bit has same priority to acquire
the lock, and cannot dequeue packets fast enough, since it must wait for
this lock for each dequeued packet.

One solution to this problem is to force all cpus spinning on a second
lock before trying to get the main lock, when/if they see
__QDISC_STATE_RUNNING already set.

The owning cpu then compete with at most one other cpu for the main
lock, allowing for higher dequeueing rate.

Based on a previous patch from Alexander Duyck. I added the heuristic to
avoid the atomic in fast path, and put the new lock far away from the
cache line used by the dequeue worker. Also try to release the busylock
lock as late as possible.

Tests with following script gave a boost from ~50.000 pps to ~600.000
pps on a dual quad core machine (E5450 @3.00GHz), tg3 driver.
(A single netperf flow can reach ~800.000 pps on this platform)

for j in `seq 0 3`; do
  for i in `seq 0 7`; do
    netperf -H 192.168.0.1 -t UDP_STREAM -l 60 -N -T $i -- -m 6 &
  done
done

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Acked-by: Alexander Duyck <alexander.h.duyck@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
 		2010-06-02 05:09:29 -07:00
..
Kconfig cgroups: net_cls as module 2010-03-23 13:06:14 -07:00
Makefile net_sched: add classful multiqueue dummy scheduler 2009-09-06 02:07:05 -07:00
act_api.c net sched: printk message severity 2010-05-17 23:23:12 -07:00
act_gact.c net sched: printk message severity 2010-05-17 23:23:12 -07:00
act_ipt.c net sched: printk message severity 2010-05-17 23:23:12 -07:00
act_mirred.c net sched: printk message severity 2010-05-17 23:23:12 -07:00
act_nat.c tc: propogate errors from tcf_hash_create 2008-11-25 21:13:25 -08:00
act_pedit.c net sched: printk message severity 2010-05-17 23:23:12 -07:00
act_police.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
act_simple.c net sched: printk message severity 2010-05-17 23:23:12 -07:00
act_skbedit.c pkt_sched: skbedit add support for setting mark 2009-10-22 21:56:42 -07:00
cls_api.c Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6 2010-04-11 14:53:53 -07:00
cls_basic.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
cls_cgroup.c cls_cgroup: Store classid in struct sock 2010-05-24 00:12:34 -07:00
cls_flow.c net: Remove unnecessary returns from void function()s 2010-05-17 23:23:14 -07:00
cls_fw.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
cls_route.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
cls_rsvp.c [NET_SCHED]: Remove unnecessary includes 2007-07-10 22:16:41 -07:00
cls_rsvp.h net: Move && and || to end of previous line 2009-11-29 16:55:45 -08:00
cls_rsvp6.c [NET_SCHED]: Remove unnecessary includes 2007-07-10 22:16:41 -07:00
cls_tcindex.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
cls_u32.c net sched: printk message severity 2010-05-17 23:23:12 -07:00
em_cmp.c net: em_cmp.c use unaligned access helpers 2008-09-22 19:20:51 -07:00
em_meta.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
em_nbyte.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
em_text.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
em_u32.c [PKT_SCHED]: annotate cls_u32 2008-03-17 22:46:46 -07:00
ematch.c net sched: printk message severity 2010-05-17 23:23:12 -07:00
sch_api.c net_sched: Fix qdisc_notify() 2010-05-23 23:11:07 -07:00
sch_atm.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
sch_blackhole.c pkt_sched: Add qdisc->ops->peek() implementation. 2008-10-31 00:45:55 -07:00
sch_cbq.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
sch_drr.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
sch_dsmark.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
sch_fifo.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
sch_generic.c net: add additional lock to qdisc to increase throughput 2010-06-02 05:09:29 -07:00
sch_gred.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
sch_hfsc.c net: Remove unnecessary returns from void function()s 2010-05-17 23:23:14 -07:00
sch_htb.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
sch_ingress.c net: Remove unnecessary returns from void function()s 2010-05-17 23:23:14 -07:00
sch_mq.c net: Remove unnecessary returns from void function()s 2010-05-17 23:23:14 -07:00
sch_multiq.c net: Remove unnecessary returns from void function()s 2010-05-17 23:23:14 -07:00
sch_netem.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
sch_prio.c net: Remove unnecessary returns from void function()s 2010-05-17 23:23:14 -07:00
sch_red.c net: Remove unnecessary returns from void function()s 2010-05-17 23:23:14 -07:00
sch_sfq.c net: Fix various endianness glitches 2010-04-20 19:06:52 -07:00
sch_tbf.c tbf: stop wanton destruction of children (v2) 2010-05-17 17:44:35 -07:00
sch_teql.c include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00