alistair23-linux/kernel/trace
Steven Rostedt (Red Hat) 0fc1b09ff1 tracing: Use temp buffer when filtering events
Filtering of events requires the data to be written to the ring buffer
before it can be decided to filter or not. This is because the parameters of
the filter are based on the result that is written to the ring buffer and
not on the parameters that are passed into the trace functions.

The ftrace ring buffer is optimized for writing into the ring buffer and
committing. The discard procedure used when filtering decides the event
should be discarded is much more heavy weight. Thus, using a temporary
filter when filtering events can speed things up drastically.

Without a temp buffer we have:

 # trace-cmd start -p nop
 # perf stat -r 10 hackbench 50
       0.790706626 seconds time elapsed ( +-  0.71% )

 # trace-cmd start -e all
 # perf stat -r 10 hackbench 50
       1.566904059 seconds time elapsed ( +-  0.27% )

 # trace-cmd start -e all -f 'common_preempt_count==20'
 # perf stat -r 10 hackbench 50
       1.690598511 seconds time elapsed ( +-  0.19% )

 # trace-cmd start -e all -f 'common_preempt_count!=20'
 # perf stat -r 10 hackbench 50
       1.707486364 seconds time elapsed ( +-  0.30% )

The first run above is without any tracing, just to get a based figure.
hackbench takes ~0.79 seconds to run on the system.

The second run enables tracing all events where nothing is filtered. This
increases the time by 100% and hackbench takes 1.57 seconds to run.

The third run filters all events where the preempt count will equal "20"
(this should never happen) thus all events are discarded. This takes 1.69
seconds to run. This is 10% slower than just committing the events!

The last run enables all events and filters where the filter will commit all
events, and this takes 1.70 seconds to run. The filtering overhead is
approximately 10%. Thus, the discard and commit of an event from the ring
buffer may be about the same time.

With this patch, the numbers change:

 # trace-cmd start -p nop
 # perf stat -r 10 hackbench 50
       0.778233033 seconds time elapsed ( +-  0.38% )

 # trace-cmd start -e all
 # perf stat -r 10 hackbench 50
       1.582102692 seconds time elapsed ( +-  0.28% )

 # trace-cmd start -e all -f 'common_preempt_count==20'
 # perf stat -r 10 hackbench 50
       1.309230710 seconds time elapsed ( +-  0.22% )

 # trace-cmd start -e all -f 'common_preempt_count!=20'
 # perf stat -r 10 hackbench 50
       1.786001924 seconds time elapsed ( +-  0.20% )

The first run is again the base with no tracing.

The second run is all tracing with no filtering. It is a little slower, but
that may be well within the noise.

The third run shows that discarding all events only took 1.3 seconds. This
is a speed up of 23%! The discard is much faster than even the commit.

The one downside is shown in the last run. Events that are not discarded by
the filter will take longer to add, this is due to the extra copy of the
event.

Cc: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2016-05-03 17:59:24 -04:00
..
blktrace.c kernel/...: convert pr_warning to pr_warn 2016-03-22 15:36:02 -07:00
bpf_trace.c bpf: prevent kprobe+bpf deadlocks 2016-03-08 15:28:30 -05:00
ftrace.c Nothing major this round. Mostly small clean ups and fixes. 2016-03-24 10:52:25 -07:00
Kconfig tracing: Add 'hist' event trigger command 2016-04-19 12:16:14 -04:00
Makefile tracing: Add 'hist' event trigger command 2016-04-19 12:16:14 -04:00
power-traces.c cpufreq: powernv/tracing: Add powernv_throttle tracepoint 2016-02-05 02:38:02 +01:00
ring_buffer.c ring-buffer: Process commits whenever moving to a new page. 2015-11-25 15:24:05 -05:00
ring_buffer_benchmark.c ring_buffer: Remove unneeded smp_wmb() before wakeup of reader benchmark 2015-11-03 16:19:02 -05:00
rpm-traces.c PM / Runtime: Introduce trace points for tracing rpm_* functions 2011-09-27 22:53:27 +02:00
trace.c tracing: Use temp buffer when filtering events 2016-05-03 17:59:24 -04:00
trace.h tracing: Use temp buffer when filtering events 2016-05-03 17:59:24 -04:00
trace_benchmark.c tracing: Only benchmark the time tracepoints take if tracing is on 2015-11-02 13:34:58 -05:00
trace_benchmark.h tracing: Add tracepoint benchmark tracepoint 2014-05-29 22:49:54 -04:00
trace_branch.c tracing: Remove {start,stop}_branch_trace 2015-10-21 10:10:09 -04:00
trace_clock.c tracing: Export tracing clock functions 2015-05-12 15:56:57 -04:00
trace_entries.h tracing: %pF is only for function pointers 2015-03-25 08:57:22 -04:00
trace_event_perf.c Not much new with tracing for this release. Mostly just clean ups and 2016-01-12 20:04:15 -08:00
trace_events.c tracing: Use temp buffer when filtering events 2016-05-03 17:59:24 -04:00
trace_events_filter.c tracing: Use temp buffer when filtering events 2016-05-03 17:59:24 -04:00
trace_events_filter_test.h tracing/filter: Add startup tests for events filter 2011-08-19 14:35:59 -04:00
trace_events_hist.c tracing: Add check for NULL event field when creating hist field 2016-04-26 09:40:29 -04:00
trace_events_trigger.c tracing: Add support for named triggers 2016-04-19 18:56:00 -04:00
trace_export.c tracing: ftrace_event_is_function() can return boolean 2015-11-02 14:28:05 -05:00
trace_functions.c tracing: Make tracer_flags use the right set_flag callback 2016-03-08 11:19:08 -05:00
trace_functions_graph.c arch, ftrace: for KASAN put hard/soft IRQ entries into separate sections 2016-03-25 16:37:42 -07:00
trace_irqsoff.c tracing: Remove redundant reset per-CPU buff in irqsoff tracer 2016-03-18 16:39:11 -04:00
trace_kdb.c tracing: Move trace_flags from global to a trace_array field 2015-09-30 15:22:55 -04:00
trace_kprobe.c kernel/...: convert pr_warning to pr_warn 2016-03-22 15:36:02 -07:00
trace_mmiotrace.c kernel/...: convert pr_warning to pr_warn 2016-03-22 15:36:02 -07:00
trace_nop.c tracing: Fix typoes in code comment and printk in trace_nop.c 2016-03-08 11:23:57 -05:00
trace_output.c tracing: Record and show NMI state 2016-03-22 18:04:10 -04:00
trace_output.h tracing: Turn seq_print_user_ip() into a static function 2015-09-28 10:16:12 -04:00
trace_printk.c tracing: Fix trace_printk() to print when not using bprintk() 2016-03-22 18:02:40 -04:00
trace_probe.c kernel/...: convert pr_warning to pr_warn 2016-03-22 15:36:02 -07:00
trace_probe.h kernel/trace_probe: is_good_name can be boolean 2015-09-22 13:11:30 -04:00
trace_sched_switch.c sched/core: Fix trace_sched_switch() 2015-10-06 17:08:15 +02:00
trace_sched_wakeup.c Most of the changes are clean ups and small fixes. Some of them have 2015-11-06 13:30:20 -08:00
trace_selftest.c Seems that Peter Zijlstra added a new check that is making old 2014-10-12 07:28:55 -04:00
trace_selftest_dynamic.c ftrace: Add self-tests for multiple function trace users 2011-05-18 19:24:51 -04:00
trace_seq.c tracing: use %*pb[l] to print bitmaps including cpumasks and nodemasks 2015-02-13 21:21:37 -08:00
trace_stack.c tracing, kasan: Silence Kasan warning in check_stack of stack_tracer 2016-02-19 12:36:44 -05:00
trace_stat.c kernel/...: convert pr_warning to pr_warn 2016-03-22 15:36:02 -07:00
trace_stat.h tracing/stat: Add stat_release() callback 2009-07-10 12:14:05 +02:00
trace_syscalls.c Nothing major this round. Mostly small clean ups and fixes. 2016-03-24 10:52:25 -07:00
trace_uprobe.c kernel/...: convert pr_warning to pr_warn 2016-03-22 15:36:02 -07:00
tracing_map.c tracing: Handle tracing_map_alloc_elts() error path correctly 2016-04-26 09:40:30 -04:00
tracing_map.h tracing: Update some tracing_map constants and comments 2016-04-19 12:16:06 -04:00