On the value of static tracepoints
Posted Apr 28, 2009 19:45 UTC (Tue) by compudj
Parent article: On the value of static tracepoints
As an answer to Andrew Morton (which I should have probably posted on LKML rather than here), I would say that one of the primary strength of a system-side kernel tracer is to give Linux users the ability to answer to this simple question : "Why am I not getting the expected performance or latency when I run such application or use such device on my system ?"
The answer to this question is rather easy when parts of the system are _actively_ eating up CPU time (oprofile is very good at system-wide profiling), but becomes less clear when the issue is a "worse-case latency" or involves delays caused by process "waiting time". Having a trace of wakeup dependencies and the identity of each thread consuming CPU time along with scheduler decisions are incredibly valuable in getting an overall view of the system's behavior.
If this has not been expressed clearly enough in the many presentations many of us have done in the past years, then I guess we are simply unable to reach the right audience. A good case study of the static tracepoint value has been presented in this paper 2 years ago. It presents how static tracing has been used to debug problems at Google, IBM and Autodesk.
Linux Kernel Debugging on Google-sized clusters at Ottawa Linux Symposium 2007
I have, in addition, personally been involved with and helped static tracer deployment at Google, IBM, Autodesk, Nokia, Ericsson, Siemens, Novell (SuSE Enterprise real-time), WindRiver, Montavista (Carrier Grade Linux distribution).
And if kernel developers still think that a kernel tracer is only valuable to kernel developers, then we have a big marketing job to do because they are just not getting the message : kernel tracing is _very_ valuable to Linux *users*.
P.S.: I did not reply on this topic on LKML because I think I have done my share of the explanation in the past 4 years, and I would just be repeating myself. *Linux users* have to speak up, not me.
to post comments)