LWN.net Logo

A SystemTap update

A SystemTap update

Posted Jan 29, 2009 16:36 UTC (Thu) by fuhchee (subscriber, #40059)
In reply to: A SystemTap update by jejb
Parent article: A SystemTap update

> We've spent quite a lot of effort explaining the problems with the
> utrace/uprobes dependency

Can you provide some links to discussion about these specifics: ?

> (especially the issues of having to pull the
> process symbol table into the kernel

User-space symbol tables are made available to the systemtap module
only if it is required by the script - if it performs symbolic
address or backtrace type lookups.

> and of having the kernel actually
> execute the compiled code to do the traps

Like in dtrace, instrumentation is run within the kernel because
having user-space processes instrument each other is too disruptive.
We're looking for microsecond-level probe effect, not something
involving multiple context switches, indirect address space accesses,
and so on.


(Log in to post comments)

A SystemTap update

Posted Jan 29, 2009 16:55 UTC (Thu) by jejb (subscriber, #6654) [Link]

>> We've spent quite a lot of effort explaining the problems with the
>> utrace/uprobes dependency
>
> Can you provide some links to discussion about these specifics: ?

Um, just use a search ... if you search lkml for utrace you get the less polite version .. if you search the systemtap lists on the same thing, you get the more polite one.

>> (especially the issues of having to pull the
>> process symbol table into the kernel
>
> User-space symbol tables are made available to the systemtap module
> only if it is required by the script - if it performs symbolic
> address or backtrace type lookups.

Only if you buy the premise that the kernel has to be intimately involved in the trace instead of being a simple conduit for mediating it.

>> and of having the kernel actually
>> execute the compiled code to do the traps
>
> Like in dtrace, instrumentation is run within the kernel because
> having user-space processes instrument each other is too disruptive.
> We're looking for microsecond-level probe effect, not something
> involving multiple context switches, indirect address space accesses,
> and so on.

Well, this would be the classic illustration of the problems systemtap faces. Nothing on the above laundry list is impossible even if the kernel merely controls the traced process and lets userspace poke at it ... that, after all, is how gdb works. The brick wall is that kernel developers don't think this is at all a compelling argument and apparently systemtap people think it is.

A SystemTap update

Posted Jan 29, 2009 17:26 UTC (Thu) by fuhchee (subscriber, #40059) [Link]

> Um, just use a search

I asked because I recall no serious debate about the two specific items ("process symbol tables in the kernel" and "having kernel ... execute code ... to do the traps") you listed. Please humor fellow readers and give some links.

> > User-space symbol tables are made available to the systemtap module
> > only if it is required by the script

> Only if you buy the premise that the kernel has to be intimately involved
> in the trace instead of being a simple conduit for mediating it.

There are many possible details behind such a summary. If one wants dtrace-level introspection and manipulation, never mind going beyond it, some "intimate involvement" (kernel-side processing?) is necessary. Merely "mediating" (data copying?) is not sufficient, since the choice of data and the nature of the programmed reaction is itself variable.

> [...] that, after all, is how gdb works. [...]

The work involved in how gdb does its thing is several orders of magnitude heavier.

> The brick wall is that kernel developers don't think this is at all a
> compelling argument and apparently systemtap people think it is.

Individual kernel people don't need to buy into every argument for systemtap to bloom. We have promoted numerous "dual-use" kernel-side technologies that can stand on their own feet. For example, with utrace, if you believe that user-space instrumentation is plausible, you should support utrace and forthcoming ("froggy" or "ubs"-like) layers on top, for dispatching those events to a hypothetical user-space handler.

The details deserve more in-depth discussion.

A SystemTap update

Posted Feb 3, 2009 22:41 UTC (Tue) by oak (subscriber, #2786) [Link]

> Well, this would be the classic illustration of the problems systemtap
faces. Nothing on the above laundry list is impossible even if the kernel
merely controls the traced process and lets userspace poke at it ... that,
after all, is how gdb works.

As to why to do it in kernel... Doing it from user space is just too slow.
Try e.g. get backtraces to mallocs through ptrace and you notice how
infeasible this is from user-space (at least through the interface ptrace
offers). With the modern desktop apps that use malloc pretty heavily, the
programs become unusable slow (in addition to their usability, also their
functionality may suffer if they use timeouts for responses etc).

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds