LWN.net Logo

KS2010: ABI status for tracepoints

By Jonathan Corbet
November 2, 2010
2010 Kernel Summit
Whether tracepoints should be a part of the user-space binary interface is a topic which has been covered here a number of times in the past. When the 2010 Kernel Summit took up the topic, many assumed that the result would be a controversial session. Instead, the developers present came rather quickly to a consensus on how the issue should be resolved.

Arjan van de Ven started with a demonstration of a new version of PowerTop which provides all kinds of information on just how systems are using power. It depends heavily on tracepoints for the source information. He wants to make useful tools like this, but he cannot do it if those tracepoints are not seen as part of the ABI. That interface needs to not break, or the tools will not be useful.

The other leader, Steve Rostedt, started by saying that tracepoints had been put into debugfs for a reason. Tracepoints reach deep into the internal state of the kernel; setting them in stone could limit its future development. But he recognized the need for at least some tracepoints to be relatively stable, so he proposed that there should be two types of tracepoints. One, intended mainly for in-field debugging, would have no ABI guarantee. The other set of tracepoints, aimed at "high-level questions," would be stabilized in some way.

Exactly what "stable" means is not entirely clear; the specific format of a tracepoint could change, perhaps, but the relevant fields would remain and the associated format description would tell applications how to find the information the need. There seemed to be a consensus that requiring applications to use the format description is reasonable. Ted Ts'o pointed out, though, that a lot of tools have trouble with complicated formats. So maybe stable tracepoints should only use simpler formats.

Ted also wondered about just how stable tracepoints would be marked as such. We could impose documentation requirements, but those have not always been observed or enforced in the past. They could be documented in the header file somewhere. Or, possibly, stable tracepoints could be moved out of debugfs entirely, probably to a sysfs subdirectory. That last idea proved popular, and is how things are likely to be done.

Linus was receptive to all of these ideas, but he did have one requirement: there are to be no stable tracepoints in drivers or filesystems. His long experience with ioctl() calls tells him that driver developers will never get this right and will not be able to maintain stable tracepoints in the long term. He would also rather not see them in architecture-specific code either. One idea which went over fairly well was to make stable tracepoints depend on a symbol which is not exported to modules. That would prevent stable tracepoints from appearing in any code which can be built as a module. There may also be a requirement that all stable tracepoints be designated in a single, central source file to make it evident when new ones are added.

All of this seems likely to pass; there were no voices raised in disagreement. There will still be some interesting questions, though. Steve showed a tracepoint from the scheduler which, he said, would be a good candidate for stable status - it fires when the scheduler switches from one process to another. Peter Zijlstra objected to some of the contents of that tracepoint; it output information which, he says, does not belong in a stable tracepoint. For example, there is priority information provided, but, if the deadline scheduler is merged, there will be processes with no priorities. So, while stable tracepoints may be in the future, they are still likely to provide plenty to argue about.

Next: The core kernel vision.


(Log in to post comments)

KS2010: ABI status for tracepoints

Posted Nov 2, 2010 12:24 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]

Why not make tracepoints 'semi-stable'?

For example, guarantee that they won't change for at least 2 years after kernel is released. That way, it'll still be possible to remove old tracepoints once they are not necessary.

Besides, tracepoint clients should by design follow kernel development. It's not like the situation with ioctls where we have ancient programs that we still need to support.

KS2010: ABI status for tracepoints

Posted Nov 2, 2010 13:38 UTC (Tue) by ccurtis (guest, #49713) [Link]

It's not like the situation with ioctls where we have ancient programs that we still need to support.

... today. Once tracepoints become 'stable', they'll be integrated into things like advanced IDEs and profilers. Netbeans uses DTrace on Solaris, for example.

KS2010: ABI status for tracepoints

Posted Nov 2, 2010 13:46 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]

So? IDEs are updated fairly frequently (and usually automatically). And anyway, if a tracepoint becomes obsolete its usefulness can actually be negative. They are tied to kernel internals by their design.

With ioctls it's different. Old user-space programs might not be tied to kernel at all.

KS2010: ABI status for tracepoints

Posted Nov 2, 2010 18:50 UTC (Tue) by NAR (subscriber, #1313) [Link]

Maybe tracepoints should be stable for a distribution. E.g. Netbeans for RHEL 6 or SuSE 11 could support a different but still stable set of tracepoints.

KS2010: ABI status for tracepoints

Posted Nov 2, 2010 19:07 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]

That would put a burden on both maintainers of NetBeans and RHEL/SuSE since their release cycles are not likely to be the same.

It's better to just guarantee that for N stable releases tracepoints will remain stable.

KS2010: ABI status for tracepoints

Posted Nov 3, 2010 8:35 UTC (Wed) by NAR (subscriber, #1313) [Link]

Maybe it is time to synchronize their release cycles? It is possibly wise to use the same longer supported kernel (like 2.6.27 or 2.6.32) in longer supported distributions anyway, so the tracepoints would be the same. At this point they just have to pick the same NetBeans version...

KS2010: ABI status for tracepoints

Posted Nov 3, 2010 11:50 UTC (Wed) by zdzichu (subscriber, #17118) [Link]

Exactly. PowerTOP on Solaris was implemented using its tracepoints implementation (DTrace). It was 2.5 years ago.

KS2010: ABI status for tracepoints

Posted Nov 4, 2010 16:14 UTC (Thu) by glikely (subscriber, #39601) [Link]

Linus made it absolutely clear that once something is deemed an ABI, then it is stable. Period. The pain is just too far-reaching on users with old userland to allow ABI churn (and there are many), even assuming a 2 year guarantee. That is why it is so important to be careful about how new ABIs are defined.

Copyright © 2010, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds