User: Password:
Subscribe / Log in / New account

Re: [F.A.Q.] perf ABI backwards and forwards compatibility

From:  Theodore Tso <tytso-AT-MIT.EDU>
To:  Ingo Molnar <>
Subject:  Re: [F.A.Q.] perf ABI backwards and forwards compatibility
Date:  Tue, 8 Nov 2011 05:41:33 -0500
Message-ID:  <>
Cc:  Theodore Tso <>, Pekka Enberg <>, "Frank Ch. Eigler" <>, Vince Weaver <>, Pekka Enberg <>, Anthony Liguori <>, Avi Kivity <>, " list" <>, " List" <>, qemu-devel Developers <>, Alexander Graf <>, Blue Swirl <>, =?iso-8859-1?Q?Am=E9rico_Wang?= <>, Linus Torvalds <>, Peter Zijlstra <>, Arnaldo Carvalho de Melo <>
Archive-link:  Article

On Nov 8, 2011, at 5:22 AM, Ingo Molnar wrote:

> We do even more than that, the perf ABI is fully backwards *and* 
> forwards compatible: you can run older perf on newer ABIs and newer 
> perf on older ABIs.

It's great to hear that!   But in that case, there's an experiment we can't
really run, which is if perf had been developed in a separate tree, would it
have been just as successful?

My belief is that perf was successful because *you* and the other perf
developers were competent developers, and who got things right.   Not because
it was inside the kernel tree.   You've argued that things were much better
because it was inside the tree, but that's not actually something we can put
to a scientific repeatable experiment.

I will observe that some of the things that caused me to be come enraged by
system tap (such as the fact that I simply couldn't even build the damned
thing on a non-Red Hat compilation environment, would not have been solved by
moving Systemtap into the kernel git tree --- at least not without moving a
large number of its external dependencies into the kernel tree as well, such
as the elf library, et. al.)   So there is a whole class of problems that
were seen in previous tooling systems that were not caused by the fact that
they were separate from the kernel, but that they weren't being developed by
the kernel developers, so they didn't understand how to make the tool work
well for kernel developers.

If we had gone back in time, and had the same set of perf developers working
in an external tree, and Systemtap and/or Oprofile had been developed in the
kernel tree, would it really have made that much difference?   Sure, Linus
and other kernel developers would have yelled at the Systemtap and Oprofile
folks more, but I haven't seen that much evidence that they listened to us
when they were outside of the kernel tree, and it's not obvious they would
have listened with the code being inside the kernel tree.

My claim is that is that outcome wouldn't have been all that different, and
that's because the difference was *you*, Ingo Molnar, as a good engineer,
would have designed a good backwards compatible ABI whether the code was
inside or outside of the kernel, and you would have insisted on good taste
and usefulness to kernel programmers whether perf was in our out of the
kernel, and you would have insisted on kernel coding guidelines and regular
release cycles, even if perf was outside of the kernel.   As Linus sometimes
like to say, in many cases it's more about the _people_.


-- Ted

To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to
More majordomo info at

(Log in to post comments)

Copyright © 2011, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds