User: Password:
|
|
Subscribe / Log in / New account

Merging the kvm tool

Merging the kvm tool

Posted Aug 25, 2011 2:27 UTC (Thu) by dberkholz (guest, #23346)
Parent article: Merging the kvm tool

Has anyone considered using the kernel tree like an incubator? Let things live there while the API is "growing up" and then move them out into independent repos once they've matured.


(Log in to post comments)

Merging the kvm tool

Posted Aug 25, 2011 11:09 UTC (Thu) by mingo (subscriber, #31122) [Link]

That's exactly what we've done with tools/perf/ and it has worked very well for us.

Except for the detail that we've indefinitely postponed the 'moving out into a separate project' step:

  • Dependable release schedule: the kernel gives us and our users a perfectly timed, externally enforced stable release heartbeat every 3 months.
  • No monopoly on maintenance: we cannot abandon the repository and kill/sabotage development that way. If we make a mess of tools/perf/ within the kernel repo others will pick it up and Linus will start merging them.
  • Dependable development model: well-understood and constantly evolving coding style and review/discussion methodology
  • Easy access to early adopters: Kernel testers and kernel developers are pretty curious by nature and doing 'cd tools/perf; make -j;' is not something they are scared of trying or lazy to perform, in the kernel repository they already have anyway.
  • Widespread distribution: the Linux kernel is widely distributed and packaged, so it's easy for anyone to get started hacking on tools/perf/.
  • These are IMO some of the most important attributes of a good user-space project and by living in the kernel repository we get these benefits without having to fight much to enforce them. (Where the 'fight' would often be against ourselves.)

    So I am not surprised at all that the tools/kvm/ developers (disclaimer, I write the occasional patch for tools/kvm/ myself and review their patches) are feeling about it in a similar way.

    Merging the kvm tool

    Posted Aug 25, 2011 13:10 UTC (Thu) by dberkholz (guest, #23346) [Link]

    I agree, tools where devs are the primary/only audience might be more fitting for bundling with the kernel for longer periods of time. I don't really see KVM that way, though.

    The problem with not leaving is obvious; there's a never-ending growth of semi-random packages distributed with the kernel, and pretty soon they'll call it LinuxBSD because it includes the whole OS.

    Making it explicit that the kernel tree is an incubator rather than a permanent home means that it's not unbounded growth but instead more of an adolescence for new userspace code, which will eventually move out and go to college instead of sucking your resources forever.

    Merging the kvm tool

    Posted Aug 25, 2011 14:37 UTC (Thu) by deater (subscriber, #11746) [Link]

    One thing not addressed is the stable-ABI issue.

    External tools trying to use the perf_event ABI tend to struggle, as the perf_event developers are primarily interested in perf working. Breaks in the ABI (and there have been many, though most are minor) are not deemed important if you don't catch them fast enough. And since the perf_event developers never feel the need to use the ABI from outside the kernel tree, it ends up meaning that userspace developers need to be running bleeding-edge kernels at all times or risk their tools being broken.

    Kernel and userspace development are very different processes, and if your code isn't in the kernel tree you face an extreme disadvantage.

    Merging the kvm tool

    Posted Aug 25, 2011 14:45 UTC (Thu) by deater (subscriber, #11746) [Link]

    To add an example: the whole OFFCORE_EVENTS issue on Nehalem processors with perf.

    Currently it is impossible for external tools to detect if this feature is enabled or not. If you try to use it, sometimes it will return 0, sometimes not (depending which generalized event you've used due to a buggy leak of MSR state). This bug has been there since 2.6.39. What should happen is an error code returned.

    If RAW OFFCORE_EVENT support is ever added, then there is no possible way for an external tool to detect this properly, as correct behavior is indistinguishable from the current buggy disabling.

    Does the perf tool care? No. Why? Because when it gets support it will happen at the same time that the kernel is updated, so as long as kernel/perf are updated at the same time you'll never notice. perf never has to worry about backward compatability, which is a bit of an unfair advantage.


    Copyright © 2017, Eklektix, Inc.
    Comments and public postings are copyrighted by their creators.
    Linux is a registered trademark of Linus Torvalds