|
|
Subscribe / Log in / New account

KS2012: Distributions and upstream

By Michael Kerrisk
September 5, 2012

2012 Kernel Summit

The "distributions and upstream" session of day 1 of the 2012 Kernel Summit focused on a question enunciated by Ted Ts'o: "From an upstream perspective, how can we better help distros?" Responding to that question were two distributor representatives: Ben Hutchings for Debian and Dave Jones for Fedora.

Ben Hutchings asked that, when considering merging a new feature, kernel developers not accept the argument that "this feature is expensive, but that's okay because we'll make it an option". He pointed out that this argument is based on a logical fallacy, since in nearly every case distributions will enable the option, because some users will need it. As an example, Ben mentioned memory cgroups (memcg), which, in their initial release, were rather expensive for performance.

A second point that Ben made was that there are still features that distributions are adding that are not being merged upstream. As an example from last year, he mentioned Android. As a current example, he noted the union mounts feature, which is still not upstream. Inasmuch as keeping features such as these outside of the mainline kernel creates more work for distributions, he would like to see such features more actively merged.

Dave Jones made three points. The first of these was that a lot of Kconfig help texts are "really awful". As a consequence, distribution maintainers have to read the code in order to work out if a feature should be enabled.

Dave's second point is that it would be useful to have an explicit list of regressions at around the -rc3 or -rc4 point in the release cycle. His problem is that regressions often become visible only much later. Finally, Dave noted that Fedora sees a lot of reports from lockdep that no other distributions seem to see. The basic problem underlying both of these points is of course lack of early testing, and at this point Ted Ts'o mused: "can we make it easier for users to run the kernel-of-the-day [in particular, -rc1 and rc2 kernels] and allow them to easily fall back to a stable kernel if it doesn't work out?" There was however no conclusive response in the ensuing discussion.

Returning to the general subject of Kconfig, Matthew Garrett echoed and elaborated on one of points made by Ben Hutchings, noting that Kconfig is important for kernel developers (so that they can strip down a kernel for fast builds). However, because distributors will nearly always enable configuration options (as described above), kernel developers need to ask themselves, "If you don't expect an option to be enabled [by distributors], then why is the option even present?". In passing, Andrea Arcangeli noted one of his pet irritations—one with which most people who have ever built a kernel will be familiar. When running make oldconfig, it is very easy to overstep as one types Enter to accept the default "no" for most options; one suddenly realizes that the answer to an earlier question should have been "yes". At that point of course, there is no way to go back, and one must instead restart from the beginning. (Your editor observes that improving this small problem could be a nice way for a budding kernel hacker to get their hands dirty.)


to post comments

KS2012: Distributions and upstream

Posted Sep 6, 2012 17:22 UTC (Thu) by smurf (subscriber, #17840) [Link] (16 responses)

> "If you don't expect an option to be enabled [by distributors],
> then why is the option even present?"

Because not all kernels are being built by distributors of general-purpose Linux distributions.

Duh.

KS2012: Distributions and upstream

Posted Sep 6, 2012 17:27 UTC (Thu) by mjg59 (subscriber, #23239) [Link] (15 responses)

Why would a general purpose distribution not want to support as many use cases as possible?

KS2012: Distributions and upstream

Posted Sep 6, 2012 17:53 UTC (Thu) by smurf (subscriber, #17840) [Link] (14 responses)

You misunderstand. Not every kernel is built for a GP distro. Therefore the argument that any option that's not useful for distros simply shouldn't be there is fallacious.

KS2012: Distributions and upstream

Posted Sep 6, 2012 18:19 UTC (Thu) by mjg59 (subscriber, #23239) [Link] (13 responses)

You've misunderstood the context. Adding options that disable features when set to N is fine, adding options that enable features when set to Y and then being surprised when distributions enable them isn't.

KS2012: Distributions and upstream

Posted Sep 6, 2012 18:46 UTC (Thu) by hummassa (subscriber, #307) [Link] (12 responses)

> Adding options that disable features when set to N is fine, adding options that enable features when set to Y and then being surprised when distributions enable them isn't.

Why "being surprised" == "not fine"?

KS2012: Distributions and upstream

Posted Sep 6, 2012 18:47 UTC (Thu) by mjg59 (subscriber, #23239) [Link]

Could you rephrase that? I'm not sure what you're asking.

KS2012: Distributions and upstream

Posted Sep 6, 2012 21:17 UTC (Thu) by bronson (subscriber, #4806) [Link] (10 responses)

Because obviously distributions are going to enable those features. Claiming, "I'm shocked, shocked that anyone should enable this useful feature!" == not ok

KS2012: Distributions and upstream

Posted Sep 6, 2012 21:55 UTC (Thu) by dlang (guest, #313) [Link] (9 responses)

The problem is that sometimes it IS expensive to implement some feature.

so you have the choice of either having the feature available as an option, or not having it available at all.

Distros are not supposed to just enable every possible option, they are supposed to be selecting a reasonable set of options. Sometimes this means that they provide multiple kernels for you to choose from. Sometimes this means that to get some specific option that's got a particularly high cost for those that don't care about it, you have to compile your own kernel (and the distros that strongly discourage this really should be slapped, and yes, I am thinking of RHEL)

You also sometimes have options that the distro decides are probably important enough to enough people that they are going to make them be the default, and if you don't want to pay the cost of that feature, you will need to compile a kernel without it (again, if the distro doesn't offer an option)

In recent years Distros have in many cases been going too far in just enabling everything "because someone may want it" without considering, or testing the impact of the options.

upstream could be better in figuring out and communicating what the expected impact of options are, but that doesn't help if the Distros don't pay attention.

KS2012: Distributions and upstream

Posted Sep 6, 2012 21:56 UTC (Thu) by dlang (guest, #313) [Link] (5 responses)

short form

if enabling A has a cost B saying "I want the benefits of A and I'm shocked that I'm having to pay the penalty B" == not good as well.

KS2012: Distributions and upstream

Posted Sep 6, 2012 22:09 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link] (4 responses)

"the distros that strongly discourage this really should be slapped, and yes, I am thinking of RHEL"

You are free to run your custom kernel in RHEL. If there are issues, you might be asked to reproduce it against the distribution provided kernel. Anything else would be a undue burden on support.

KS2012: Distributions and upstream

Posted Sep 6, 2012 22:33 UTC (Thu) by dlang (guest, #313) [Link] (3 responses)

when you are paying as much as RHEL charges for support for hundreds or thousands of servers, taking the attitude of "user our stock everything or you are on your own" is a very proprietary software way of looking at it.

I understand that this is easier for Red Hat to provide support for, but this is a pretty strict straightjacket that defeats many of the benefits of FOSS software.

KS2012: Distributions and upstream

Posted Sep 7, 2012 3:02 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link] (2 responses)

Singling out RHEL seems strange since I don't think SUSE or Canonical or Oracle is any different in this regard. If you are using something compiled on your own, you can't really expect a vendor to support it. Especially for the kernel, considering the enormous number of experimental options, flaky drivers, random third party patches etc, it would be a nightmare.

KS2012: Distributions and upstream

Posted Sep 7, 2012 11:24 UTC (Fri) by nix (subscriber, #2304) [Link]

Quite. Note that if you have some module that you rely on that you need to use even when doing bug reproduction, but that is not included in the distro kernel, you can often take the distro config and kernel source and enable and compile that module, then load the module atop the distro kernel. This will usually work (modulo only those strange cases where enabling a module will also change code in the core kernel). (It's probably best to mention that you did this to the support people you're talking to, just in case it did break something, but I doubt that this approach will rouse too many hackles.)

KS2012: Distributions and upstream

Posted Sep 7, 2012 18:35 UTC (Fri) by dlang (guest, #313) [Link]

RHEL is the leader in the "enterprise Linux" market, and as such, to a large extent their example drives that segment of the market.

That's why I specifically mentioned them, even though Suse and Oracle have the same policies (I don't know about Canonical)

KS2012: Distributions and upstream

Posted Sep 7, 2012 15:01 UTC (Fri) by xav (guest, #18536) [Link] (2 responses)

> Sometimes this means that to get some specific option that's got a particularly high cost for those that don't care about it, you have to compile your own kernel (and the distros that strongly discourage this really should be slapped, and yes, I am thinking of RHEL)

Just wait a bit for your new secureboot server. You'll have no other option left than running the distro-provided kernel.

KS2012: Distributions and upstream

Posted Sep 10, 2012 13:44 UTC (Mon) by vonbrand (subscriber, #4458) [Link] (1 responses)

You misrepresent what folks are trying to do. The work in this area is to enable users a frictionless installation of their choosen distribution, and allow hachkish types to set up secure boot to handle self-compiled kernels without disabling it completely. I do agree that this whole mess is a misguided "security" feature at best, and a thinly veiled attempt from software-vendors/"content"-distributors to stab at controlling our machines at will at worst. In any case, it is being forced down out throats, we have to see how to make the best of it.

KS2012: Distributions and upstream

Posted Sep 10, 2012 13:55 UTC (Mon) by xav (guest, #18536) [Link]

Do you know that you sound more pessimistic than I do ? :)

KS2012: Distributions and upstream

Posted Sep 7, 2012 21:58 UTC (Fri) by dlang (guest, #313) [Link]

you can't both say that you want to keep things out of the kernel if they are expensive and say that you want to have everything that distros include merged upstream.

Frequently the reason something has not been merged upstream is that it isn't going to work well, isn't maintainable yet, or otherwise has some significant drawback to the code.

Also, the question that Matt Asks

> "If you don't expect an option to be enabled [by distributors], then why is the option even present?"

what distributions are you considering?

There are a lot of config options that make sense for OpenWRT that would be horrible for Fefora. The converse is also true.

unionfs

Posted Sep 18, 2012 17:49 UTC (Tue) by henrich (guest, #56730) [Link]

Just a question, now we have aufs, do we still need unionfs?


Copyright © 2012, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds