Toward a kernel maintainer's guide
In theory, Williams said, kernel maintenance is a straightforward task. All it takes is accumulating patches and sending a pull request or two to Linus Torvalds during the merge window. In this ideal world, subsystems are the same and there is plenty of backup to provide continuity when a maintainer takes a vacation. In the real world, though, the merge window is a stressful time for maintainers. It involves a lot of work juggling topic branches, a lot of talking to people (which is an annoying distraction), and the fact that Torvalds can instinctively smell a patch that is not yet fully cooked. Maintenance practices vary between subsystems, and there is no backup for the maintainers in many of them. It is hard for a maintainer to take a break.
Kernel maintainers, he said, are a gang of opinionated people. They don't
always agree on things, but the good news is they don't have to. So why
would we want a maintainer's handbook? The idea is to create a reference
manual for both maintainers and contributors, a collection of "tribal
knowledge" and best practices rather than a set of rules. There is a lot
of good advice for maintainers to be found in email discussions, but nobody
has, yet, gone to the effort to capture that information and present it in
a useful form.
Another way of putting it, he said, is that there is a fair amount of pain in the community, and he would like to try to alleviate it. Contributors feel the pain of trying to get a maintainer to do something; maintainers, instead, feel the pain of simply trying to hold everything together. He noted that he, too, is guilty of doing things as a maintainer that have caused him stress as a contributor; it is easy to unintentionally make the process harder for others. By addressing some of those pain points, Williams hopes he can help to create a better experience for all involved.
For example, one painful experience for contributors is getting silence in response to patches sent to a maintainer. Different maintainers exhibit different latencies, so it is hard to know when to press further. One way to address this problem might be for maintainers to advertise an equivalent to a service-level agreement (SLA) documenting the response time they agree to provide. Associated with the SLA could be information like a set of trusted reviewers who could stand in for the maintainer for many review tasks, the location of the subsystem's test suite, and more. By setting the contributor's expectations, the guide should make their life easier; they will know when to resend a patch.
Another part of the guide would concern itself with preventable maintainer mistakes. There have been a lot of lectures on the proper use of Git posted by Torvalds (and others) over the years; it should be collected and put into a place where maintainers can find it before they make a mistake. Torvalds, Williams said, provides great explanations of how things should be done "after the storm passes"; he does so patiently, repeatedly, as the same mistakes are discovered anew. Why, Williams asked, isn't this information written down anywhere?
As Williams looked into the creation of a kernel maintainer's guide, he discovered that one already exists; it was created by Tobin Harding in 2017 and hasn't been changed since. His first objective is to add subsystem profiles to this guide; the profile is meant to tell contributors how to work with the subsystem. It would include information like:
- Whether the subsystem accepts pull requests or, instead, requires that all submissions be posted as patches to a mailing list.
- The last day before the merge window that new features can be posted and the last day that any new features could actually be merged. This "last day" is likely to be expressed in terms like "when -rc5 comes out".
- What the requirements are for Reviewed-by or Acked-by tags on patches and whether the maintainer is allowed to merge unreviewed patches.
- Whether the subsystem has a test suite and where it can be found.
- A list of trusted reviewers for the subsystem.
- The "resend cadence" for the subsystem — how long should a contributor wait before resending a patch?
- The time zone(s) in which the maintainers operate, which would be a hint for when contributors could expect a response to an email.
- The maintainer's opinion on trivial cleanup patches.
- Whether the maintainer trusts off-list patch reviews. These often take the form of a Reviewed-by tag from somebody who works for the same company as the submitter; not all maintainers put much faith in such tags.
If nothing else, this list is an interesting overview of how different kernel maintainers approach their job. It drew the obvious question ("why do we have these differences in the first place?") from the audience, but there isn't really an answer beyond "it has always been that way".
The session wound down with some unfocused discussion on the details of the subsystem profiles. Should they be listed in the MAINTAINERS file? Should they include the maintainer's expectations on the documentation of new features? Answers to those questions will have to await the conclusion of the wider discussion, which is ongoing as of this writing. But, as Mel Gorman noted at the end of the talk, the work that has been done so far is a useful enumeration of the problem space, which is a good start.
[Thanks to the Linux Foundation, LWN's travel sponsor, for supporting my
travel to the event.]
Index entries for this article | |
---|---|
Kernel | Documentation |
Conference | Linux Plumbers Conference/2018 |
Posted Nov 26, 2018 19:59 UTC (Mon)
by l1k (subscriber, #112260)
[Link]
Posted Nov 27, 2018 2:54 UTC (Tue)
by unixbhaskar (guest, #44758)
[Link] (1 responses)
Posted Nov 27, 2018 10:25 UTC (Tue)
by k3ninho (subscriber, #50375)
[Link]
Even if you find a common structure to the workflow, each person will deliver it in their own unique way, from their perspective on the tasks at hand or their understanding of the problem and its solutions to how they work through each task -- the challenge is to share enough information among a team responsible for common success so that people can make progress.
Cat herding teaches us that cats are going to be cats, so let's work with that to communicate and interact in ways which get the best from each kitteh.
K3n.
Posted Nov 27, 2018 7:28 UTC (Tue)
by marcH (subscriber, #57642)
[Link] (3 responses)
Why indeed?
I almost never answer generic questions directly; I just hate repeating myself too much. I update or submit a documentation update instead and answer with a one-line pointer to the update.
https://en.wikipedia.org/wiki/Don%27t_repeat_yourself
> It would include information like:
As a (former) "consumer" of code freshly merged by maintainers, I would recommend adding the following information:
- where is the git repo where maintainer X shares code he or she just merged, on its way up(stream)
These seem to be highly maintainer-dependent.
https://chromium.googlesource.com/chromiumos/docs/+/maste...
> It drew the obvious question ("why do we have these differences in the first place?") from the audience, but there isn't really an answer beyond "it has always been that way".
"awareness" is always the first step and documentation achieves that (and more)
Posted Dec 15, 2018 5:12 UTC (Sat)
by djbw (subscriber, #78104)
[Link] (2 responses)
The details on the git repo and a repo map are a good idea. They also dovetail with the recent addition of a kernel.org patchwork-bot that uses a map of the maintainer's git repo to determine when to mark patches "Accepted" vs "Mainline".
Posted Dec 16, 2018 1:24 UTC (Sun)
by marcH (subscriber, #57642)
[Link] (1 responses)
The point I tried to make: it's barely more effort to wrap an email answer / "upgrade" it to a documentation patch (submitted by... email?). So little more that it pays off as soon as the second time the same question comes again.
Now of course I forgot the whole documentation system has to be implemented in the first place: https://xkcd.com/974/
PS: I'm considering my next Linux distro to be Arch purely based on their... wiki.
Posted Dec 16, 2018 2:06 UTC (Sun)
by djbw (subscriber, #78104)
[Link]
Posted Nov 27, 2018 11:21 UTC (Tue)
by mfuzzey (subscriber, #57966)
[Link] (5 responses)
Having different coding style rules for different parts of the kernel seems to be mostly a pain for everyone involved and I don't see any reasonable justification other than inertia.
On the other hand having somewhat different *processes* in terms of git trees, review rules, cycle time etc for different subsystems makes a lot more sense since maintainers do have different workflows and the "best" workflow is certainly also very dependent on the size of the subsystem (in terms of patch volume and number of developers).
Posted Nov 27, 2018 15:29 UTC (Tue)
by bfields (subscriber, #19510)
[Link] (1 responses)
I can think of a few.
It takes effort to reach consensus: is a flame war on Christmas-tree declaration ordering really such a high priority?
People have more fun if they're given a little leeway.
The original Documentation/CodingStyle was 193 lines. Looks like currently Documentation/process/codingstyle.rst is 1079 lines. I wonder if people read it.
Posted Nov 30, 2018 11:30 UTC (Fri)
by ale2018 (guest, #128727)
[Link]
Posted Nov 27, 2018 16:01 UTC (Tue)
by rweikusat2 (subscriber, #117920)
[Link] (2 responses)
Empirically, this model has worked exceedingly well for a long time, hence, there's no reasonable justification to change it now just because that's unfamiliar to certain people. The only "pain" involved here is if style conventions are being enforced (or sometimes enforced, depending on the social situation) which aren't documented anywhere: There's no way someone who isn't already familiar with whatever petty regulations are considered important can divine knowledge about them. Hence, trainwrecks are guaranteed to occur: Someone's being annoyed that things were done differently despite "everybody knows that THIS is the proper colour for a bikeshed!", someone else finds himself suddenly stepping on hidden landmines despite painstakingly trying to do everything "in the right way".
As the paragraph above probably does (and certainly should) suggest, ideally, there would be no (or at least as few as possible) petty regulations at all. That different conventions about these can successfully coexist suggests that they aren't really necessary.
Posted Nov 27, 2018 16:56 UTC (Tue)
by mfuzzey (subscriber, #57966)
[Link] (1 responses)
Yes, that's what I meant by inertia
>The only "pain" involved here is if style conventions are being enforced (or sometimes enforced, depending on the social situation) which aren't documented anywhere
Obviously if they aren't being enforced there is no pain :)
>As the paragraph above probably does (and certainly should) suggest, ideally, there would be no (or at least as few as possible) petty regulations at all. That different >
Indeed, I completely agree with this. I wasn't suggesting that there should be a great "christmas tree debate" kernel wide. But that a rule should either be considered sufficiently important to be in the global CodingStyle or it should not exist.
Posted Nov 27, 2018 22:59 UTC (Tue)
by rweikusat2 (subscriber, #117920)
[Link]
I don't think that's realistic, especially considering that some maintainers enforce coding style regulations which contradict what's written in the CodingStyle text (Patrick "This function is used only once. Get rid of it" McHardy being an example I personally encountered). OTOH, expecting outside contributors (or prospective outside contributors) to read throuhg a few years of a large number of high-volume mailing lists in order to discover "undocumented preferences" of over 1000 people is also not realistic. That's nothing but a camouflaged and guarded "No entry"-sign.
There must be a way for someone to determine which guidelines to follow, ie, a document listing them. Otherwise, one can only try to imitate how the other code is written and that's not a trusty guide: Some "style regulations" are actually not about style at all, they mandate functional differences, even to the point of requiring that code works in an unlogical way. Eg, some people are strongly convinced that all loops ought to be pre-test loops. I don't think so because the initial state is often known and testing a condition known be either true or false seems absurd to me. Nevertheless, I'd gladly follow such a rule if someone considered it important. But there's no way to infer this from code.
Posted Nov 27, 2018 15:31 UTC (Tue)
by kdave (subscriber, #44472)
[Link] (1 responses)
Posted Nov 27, 2018 19:19 UTC (Tue)
by nilsmeyer (guest, #122604)
[Link]
Posted Nov 28, 2018 21:41 UTC (Wed)
by ncm (guest, #165)
[Link]
The most pervasive and harmful of these is that programmers get confused by seeing more than one style. Appeals to this myth block improvements. In fact it is always easy to tell which of two styles in a file is the new one, and which is legacy. The useful unit of consistency is the equivalent of a paragraph -- a function, or a block in a bigger function.
The second is that any rule is as good as any another -- that all that matters is consistency. Not having objective evidence for one or other is not the same as that no such evidence is possible. Usually it would be way more work to collect than the difference is worth to one project, but there are plenty of projects conducting the experiment. So, sometimes any rule is better than no rule, but then comes time to step to a better rule. Take it, because it will make a difference.
Posted Nov 29, 2018 16:44 UTC (Thu)
by bored (subscriber, #125572)
[Link] (1 responses)
Humans are error prone, and don't generally enforce rules consistently ("do as I say, not as I do"), no amount of subsystem specific style guides/etc is going to change that unless its programmatically enforced.
Posted Nov 29, 2018 17:53 UTC (Thu)
by karkhaz (subscriber, #99844)
[Link]
There are probably a few tiny cases where an automated tool wouldn't suffice or should be overridden, but I don't see why a lot of this work can't be replaced with a very small shell script.
Posted Dec 3, 2018 19:20 UTC (Mon)
by ecree (guest, #95790)
[Link] (4 responses)
Posted Dec 16, 2018 0:05 UTC (Sun)
by marcH (subscriber, #57642)
[Link] (3 responses)
I doubt Dan or anyone else will ever surface a "reverse Christmas tree rule" at the very *top* of any new documentation, it'll be buried somewhere in the middle as it should. So gathering and structuring this sort of documentation will only make it look *more* like a [Van Halen] contract that is finally possible to read entirely and will make complying with some "brown M&Ms clause" easier if any.
Posted Dec 16, 2018 0:47 UTC (Sun)
by excors (subscriber, #95769)
[Link] (2 responses)
Posted Dec 16, 2018 1:12 UTC (Sun)
by marcH (subscriber, #57642)
[Link] (1 responses)
Lame excuse: the previous comment "forgot" a smiley.
Posted Dec 17, 2018 11:02 UTC (Mon)
by sdalley (subscriber, #18550)
[Link]
FWIW, a document that might be called a "subsystem profile" exists for the PCI subsystem, but isn't part of the tree yet, it was only posted to the list so far.
Toward a kernel maintainer's guide
Toward a kernel maintainer's guide
Toward a kernel maintainer's guide
...should always draw the answer that we're adults working together in the varied ways that different people do.
Toward a kernel maintainer's guide
- short "map" of said git repo, notably which git branches are "volatile" versus not (in other words: does using the cherry-pick "-x" option make sense or not)
https://public-inbox.org/git/70ccb8fc-30f2-5b23-a832-9e47...
Toward a kernel maintainer's guide
Toward a kernel maintainer's guide
Cheap shot sorry, I can imagine there's a real effort to get all this started and fair play Dan.
Toward a kernel maintainer's guide
Toward a kernel maintainer's guide
Having different coding style rules for different parts of the kernel seems to be mostly a pain for everyone involved and I don't see any reasonable justification other than inertia.
Toward a kernel maintainer's guide
Toward a kernel maintainer's guide
Toward a kernel maintainer's guide
Toward a kernel maintainer's guide
But if they are enforced, even if documented, it is a pain to those that work in multiple subsystems whereas those that only work in one probably never notice.
>conventions about these can successfully coexist suggests that they aren't really necessary.
Toward a kernel maintainer's guide
> But that a rule should either be considered sufficiently important to be in the global CodingStyle or it should not exist.
Toward a kernel maintainer's guide
Toward a kernel maintainer's guide
Toward a kernel maintainer's guide
Toward a kernel maintainer's guide
Toward a kernel maintainer's guide
Local customs as signalling mechanism
If this theory is correct, then making these "local customs" more publicly visible could undermine their purpose, and drive maintainers to impose ever more obscure requirements in an arms race against the people documenting them.
Local customs as signalling mechanism
Local customs as signalling mechanism
Local customs as signalling mechanism
Local customs as signalling mechanism
[pauses to collect myself]