|
|
Log in / Subscribe / Register

KS2010: Core kernel vision

By Jonathan Corbet
November 2, 2010

2010 Kernel Summit
Neil Brown's session on the notion of a core vision for the Linux kernel started with the idea that Linux has, on the whole, been reasonably successful. There are three reasons for that success, he said: the strengths in Linux's Unix history, the GPL, and the development community. For this talk, he focused mostly on the first of those. According to Thompson and Ritchie, the design of Unix was based on "the full exploitation of a carefully selected set of fertile ideas" (this is an idea that Neil has discussed in this LWN article as well). But, Neil said, this "careful selection" was done by two engineers. Do we have a process in place to scale this selection to 1000 engineers?

Al Viro jumped in with the claim that Neil had missed one important component in the success of Unix: those two engineers had taste. Therein lies the problem for Linux; a lot of the code coming in is tasteless. It's hard to fix; taste is a hard thing to formalize, and trying to educate developers in taste is not even encouraged. Any discussion of taste is seen as unfriendly, no matter how it's done. Paul McKenney pointed out that there was more than taste involved, though; Unix was also full of things like gets(), which clearly are not in good taste. Economics was a big part of why Unix has been so successful for so long.

Mel Gorman suggested that developers should ask themselves if they would submit their patch to any other operating system. Would it also be suitable for one of the BSDs? If the answer is "no," that might be a good indication that taste is lacking. He also said that it's hard to tell developers that their work lacks taste. It's not like a technical problem which can be fixed; something without taste often needs to be rethought from the beginning.

Looking at the design of a patch before its implementation might be one way to ensure better taste. But, as Ted Ts'o pointed out, that tends not to work well in the Linux environment. Developers are strongly tuned to code; they will be suspicious of design documents which appear without an accompanying implementation. When a document is all there is, it's hard to tell if the developers are serious enough to see things through.

Linus said that one thing he has been doing in recent times is pushing back on features which only have one user. Good taste is important, he said, but it's very hard to explain to developers. But if those developers go through the process of making their work applicable to multiple groups of users, they will often learn a lot. One feature which is being held back for this reason currently is Cleancache, which hasn't yet found enough users to be put into the kernel. Proposed new system calls have also been rejected for the same reason. They must have at least two users, though Linus wondered if that should be raised to three.

Al cited the proposed the "open by handle" system call as not showing great taste; it reveals too much information from within filesystems. It could maybe be made acceptable if the resulting file descriptor had the O_NOACCESS flag set, so that it could only be used in system calls like openat(). He indicated that this idea needed further thought, though.

Linus requested that subsystem maintainers get better at saying "no." If he has to reject a patch it is, by his reckoning, a failure of the process. Maintainers especially need to learn how to say "no" to their own code.

One place where he should have said "no," he said, was the fanotify patches. Indeed, he should have rejected dnotify and inotify as well. There comes a point, after two or three failed attempts, where one has to realized that an idea is just broken. So he will never again accept a file notification API; anybody proposing one may get a stronger reaction than he bargained for. Some problems, Linus said, are simply too hard to be properly solved.

Next: A staging process for ABIs.


to post comments

KS2010: Core kernel vision

Posted Nov 2, 2010 12:07 UTC (Tue) by mjthayer (guest, #39183) [Link] (4 responses)

> Indeed, he should have rejected dnotify and inotify as well. There comes a point, after two or three failed attempts, where one has to realized that an idea is just broken.

Does hard to get right necessarily imply broken? And if it is broken, what would the proper fix be? Always poll for file changes? Always require the user to check manually? Or is there just some fundamental insight into the problem missing?

KS2010: Core kernel vision

Posted Nov 2, 2010 12:39 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

The problem is in Unix filesystems themselves. They are too racy to do anything guaranteed to work.

KS2010: Core kernel vision

Posted Nov 2, 2010 12:50 UTC (Tue) by mjthayer (guest, #39183) [Link] (2 responses)

> The problem is in Unix filesystems themselves. They are too racy to do anything guaranteed to work.

What is the race with notification? Or what races are there that can't be fixed by a second notification and no guarantee that no spurious notifications are sent?

KS2010: Core kernel vision

Posted Nov 2, 2010 13:06 UTC (Tue) by mjthayer (guest, #39183) [Link] (1 responses)

>> The problem is in Unix filesystems themselves. They are too racy to do anything guaranteed to work.

> What is the race with notification? Or what races are there that can't be fixed by a second notification and no guarantee that no spurious notifications are sent?

One answer to myself: http://lwn.net/Articles/361566/ points out that inotify doesn't let you exclude changes to files that you caused yourself, so that if you update a file every time it is modified you will get into a loop. Sounds like a somewhat specialised use case, but I'm sure that there is a good reason for it.

KS2010: Core kernel vision

Posted Nov 4, 2010 9:14 UTC (Thu) by liljencrantz (guest, #28458) [Link]

So you do a check in the userspace code.

That sounds very much like how you have to re-call syscalls that were interruped by a signal. Is it a pain that you have to replace every call to close with a loop that breaks if errno != EINTR? Sure. But handling everything transparently is just way too complex, so some complexity is pushed onto the user. Sounds like inotify does the same. I see zero problem with that.

dnotify

Posted Nov 2, 2010 13:21 UTC (Tue) by willy (subscriber, #9762) [Link]

I wrote dnotify and I didn't ask for it to be included!

It actually works pretty well for its sole user (Samba). Unfortunately, because the desktop file manager people wouldn't respond when I asked them for feedback on the API, it turned out to be useless for them, and so we got inotify.

Moral: when a kernel hacker asks you for feedback on an API, take them seriously, or you may be responsible for a giant fuckup.

KS2010: Core kernel vision

Posted Nov 3, 2010 0:36 UTC (Wed) by nix (subscriber, #2304) [Link]

Other OSes have working file notification APIs. Thus, this idea is plainly *not* too hard to be properly solved.

KS2010: Core kernel vision

Posted Nov 4, 2010 1:27 UTC (Thu) by Lennie (subscriber, #49641) [Link]

I think this is a hard problem to solve, doesn't taste come, for a large part, from experience ? You have to mess up first, before you can get taste.

Even with food, you have to know what is out there before you can appreciate that one fine... whatever, before you can say: this is best damn whatever I ever had !


Copyright © 2010, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds