Looking forward to 2.7

So what are the crackpots wishing for? Some of the items they want (marked "mandatory features" on the list) are already in the works; these include support for CPU hotplugging, full NTFS support and virtual machine support. Others are somewhat vague, including "complete user quota centralization" and "improve kobject model for security, quota rendering." And some will never happen; there is just not a whole lot of call for features like an in-kernel Gopher server or a /proc implementation of the loadable module tools.

Kernel hackers have far more respect for code (and those who produce it) than they do for list makers. The 2.7 thoughts list may yet inspire somebody to do some hacking, but its influence on the development process is likely to remain small.

A more interesting view into what could happen with 2.7 might be found in a conversation between Linus and Joel Becker of Oracle. The discussion turned to what information was needed from the kernel to perform direct I/O, which lead to this outburst from Linus:

Linus went on to wish an early death upon disk-based databases; he seems to think that all but the largest databases should just be done in-memory.

Direct I/O does bring its share of problems. It is hard to keep the kernel page cache in a coherent condition when I/O operations are allowed to circumvent it; page cache confusion can lead to corrupted data. Getting good performance out of direct I/O is hard unless asynchronous I/O is used as well. Direct I/O can also confuse the disk I/O scheduler by creating request patterns (especially overlapping requests) which don't otherwise happen. In other words, the direct I/O idea is hard to get right for both kernel and user space.

But systems like Oracle do need some of the capabilities that direct I/O provides. They need to be able to move large amounts of data without polluting the page cache with stuff that will not be used. Databases which use shared storage need to be able to force data to be reread from disk when another system has changed it. Large applications also tend to have a better idea of how their access patterns work than the kernel does; they know when a particular block of data will not be used any more. The need for the level of control and performance direct I/O can provide will persist, whether it is a "piece of crap" or not.

Linus seems to understand this need; he would just like to push development toward what he sees as a better interface. Such an interface would work with the page cache, rather than trying to circumvent it. Some of his thoughts, as expressed in this posting, include:

• A mechanism for moving pages between user space and the page cache. An application wishing to do a direct write would then just transfer ownership of the pages containing the data to the kernel, which would put them into the page cache. A simple flush finishes the job.

• A way for an application to tell the kernel that certain pages in the cache are stale and should not be used. This mechanism could also be used to tell the kernel about pages which are no longer needed and can be dropped from the cache. The fadvise() system call already does part of this task.

• The ability to mark I/O on a particular file descriptor (or by a particular process) as being a one-shot affair that should not be cached. This idea was suggested in response to a description of performance problems triggered by the PostgreSQL vacuum operation, which touches much of the database exactly once.

Much time and effort over the 2.5 development series went into making direct I/O work well. This work helped to close a gap between Linux and some proprietary Unix systems. It could well be that, in 2.7, that effort goes into coming up with a better way of solving the problem altogether.

Note: you can avoid this step in the future by logging into your LWN account.