Kernel development

Kernel release status

The current development kernel is 2.5.54, which was released by Linus on New Years Day. This release contains a large number of patches, most of which are the sorts of fixes that one would expect during a feature freeze. There is also a new bit of compiler trickery to issue warnings when deprecated functions are called, a number of kbuild fixes, a new dev_printk() function for standardized device error reporting, the removal of the much disliked hugetlb system calls (in favor of hugetlbfs), a new "kmalloc for each CPU" API, a partial lm_sensors merge (see below), and more loadable module fixes. As usual, the details can be found in the long-format changelog.

Linus's (pre-2.5.55) BitKeeper tree includes a number of big architecture updates (PowerPC, ARM, x86-64), some kbuild work, a knfsd update, more module fixes, another set of driver model patches, some device mapper updates, a number of video4linux tweaks, and numerous other fixes and updates.

The current stable kernel is 2.4.20. Marcelo continued the 2.4.21 process with 2.4.21-pre3, released on January 6. This prepatch includes fixes for some (potential) security bugs, a number of USB driver updates, some IPv6 tweaks, and a number of otheir fixes and updates.

Alan Cox has released 2.4.21-pre3-ac1, which adds another set of fixes and updates. This patch no longer includes the reverse mapping virtual memory code. (Update: 2.4.21-pre3-ac2 came out, with an important bug fix, just as the Weekly Edition was being published).

Comments (none posted)

lm_sensors merged

Included in the 2.5.54 kernel was the long-awaited merge of (part of) the lm_sensors project. lm_sensors has existed as a separate patch for years, but has never found its way into the mainline kernel - partly as a result of concern over its tendency to destroy certain models of Thinkpad laptop. Linus evidently decided that the time has come, however.

lm_sensors is really just a set of drivers which provide access to the health-monitoring hardware present on most motherboards. With lm_sensors, Linux users (or system monitoring daemons) can keep an eye on processor temperatures, supply voltages, fan speeds, and even cabinet intrusions on some systems. This is a worthwhile capability; it's nice to know that a fan has failed on a system before it's too late. So it's good to see that this patch has finally found its way in. See the lm_sensors page for detailed information, a lengthy FAQ, and user-space programs.

Comments (2 posted)

Kernel latency hits a new low

Tucked away in Andrew Morton's 2.5.54-mm3 patchset is a new bit of work aimed at reducing the latency of the Linux kernel. Latency, from the point of view of this work, is the time lag between when a high-priority process becomes runnable and when it actually gets the processor. Scheduling latency is important in a number of contexts, and it can be especially important for desktop users. When you move your mouse, it is nice not to have to wait until the pointer on the screen moves to keep up with it. Low latency is crucial for certain applications, including streaming media recording and playback, CD recording, data acquisition, and so on. If the system is not sufficiently responsive, these applications just do not work at all.

The last source of long delays in the kernel, says Andrew, is in the page table teardown code. This delay is easily seen - simply shut down a large application (Mozilla or OpenOffice will do nicely) and try to get anything else done while the cleanup is happening. This delay happens because teardown code holds the process's page_table_lock for the entire cleanup task. If the process is large, the cleanup can take a long time. Since the kernel is holding a lock, it can not be dislodged from the processor even if the kernel is compiled for preemption. So anything else that wants to run has to wait until the whole cleanup job is done.

The solution is to create a new "uber-zapper" function (unmap_vmas()) which handles the page table cleanup task. The page range to be torn down is split into smaller chunks (between 256 and 2048 pages, depending on the architecture and kernel config options); between chunks, the lock is dropped and the processor rescheduled if necessary. When the high-priority task has finished doing its thing, the lock is reacquired and the next block of pages is freed. Along with reducing latency, the patch has the additional advantage of cleaning up the separate unmapping code which was duplicated in three different places.

The result, it is claimed, is a worst-case scheduling latency of 500 microseconds on a 500 MHz Pentium processor. At least, if you are using the ext2 filesystem and if you are not mounting and unmounting filesystems. That should be fast enough for most users.

Comments (12 posted)

Deprecated kernel functions

One small patch that slipped into 2.5.54 is the addition of a new __deprecated function attribute. With a suitably modern compiler (gcc 3.1 or newer), calls to a function marked as being deprecated will generate compile-time warnings. The hope is that the warnings will inspire people to remove calls to the deprecated functions, making it easier to remove them altogether.

So far, this attribute has been used sparingly; the only functions which have been marked are check_region() (which has long been obsoleted by the race-free version of request_region() which returns a success value), and the old module use count macros (MOD_INC_USE_COUNT and MOD_DEC_USE_COUNT). Patches have been put forward to mark other functions, but there has been resistance to doing so, for a couple of reasons.

One reason, as expressed by Linus, is that the functions that have deprecated so far already generate far too many warnings. A quick grep turns up over 1000 check_region() calls in the 2.5 kernel. Adding more warnings to the mix is not going to help get things fixed, and may well just mask other warnings about real problems.

The other objection that has been raised is that trying to clean up use of deprecated functions at this stage distracts attention from the most important task: stabilizing the 2.5 kernel for release. The current code works, for the most part, even if it's using deprecated functions. It's hard to imagine cleaning up 1000 check_region() calls without breaking something, somewhere.

The end result is that, probably, not too many other kernel functions will be marked as deprecated in this development cycle. Some functions that had been expected to disappear (e.g. sleep_on()) will persist into 2.7 - they are still extensively used in some places. Cleaning up old stuff is never easy; it is simpler to put code into the kernel than to take it out.

Comments (3 posted)

Smatch - a Stanford Checker for the rest of us

The "Stanford Checker" (also known as "MC") is a project headed up by Stanford professor Dawson Engler; MC uses a modified version of the gcc compiler to find potential errors in C code. Occasionally, the Stanford Checker group surfaces on linux-kernel with a list of new problems found by MC; the last such posting listed a set of potential buffer overrun vulnerabilities on January 2.

The Checker postings are appreciated by the kernel developers, since they have pointed out a large number of real bugs. It would be even nicer if the Checker were available for others to use, but that is not the case. The MC group still has not released its work, which, it claims, remains incomplete. So there is little to do except to wait for the next posting.

Dan Carpenter, however, got tired of waiting and set out to create his own MC-like system. The result is Smatch, which was announced on the kernel janitors' list on January 1. Smatch attempts to duplicate the techniques used in the Stanford Checker, as derived from papers published by the MC group. It is still very much a work in progress; rather than producing nice reports, Smatch creates large amounts of raw data which must then be filtered with Perl scripts. An initial set of scripts exists, but quite a bit of work remains to be done in that area.

So Smatch probably will not be putting the Stanford Checker out of business anytime soon. But it will provide a platform for the development of freely-available checking tool with similar capabilities. With luck, and some development time, Smatch should help in the creation of more stable kernels in the near future.

Comments (2 posted)