User: Password:
Subscribe / Log in / New account

Kernel development

Brief items

Kernel release status

The current 2.6 development kernel is 2.6.27-rc9, released on October 6. Says Linus: "I know, I know, I said that -rc8 was supposed to be the last -rc, and that I'd release 2.6.27 this weekend. I lied. Sue me. I merged two subtle regression fixes today, and while both looked perfectly fine and had been tested by the people involved in the regressions, I just couldn't bring myself to then just slap a 'v2.6.27' on it without some more testing." Expect the final 2.6.27 release in the near future.

It is worth noting that, as of this writing, 2.6.27 does not contain a fix for the e1000e hardware corruption bug. What it does contain, though, is a series of patches which will prevent that bug from actually damaging the hardware. That makes the kernel safer to run, which is an important step in the right direction.

No stable kernel releases have been made over the last week. As of this writing, though, there were large updates for the 2.6.25 and 2.6.26 kernels in the review process.

Comments (2 posted)

Kernel development news

Quotes of the week

Of greatest interest to me were the descriptions given by Patrick McHardy for his new filtering framework, where all the complexity is in userspace and the kernel just runs filtering scripts and lookup datastructures fed to it by the user tools. In short, I think this stuff is great, and unlike some folks I don't think this will decrease netfilter participation by other developers at all.

And frankly, iptables was absolutely too accessible to contributors. Look at how much stinking poo is in the patch-o-matic, oft called "crap-o-matic".

-- David Miller

But then voting season comes and reminds you that all those Americans that are individually sane and normal tend to be collectively crazy and very odd. And that's when you really notice that you're not in Finland any more.
-- Linus Torvalds starts a blog

Comments (none posted)

Some development statistics for 2.6.27

By Jonathan Corbet
October 7, 2008
It's that time of the development cycle again: the 2.6.27 kernel, if not yet released by the time you read this, will be shortly. Various other LWN articles have looked at features found in this release; here we will look at where that code came from.

As of 2.6.27-rc9, a total of 10,604 non-merge changesets had been added to the mainline for the 2.6.27 kernel; those patches added a total of 826,000 lines of code while removing 608,000, for a net growth of 217,000 lines. There were 1,109 developers who contributed to 2.6.27, representing over 150 employers. 376 of those developers contributed a single patch during this development cycle.

The most active developers for 2.6.27 were:

Most active 2.6.27 developers
By changesets
Ingo Molnar2382.2%
Bartlomiej Zolnierkiewicz2352.2%
Adrian Bunk2212.1%
David S. Miller2061.9%
Alan Cox1961.8%
Yinghai Lu1921.8%
Jeremy Fitzhardinge1621.5%
Tomas Winkler1281.2%
Ben Dooks1201.1%
Jean Delvare1131.1%
Steven Rostedt1081.0%
Harvey Harrison1051.0%
Pavel Emelyanov1031.0%
Thomas Gleixner1011.0%
Jean-Francois Moine890.8%
Lennert Buytenhek880.8%
Hans Verkuil810.8%
Joerg Roedel810.8%
Arnd Bergmann760.7%
David Brownell750.7%
By changed lines
Paul Mackerras13837412.1%
David Woodhouse447593.9%
Jean-Francois Moine411573.6%
Adrian Bunk351603.1%
Artem Bityutskiy345453.0%
Luis R. Rodriguez318252.8%
Sam Ravnborg274432.4%
Karsten Keil246742.2%
Russell King228612.0%
Eilon Greenstein194701.7%
Alan Cox169571.5%
Felipe Balbi162871.4%
Kumar Gala144901.3%
David Brownell125511.1%
Ralf Baechle110571.0%
Lennert Buytenhek97350.9%
David S. Miller86210.8%
Juergen Beisert85160.7%
Steven Rostedt84550.7%
Ben Dooks83990.7%

On the changeset side, Ingo Molnar ended up on top by virtue of the creation of large numbers of mostly x86-related changes, including a big subarchitecture reorganization; Ingo's count also includes the addition of ftrace, though much of that code was written by others. Bartlomiej Zolnierkiewicz continues to rework the old IDE layer, and Adrian Bunk, as always, energetically cleans up code all over the tree. David Miller's total includes the multiqueue networking code and a lot of other changes; Alan Cox did a lot of TTY work and big kernel lock removal.

Your editor was disappointed to come in at #23, and, thus, off the bottom of the table. Time to send in some quick white space fixes. More seriously, though, it's worth noting that there are relatively few patches of the "trivial change" variety in the mix this time around.

If we look at changed lines, Paul Mackerras comes out on top as the result of a single patch removing the obsolete ppc architecture. David Woodhouse reworked the management of firmware throughout the driver tree. Jean-François Moine brought the GSPCA webcam drivers into the tree, then put vast amounts of effort into cleaning them up. Artem Bityutskiy added the UBIFS flash filesystem, and Luis Rodriguez merged the ath9k wireless driver.

If we look at the companies behind this work, we get the following results (note that, as always, these results are somewhat approximate):

Most active 2.6.27 employers
By changesets
Red Hat140513.2%
Renesas Technology1251.2%
By lines changed
Red Hat1099709.6%
Renesas Technology157231.4%
MIPS Technologies157011.4%
Analog Devices107250.9%

There are not too many surprises in this table - in particular, the list of companies at the top tends not to change very much. That said, a few things are worthy of note. One is that Sun Microsystems has made its first appearance on this list. People complain about this company, but Sun's engineers have been quietly fixing things all over the tree. Broadcom is another company with a mixed reputation in the Linux community, but Broadcom is happy to provide support for some of its network adapters. Nokia's strong showing in the lines-changed table results primarily from the contribution of the UBIFS filesystem.

The most welcome change, though, is the first appearance of Atheros on this list. Atheros is a company which has quickly moved from a position of complete non-cooperation to one of supporting all of its hardware in the mainline kernel. To say that this is an encouraging development would be an understatement.

All told, the 2.6.27 development cycle shows that the process continues at full pace in a seemingly healthy state. Developers from all over the industry are all working together to make the kernel better for all. The number of companies which see participation in the process as being in their interest is growing, as is the number of developers who contribute patches. The Linux kernel, it seems, is in good shape.

Comments (22 posted)

Btrfs to the mainline?

By Jake Edge
October 8, 2008

One of the kernel projects that seems to be attracting a fair amount of attention these days is the new, copy-on-write filesystem, Btrfs. While still rather immature—the disk format is slated to be finalized by the end of the year—Btrfs has reached a point where lead developer Chris Mason wants to start talking about when to merge it into the mainline. Some are advocating moving quickly, while others are a bit more skeptical that merging it will lead to faster development.

Merging Btrfs would have a number of advantages, but more eyes is what Mason is seeking:

But, the code is very actively developed, and I believe the best way to develop Btrfs from here is to get it into the mainline kernel (with a large warning label about the disk format) and attract more extensive review of both the disk format and underlying code.

The Btrfs developers are committed to making the FS work and to working well within the kernel community. I think everyone will be happier with the final result if I am able to attract eyeballs as early as possible.

Typically, kernel code is not merged until it is ready, but an argument can be made that filesystems, like device drivers, are sufficiently isolated from the rest of the kernel that an early inclusion will do little harm. Also, a kind of precedent was set by the early "merge" of ext4, though that was an evolution of the existing ext3 filesystem, while Btrfs is entirely new. Andrew Morton has been encouraging Mason to get Btrfs "into linux-next asap and merge it into 2.6.29." He describes his reasoning:

My thinking here is that btrfs probably has a future, and that an early merge will accelerate its development and will broaden its developer base. If it ends up failing for some reason, well, we can just delete it again.

For various reasons this approach often isn't appropriate as a general policy thing, but I do think that Linux has needed a new local filesystem for some time, and btrfs might be The One, and hence is worth a bit of special-case treatment.

Adrian Bunk is not convinced that an early merge will bring the benefits that Morton is touting. He points to an early ext4 development plan, noting that the timelines outlined in that message were, perhaps, overly optimistic. "When comparing with what happened in reality it kinda disproves your 'acceleration' point."

There is a difference, though, between ext4 and Btrfs, that Serge Hallyn points out:

OTOH, maybe it's just me, but I think there is more excitement around btrfs. Myself I'm dying for snapshot support, and can't wait to try btrfs on a separate data/scratch partition (where i don't mind losing data). btrfs and nilfs - yay. Ext4? <yawn> That can make all the difference.

The original timeline showed mid-2007 as a target for a stable ext4 filesystem, but the project overshot that by a year or so. A recent patch proposes renaming ext4dev to ext4 because it "is getting stable enough that it's time to drop the 'dev' prefix." Unexpected difficulties led to ext4 development taking longer, as Mason describes:

Ext4 has always had to deal with the ghost of ext3. Both from a compatibility point of view and everyone's expectations of stability. I believe that most of us underestimated how difficult it would be to move ext4 forward.

Many seem to think that Btrfs is different, but it still has a ways to go. Currently, it does not handle I/O errors very well, while running out of space on the disk can be fatal. But it is getting close to usable—at least for testing and benchmarking. Getting the code into the mainline would cause more folks to look at it, as well as test various filesystem changes against it. Mason gives an example of how that can work:

For example, see the streaming write patches I sent to fsdevel last week. I wouldn't test against ext4 as often if I had to hunt down external repos just to get something consistent with the current development kernels. ext4 in mainline makes it much easier for me to kick the tires.

Btrfs has an aggressive schedule that targets a 1.0 release this year. The focus of that release is to nail down the on-disk format so that changes after that point will be backward compatible. Given that 2.6.29 will likely be released in early to mid-2009, it seems quite possible that Btrfs will be "merge-worthy" by then, which means that it really is not premature to start considering it now.

Comments (6 posted)

Moving interrupts to threads

By Jake Edge
October 8, 2008

Processing interrupts from the hardware is a major source of latency in the kernel, because other interrupts are blocked while doing that processing. For this reason, the realtime tree has a feature, called threaded interrupt handlers, that seeks to reduce the time spent with interrupts disabled to a bare minimum—pushing the rest of the processing out into kernel threads. But it is not just realtime kernels that are interested in lower latencies, so threaded handlers are being proposed for addition to the mainline.

Reducing latency in the kernel is one of the benefits, but there are other advantages as well. The biggest is probably reducing complexity by simplifying or avoiding locking between the "hard" and "soft" parts of interrupt handling. Threaded handlers will also help the debuggability of the kernel and may eventually lead to the removal of tasklets from Linux. For these reasons, and a few others as well, Thomas Gleixner has posted a set of patches and a "request for comments" to add threaded interrupt handlers.

Traditionally, interrupt handling has been done with top half (i.e. the "hard" irq) that actually responds to the hardware interrupt and a bottom half (or "soft" irq) that is scheduled by the top half to do additional processing. The top half executes with interrupts disabled, so it is imperative that it do as little as possible to keep the system responsive. Threaded interrupt handlers reduce that work even further, so the top half would consist of a "quick check handler" that just ensures the interrupt is from the device; if so, it simply acknowledges the interrupt to the hardware and tells the kernel to wake the interrupt handler thread.

In the realtime tree, nearly all drivers were mass converted to use threads, but the patch Gleixner proposes makes it optional—driver maintainers can switch if they wish to. Automatically converting drivers is not necessarily popular with all maintainers, but it has an additional downside as Gleixner notes: "Converting an interrupt to threaded makes only sense when the handler code takes advantage of it by integrating tasklet/softirq functionality and simplifying the locking."

A driver that wishes to request a threaded interrupt handler will use:

    int request_threaded_irq(unsigned int irq, irq_handler_t handler,
	    		     irq_handler_t quick_check_handler,
			     unsigned long flags, const char *name, void *dev)
This is essentially the same as request_irq() with the addition of the quick_check_handler. As requested by Linus Torvalds at this year's Kernel Summit, a new function was introduced rather than changing countless drivers to use a new request_irq().

The quick_check_handler checks to see if the interrupt was from the device, returning IRQ_NONE if it isn't. It can also return IRQ_HANDLED if no further processing is required or IRQ_WAKE_THREAD to wake the handler thread. One other return code was added to simplify converting to a threaded handler. A quick_check_handler can be developed prior to the handler being converted; in that case, it returns IRQ_NEEDS_HANDLING (instead of IRQ_WAKE_THREAD) which will call the handler in the usual way.

request_threaded_irq() will create a thread for the interrupt and put a pointer to it in the struct irqaction. In addition, a pointer to the struct irqaction has been added to the task_struct so that handlers can check the action flags for newly arrived interrupts. That reference is also used to prevent thread crashes from causing an oops. One of the few complaints seen so far about the proposal was a concern about wasting four or eight bytes in each task_struct that was not an interrupt handler (i.e. the vast majority). That structure could be split into two types, one for the kernel and one for user space, but it is unclear whether that will be necessary.

Andi Kleen has a more general concern that threaded interrupt handlers will lead to bad code: "to be honest my opinion is that it will encourage badly written interrupt code longer term," but he seems to be in the minority. There were relatively few comments, but most seemed in favor—perhaps many are waiting to see the converted driver as Gleixner promises to deliver "real soon". If major obstacles don't materialize, one would guess the linux-next tree would be a logical next step, possibly followed by mainline merging for 2.6.29.

Comments (14 posted)

Patches and updates

Kernel trees


Core kernel code

Development tools

Device drivers


Filesystems and block I/O

Memory management



Virtualization and containers

Benchmarks and bugs


Page editor: Jonathan Corbet
Next page: Distributions>>

Copyright © 2008, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds