Kernel development

Kernel release status

The 2.6.37 kernel is out, released by Linus Torvalds on January 4. New features in 2.6.37 include ext4 scalability improvements, jump labels, "tiny preempt RCU", more BKL removal work, getting rid of barriers in the block subsystem, much of the Xen Dom0 support, fanotify has been re-enabled, and more. As always, see the KernelNewbies 2.6.37 page for lots more information. "As to the big picture, on the whole I think 2.6.37 has been fairly calm. That will likely change with the merge window, as I'm looking forward to getting the new RCU-based pathname lookup and similar exciting features. So enjoy the calm sanity while it lasts". While the merge window is theoretically open, Torvalds said that he will wait a bit before starting the process to "let 2.6.37 cool for a few days to try to encourage people to look at the release rather than go all crazy with newly merged features in the next tree".

While we were on our holiday break, 2.6.37-rc8 was released on December 28. "The -rc8 release shouldn't be all that exciting. The most noticeable is probably the fact that hopefully the 'blank screen' problem with intel graphics is fixed. But on the whole, it's all just a collection of random fixes all over." See the full changelog for all the details.

There have been no stable releases made in the last two weeks. The 2.6.34.8 and 2.6.32.28 longterm kernels along with the 2.6.36.3 stable kernel are in the review process at the time of this writing.

Comments (1 posted)

Quotes of the fortnight

Comments (53 posted)

Paul McKenney's parallel programming book

Comments (29 posted)

Gettys: Bufferbloat in 802.11 and 3G Networks

Comments (16 posted)

Announcing the beta release of PowerTOP 2.0

Full Story (comments: 5)

A Linux kernel compatibility layer for FreeBSD?

As part of his work in porting a Linux-based InfiniBand stack to FreeBSD, Jeff Roberson put together a compatibility layer that maps Linux-specific kernel functionality to FreeBSD equivalents. He announced this effort on the freebsd-arch mailing list in order to get a general sense of what other FreeBSD developers thought of the idea—and whether it might make sense to combine it with other, similar, compatibility shims. The thread gives an interesting look into the problems that less popular operating systems have when trying to support new functionality that is already well-established in Linux, even if all of that code is not in the mainline.

The InfiniBand stack that Roberson ported was developed by the OpenFabrics Alliance as part of its OpenFabrics Enterprise Distribution (OFED). It largely consists of patches on top of the existing Linux InfiniBand code. All of the code is available under both the GPL and BSD licenses, which is what allowed Roberson to port it to FreeBSD. In order to minimize changes to that stack, though, he created a fairly large compatibility layer:

Roberson goes on to list the kinds of things that the compatibility layer supports, including atomic operations, various device types, interrupts, kobjects, radix and rbtrees, lists, spinlocks, and so on. For the most part, it is just translating Linux-isms into their FreeBSD counterparts. The current state of the wrapper can be seen in Roberson's svn repository, which consists of a linux kernel include tree along with code to implement those Linux-isms. The OFED code can then be built by pointing it at the include tree.

Only those parts of Linux that were needed by the OFED code were actually implemented in the compatibility layer, but Roberson wonders if it makes sense to combine his effort with others:

While there was some predictable Linux-bashing in the thread, most participants seemed favorably inclined toward the compatibility layer itself, as well as making it more widely available to other parts of the FreeBSD kernel tree. There are some difficulties, of course, including exactly which Linux kernel version should be tracked. As was pointed out, somewhat caustically, by Garrett Wollman in the thread, internal Linux interfaces change with some frequency, and without much warning:

It's not just the size and ability of the Linux community that has led to a stable and usable system, though. As Matthew Jacob notes, Linux developers point to the ability to change the internal interfaces as one of the strengths of their development model. Although the FreeBSD developers may not agree with it, the stable_api_nonsense document does provide reasons for the Linux developers' disdain for stable internal APIs.

Still, choosing a kernel version to track is not necessarily easy. Based on some previous work maintaining an out-of-tree Linux filesystem, Chris BeHanna was able to offer some advice on that:

There are some obvious advantages to using the OFED code with minimal modifications. Bug fixes and enhancements are being made regularly that could be incorporated into the FreeBSD kernel fairly easily using the compatibility layer. But some argue that adding such a layer will result in fewer vendors trying to get their code working natively for FreeBSD, likening it to the linuxulator that allows Linux binaries to run on FreeBSD. Alexander Kabaev put it this way:

But Roberson is not convinced that the linuxulator has had a negative effect on FreeBSD market share, and instead sees it as allowing FreeBSD to "stay relevant in areas where companies can not justify an independent FreeBSD effort. Adobe is a good example of this."

Roberson seems to be taking a pragmatic approach to getting InfiniBand support into current versions of FreeBSD. Evidently there is an InfiniBand stack that runs on earlier versions of FreeBSD, but it hasn't been maintained for more recent kernels. There aren't any real alternatives, as he sees it:

So far, there has been no answer to his question. It may be that the compatibility layer continues to live in the InfiniBand tree, rather than move to a more widely accessible place, but there seems to be no one stepping up to develop and maintain a FreeBSD-specific stack. While there is some resistance to the 10,000-line size of the layer, it is, for the most part, a pretty thin shim, as Roberson describes:

It wasn't brought up in the thread, but there may also be questions about the license of the compatibility layer (seemingly BSD-only) at some point. Lawyers might argue that it is a derivative work of the Linux kernel. It's unlikely to be a real problem unless someone takes the code into a closed-source product, which is something that the BSD license allows. Specifically licensing it under the GPL would avoid that particular problem, though that might not fit well with the goals of the FreeBSD project.

It is undoubtedly galling to some in the FreeBSD community to adopt a big chunk of Linux code along with a Linux mapping layer, but the reality is that the community isn't large enough to go it alone, at least for InfiniBand. In some ways it is reminiscent of the problems that came about when the new direct rendering manager (DRM) development process was announced. In both cases, the community of FreeBSD users and developers is just not large enough to justify some kind of dual-OS approach.

On the other hand, though, in both cases the licenses specifically allow Linux and FreeBSD (and others) to incorporate the code. While there are lots of technical and development process barriers to more collaboration between Linux and the BSD world, the usually painful license incompatibility issue did not rear its head. With luck, that will play out well for both sides, with dual-licensed patches going in both directions, hopefully without the rancor that sometimes occurs.

[Thanks to James Andrewartha for giving us a heads-up about this topic.]

Comments (6 posted)

The trouble with firmware

Within the various communities that make up the free software world, there are many different opinions on exactly what constitutes "freedom". Those opinions run the gamut from extremely pragmatic to exceedingly strict definitions of "free". Firmware blobs for drivers—almost always released without source code and often without any clear license—have been particularly troublesome for certain projects. Debian has struggled with the problem over the years, finally resolving it by moving the non-free firmware out of its main repository for the upcoming 6.0 ("Squeeze") release. But there are others who find even that insufficient and would like to see any mention of the non-free firmware files removed from the kernel.

The Linux-libre project aims to deliver a completely free (under its definition) Linux distribution. It does that by removing any programs that either don't come with source code, or that has source code that is licensed in a way that does not allow users to modify it. Many Linux distributions already follow those guidelines, but Linux-libre takes things one step further and removes anything that "induces or requires you to install additional pieces of non-Free Software".

As might be guessed, Linux-libre has a rather liberal interpretation of what "induces" means, which is where the kernel firmware loading mechanism runs afoul of the distribution's goals. Linux-libre has already eliminated the non-free firmware from its repositories, but there is still a problem: the kernel will make a request to user space for (potentially) non-free firmware when it loads a driver that requires it. So in the minds of the Linux-libre developers, the kernel is now inducing users to install non-free software.

If user space cannot find the firmware file, then the driver obviously cannot load it into the hardware. That makes the hardware non-functional, presumably, but far worse from an inducement perspective, the kernel puts a message into the log specifically noting the filename that couldn't be found. Evidently, that will induce users to go find a file by that name and install it, thus resulting in a non-free system.

In a discussion about Debian's Squeeze kernel on the linux-libre mailing list, Richard Stallman mentioned a plan to obfuscate the names of the firmware files within the Linux-libre kernel. Part of his message was picked up as a Kernel page "quote of the week", partly because it seems like something of a contradiction to promote freedom by hiding the proper names of firmware files. It's also a little puzzling because the GPL and the software freedom movement is so clearly focused on the rights of users.

But Stallman isn't trying to stop users from finding the firmware, he just wants to ensure that the distribution doesn't induce them to install it. That distinction may seem rather arbitrary to many, but not to him. In response to a concern expressed by Alexandre Oliva about web pages that would show the mapping between the Linux-libre obfuscated names and those of the actual filenames, Stallman explained:

The plan seems to go far beyond just creating a kernel with obfuscated firmware names, though. It would be fairly straightforward to run a script on the kernel source that applied some kind of obfuscation filter to the firmware filenames before building the kernel, but that doesn't completely alleviate the problem for Linux-libre. It would also like to have a kernel Git repository with the mangled filenames, but there are problems with that approach as Oliva notes:

From there, the conversation starts to take on some Orwellian tones. It is not enough to just have the new firmware filenames in the HEAD of the Git repository, Oliva and Stallman would like to eliminate references to the original filenames (and the firmware files themselves) from the repository history. But they also would like to be able to pull from Linus Torvalds's mainline tree periodically. It is a tricky set of conditions to try to fulfill.

As part of the discussion on how to handle it, Stallman seems to bristle over the term "rewriting history"—perhaps because of its Orwellian overtones. He sees that term as (somehow) pushing Torvalds's views:

Stallman doesn't necessarily see being able to pull from the mainline as much more than a convenience. But Oliva, who clearly has more Git and kernel workflow experience than Stallman, is fairly adamant:

To that end, Oliva asked for ways to accomplish this goal on the Git mailing list. He comes up with some plausible scenarios where others might want this kind of functionality, beyond just the needs of Linux-libre:

A few suggestions of possible workflows were made in that thread, but Oliva seems to be leaning toward using git-filter-branch and either some scripts or changes to Git to facilitate this effort. While his scenarios are a little bit of a stretch—why distribute a project that contains these problematic pieces?—they aren't completely beyond the pale. Others could benefit from whatever solution Linux-libre finally settles on.

It's certainly not the first time that Stallman has seemed to be tilting at windmills, and plenty of folks will find this effort to be pointless, even comical. There are some good arguments on both sides of the firmware debate, but it is a bit hard to see how making it more difficult to get hardware working on Linux-libre is going to lead to more adoption of that distribution. Far more likely, it would seem, is that folks just grab a different distribution (or operating system entirely) and move on.

Comments (24 posted)

Who wrote 2.6.37

The 2.6.36 kernel was unique in that it was actually smaller than its predecessor. 2.6.37 does not continue that trend; some 1,140,000 lines of code were added, and 641,000 lines were removed, for a net growth of 494,000 lines. Most notably, perhaps: the 2.6.37 kernel includes patches from 1,250 developers, the highest ever. The development community, clearly, is active and growing.

The most active contributors this time around were:

Most active 2.6.37 developers

By changesets Chris Wilson 211 1.9% Greg Kroah-Hartman 171 1.5% Eric Dumazet 166 1.5% Johannes Berg 149 1.3% Thomas Gleixner 147 1.3% Paul Mundt 140 1.2% Mauro Carvalho Chehab 122 1.1% Joe Perches 107 1.0% Avi Kivity 105 0.9% Mark Brown 100 0.9% Hans Verkuil 100 0.9% Namhyung Kim 100 0.9% Dan Carpenter 99 0.9% Christoph Hellwig 93 0.8% Jean Delvare 88 0.8% Axel Lin 88 0.8% Daniel Vetter 87 0.8% Vasiliy Kulikov 86 0.8% Arnd Bergmann 86 0.8% Julia Lawall 84 0.7%

By changed lines Henry Ptasinski 143303 9.6% Greg Kroah-Hartman 100861 6.7% Vipin Mehta 92398 6.2% Luis R. Rodriguez 71636 4.8% Mark Brown 53690 3.6% David Cross 48198 3.2% Dmitry Kravkov 47198 3.1% Larry Finger 40378 2.7% Krishna Gudipati 38712 2.6% Stefan Richter 29434 2.0% Stephen Hemminger 28504 1.9% Rasesh Mody 22335 1.5% Prashant P. Shah 15086 1.0% Michael Chan 14118 0.9% Christian Lamparter 13530 0.9% Liam Girdwood 13324 0.9% William Hubbs 12971 0.9% Vinod Koul 12944 0.9% Marek Belisko 11531 0.8% Al Cho 10097 0.7%

These lists feature a combination of old and new names. Chris Wilson got to the top of the by-changesets list by virtue of his work with the Intel graphics drivers. Greg Kroah-Hartman worked on the USB and TTY subsystems, but the bulk of his changes were made to the staging tree, and the new brcm80211 driver in particular. Eric Dumazet has been active all over the networking layer, Johannes Berg has been busy with wireless networking, and Thomas Gleixner rewrote the generic interrupt handling code (among many other things).

Looking at lines changed: Henry Ptasinski arrived at the top of the list through the addition of the brcm80211 driver which, as can be seen, is not a small piece of code. Vipin Mehta added the ath6kl driver to the staging tree, Luis Rodriguez worked on the mac80211 layer and the ath9k driver, and Mark Brown, as always, did a massive amount of work within the ALSA sound layer. It's interesting to note that five of the top ten in this column were mainly involved with the staging tree. One of the others (Krishna Gudipati) submitted a single "driver cleanup patch" which drew Linus's ire at the time - calling a nearly 40,000-line patch a "cleanup" seemed like a bit much.

A minimum of 193 employers supported work on the 2.6.37 kernel; the top supporters were:

Most active 2.6.37 employers

By changesets (None) 1864 16.6% Red Hat 1265 11.3% (Unknown) 946 8.4% Intel 746 6.6% Novell 644 5.7% IBM 447 4.0% Oracle 272 2.4% Texas Instruments 250 2.2% Nokia 235 2.1% Renesas Technology 210 1.9% Samsung 205 1.8% Broadcom 172 1.5% Societe Française de Radiotelephone 166 1.5% AMD 163 1.5% (Consultant) 155 1.4% Pengutronix 154 1.4% (Academia) 146 1.3% Wolfson Micro 129 1.1% Google 128 1.1% Fujitsu 127 1.1%

By lines changed Broadcom 246749 16.4% (None) 188830 12.6% Atheros 165693 11.0% Novell 127476 8.5% Wolfson Micro 100196 6.7% Brocade 64012 4.3% (Unknown) 59325 4.0% Red Hat 59000 3.9% Intel 53893 3.6% Cypress Semiconductor 48241 3.2% Vyatta 30720 2.0% Texas Instruments 22874 1.5% Samsung 16771 1.1% IBM 16567 1.1% Nokia 14731 1.0% Renesas Technology 14488 1.0% (Consultant) 13592 0.9% Slimlogic Ltd 13324 0.9% ST Ericsson 13056 0.9% Chelsio 10770 0.7%

Looking at the by-changesets column, the situation looks mostly like business as usual. Red Hat remains, by far, the largest corporate contributor to the kernel. On the lines-changed side, instead, Red Hat had to settle for eighth place behind companies which, for the most part, have contributed a lot of driver code.

It has been some time since we looked at the reviewing and testing of code. In the 2.6.37 development cycle, the developers with the most Reviewed-by tags were:

Most active 2.6.37 reviewers
Ingo Molnar	86	12.6%
Christoph Hellwig	37	5.4%
Mike Christie	29	4.3%
Michael Chan	26	3.8%
Josh Triplett	22	3.2%
Daniel Vetter	21	3.1%
Chuck Lever	21	3.1%
H. Peter Anvin	20	2.9%
Konrad Rzeszutek Wilk	19	2.8%
Alex Elder	17	2.5%
Luciano Coelho	16	2.3%
Suresh Jayaraman	16	2.3%
Swen Schillig	16	2.3%
Michal Marek	15	2.2%
Jeff Layton	15	2.2%
Sam Ravnborg	15	2.2%
Wu Fengguang	12	1.8%
Francisco Jerez	12	1.8%
KOSAKI Motohiro	12	1.8%
Christoph Lameter	11	1.6%

As always, there is only so much that can be learned from these numbers; the bulk of all patch reviews do not lead to the addition of a Reviewed-by tag. On the other hand, there has been some real social pressure to credit users who test patches and report bugs:

Most credited 2.6.37 reporters and testers

Reported-by credits Stephen Rothwell 24 4.2% Randy Dunlap 21 3.7% Linus Torvalds 19 3.3% Ingo Molnar 15 2.6% Guennadi Liakhovetski 10 1.8% Jonathan Cameron 7 1.2% David Brownell 7 1.2% Sitsofe Wheeler 6 1.1% Andrew Morton 6 1.1% Dan Rosenberg 6 1.1% Jiri Slaby 5 0.9% Ben Greear 5 0.9% Eric Dumazet 4 0.7% Daniel Vetter 4 0.7% David S. Miller 4 0.7% Robin Holt 4 0.7% Andi Kleen 4 0.7% Dan Carpenter 4 0.7% Sachin Sant 4 0.7% Dr. David Alan Gilbert 4 0.7%

Tested-by: credits Wolfram Sang 15 4.1% Luciano Coelho 11 3.0% Kevin Hilman 10 2.7% Jeff Pieper 10 2.7% Will Deacon 9 2.5% Caglar Akyuz 8 2.2% Michael Williamson 8 2.2% Emil Tantilov 7 1.9% Randy Dunlap 6 1.6% Stephen Ko 6 1.6% Ben Greear 6 1.6% Eric Benard 5 1.4% Tuomas Katila 5 1.4% Maxim Levitsky 5 1.4% Ingo Molnar 4 1.1% Juuso Oikarinen 3 0.8% Jason Wessel 3 0.8% Kuninori Morimoto 3 0.8% Rabin Vincent 3 0.8% Ben Gardiner 3 0.8%

All told, there were 568 Reported-by and 366 Tested-by tag lines found in patches merged for 2.6.37. We are, it seems, slowly getting better at recognizing the people who are doing this crucially important work.

In summary, the kernel development process continues to look healthy. We have a great deal of activity from an increasing number of developers, while, it seems, keeping a reasonable lid on the number of regressions introduced. Whether the high patch rate will continue into 2.6.38 remains to be seen; as of this writing, there are just under 5,000 changes in linux-next. Unless the subsystem maintainers put more work into linux-next in the near future, the next development cycle could be relatively slow.

Comments (9 posted)