Kernel development

Kernel release status

The current development kernel remains 2.6.0-test11; Linus seems to be waiting for Andrew Morton to return from vacation and pick up the baton. Linus has, however, accepted a few dozen small bugfixes into his BitKeeper repository.

The current stable kernel is 2.4.23; Marcelo has started off the 2.4.24 process with 2.4.24-pre1. This prepatch includes the XFS filesystem (see below) along with a fair number of networking and architecture patches.

Comments (1 posted)

2.4, XFS, and DM

As reported last week, many users had requested that the XFS filesystem be added to the 2.4 kernel despite Marcelo's stated intent to go into a maintenance-only mode. Those users have prevailed: Marcelo has announced that, after a review by Christoph Hellwig, XFS has been merged into his 2.4 tree. It will thus show up in the first 2.4.24 prepatch, whenever that is released.

What has gone into 2.4 is, in fact, not the full XFS patch. Two subsystems have been left out:

• DMAPI (Data Management API). This is an interface which allows the filesystem to communicate with hierarchical storage management systems.

• ACLs. Access control lists require more general extended attribute support, which has never been merged into 2.4.

Users needing those features in XFS will have to run 2.6 to get them. Most users, however, will most likely be happy with the core XFS filesystem.

Meanwhile, a new request has been heard: could the device mapper (LVM2) code be merged? Marcelo's answer was direct: "I believe 2.6 is the right place for the device mapper." That would seem to be the end of the matter, but arguments are now being marshalled to try to get Marcelo to change his mind. This posting by Kevin Corry covers the relevant points quite well:

• LVM2 is a lot nicer. The user-space tools, many taken from IBM's EVMS project, are easier to work with, and the device mapper code provides many capabilities that simply are unavailable in LVM1.

• LVM1 itself has been removed from the 2.6 kernel (though LVM2 does provide backward compatibility). Putting LVM2 into 2.4 would make the transition easier for LVM users; they could get their volume sets working with LVM2 before having to commit to the new kernel.

The forward compatibility argument strikes a chord with many participants in the discussion, but Marcelo is, for now, adamant. One never knows, though; he previously has taken equally strong positions against ACPI and, of course, XFS, but been won over in the end.

Comments (7 posted)

The status of dynamic devices in 2.6

Some of the new users who are just now beginning to test out the 2.6 kernel are encountering a bit of a surprise: devfs is now marked as being deprecated, and users are being pointed to the user-space "udev" solution instead. The deprecation of devfs is no surprise to those who have been following the discussions; despite significant fixups over the 2.5 development cycle, the devfs code continues to have problems which are seen as not being fixable. Some of these problems can lead to kernel deadlocks and worse. These issues are compounded by the fact that the devfs maintainer, Richard Gooch, has not been heard from in some time.

The loss of devfs bothers a relatively small number of people; of the large distributions, only Gentoo uses devfs. But those users have tended to be a little unhappy with the alternatives - many people who have looked at udev (which was first examined here back in April) have concluded that it is not yet ready. Many of the features of devfs are still missing, udev does not yet handle all types of devices, and support for the devfs naming scheme has been slow in coming. As a result, devfs users feel left out in the cold.

What some observers may have missed is that udev development has taken off in the last couple of months. udev 003, released on October 16, was mostly the work of its original author, Greg Kroah-Hartman. Since then udev has picked up its own development community and evolved considerably. The current release as of this writing (udev 008) is a much stronger offering, and several new features are queued up for the 009 release as well. Among other things, udev now offers:

• A FAQ file of its own.

• A "callout" option which allows difficult devices to be set up with an external program.

• Ability to build with klibc for embedded or initramfs usage.

• Proper disk partition support.

• Printf-style formatted dynamic name support.

• Regular expression pattern matching in rules.

• Symbolic link support.

...and quite a bit more. At this point, udev contains something close to the full set of features needed to enable it to replace devfs.

The problem is not yet entirely solved, however. udev depends on sysfs for the information needed to configure device nodes, and not all devices have complete sysfs support at this point. Thus, for example, udev will not yet work with input devices, misc devices, and some others. Certain kinds of subtleties (distinguishing CDROM drives from regular disks, for example) are not yet there. The fault is not with udev itself, but with the information (or lack thereof) that it gets from the kernel. In many cases, patches to add the relevant devices to sysfs are in circulation, but have not yet been merged into 2.6. Once that support has been added, the picture will be nearly complete.

There is one remaining question from devfs users, however, which has earned its own place in the FAQ: dynamic loading of drivers. devfs has the capability to create a device node while deferring loading of the appropriate driver module until a user program actually opens that device node. udev cannot do that, and there is no plan to add that support. In the 2.6 world, driver modules are loaded by the hotplug mechanism when the device is detected, so there is no need to load them at open time. This approach bothers some users who prefer not to load drivers for rarely-used devices, but most users will be able to live with it.

(See also: the udev 008 man page for information on how udev works, and Robert Love's HOWTO on using udev with the development version of Fedora).

Comments (8 posted)

Binary modules and derived works

This week's Front Page contains an interview with a lawyer on the issue of what makes one program a derived work of another. That topic has also been prominent on the linux-kernel mailing list recently as part of the never-ending debate over the status of closed-source kernel modules. Unusually, Linus Torvalds has taken an active part in the discussion this time around. Linus does not have the last word on this issue - his copyrights only cover a small part of the kernel code base - but his thoughts are influential in this regard and can be taken as a sort of guidance for anybody who is considering distributing a binary module.

It is reasonably widely accepted that a program which simply uses a well-documented API does not become a derived work of the code implementing that API. The obvious application of this principle is the kernel's system call interface. That interface is an API (and ABI as well) which, to a great extent, conforms to a well-documented standard, and which is meant to be stable over time. The COPYING file supplied with the kernel explicitly states that programs using the system call interface are not considered to be derived works, but, according to Linus, that statement is not strictly necessary:

Of course, there are differences of opinion. Thus, as Ted Ts'o points out, the FSF claims that users of the readline library must be licensed under the GPL, even though there are two BSD-licensed implementations of the same stable, well-documented API.

But let us accept, for the moment, that this sort of API does, in fact, create a boundary for derived works. A fundamental question that some developers have raised is: what is different about the interface used by kernel modules? Several developers have argued that the module interface is a boundary in just the same way as the system call interface. Linus's response is that the module interface is, indeed, different:

He go on to explain, in fairly graphic and non-technical terms, that the stated intent of the interface matters, even if the technical steps involved in using it are the same. The module interface is not a boundary because the copyright holders never intended it to be one.

One can go further and say that the module interface is truly not an API in the usual sense. It is, indeed, highly volatile, and has been known to change even in the middle of a stable kernel series. The module interface provides extensive access to low-level kernel functionality, often solely for the convenience of other in-tree kernel code. Code which makes use of many exported symbols is clearly Linux-specific. Some exported symbols are so low-level that the kernel developers have concluded that no code can use them without being a derived product; that is why EXPORT_SYMBOL_GPL() exists. But, as Linus notes, EXPORT_SYMBOL_GPL() should be seen as a form of documentation in specific cases; its existence does not imply that modules restricting themselves to non-GPL-only symbols are automatically not derived.

The crucial test, as seen by Linus, is whether a particular module was implemented independently of Linux. Code ported from another system (such as the Andrew filesystem) is a fairly obvious example. Device drivers are generally seen as being independent as well. Before going any further, though, distributors of closed-source modules should consider one other thing that Linus said:

Until a court makes a ruling, nobody really knows what the law really says. Even then, as Dan Ravicher told us, the ruling will only hold in the geographical area covered by that particular court. There are no clear answers to this question.

The ambiguous legal state of closed-source modules may suit some developers who want to discourage them strongly without trying to ban them outright. But it is not beneficial for the Linux community as a whole. We really do not need more messy court cases; we have better things to do with our time and energy. As LWN has said before, it would be in the community's interest to clarify the situation and make the boundary clear.

One possible step in that direction was suggested by Al Viro. He has pointed out in the past that the module interface is a mess which could do with some cleaning up. Now he suggests looking at who actually uses each exported symbol and thinking about whether that symbol should really be made available to modules or not. There are, as he points out, over 7500 EXPORT_SYMBOL() declarations in the 2.6 kernel; seemingly, only about half of them are used by in-tree modules. A lot of these symbols, Al suggests, could probably go away altogether. Others could be explicitly exported only to certain modules with a clear need to use them - though the mechanism to restrict exports in this manner does not yet exist.

An effort to clean up the list of exported symbols could, eventually, lead to some clarity in the legal status of the users of those symbols. The export list could be trimmed down to a point where most observers could see it as a somewhat well-defined programming interface. At that point, maybe most developers might be able to agree on a subset that binary-only modules could safely use. The benefits of this agreement would be significant. There would be one minor disadvantage in that the kernel developers would have to find something else to argue about. One suspects that they are probably up to the task, however.

Comments (9 posted)