Kernel development [LWN.net]

Brief items

Kernel release status

The current stable 2.6 release is 2.6.13.3, announced on October 3. It adds a handful of fixes, many in the networking subsystem.

The current 2.6 prepatch is 2.6.14-rc3, released by Linus on September 30. This prepatch is fairly large; most of the patches are small fixes, but there's also some key management improvements, a SCSI update, some netfilter patches, and an InfiniBand update. See the long-format changelog for the details.

Linus's git repository contains a relatively small number of fixes added after -rc3.

The current -mm tree is 2.6.14-rc2-mm2. Recent changes to -mm include the (temporary) dropping of a big set of PCMCIA patches, some memory management work, a workqueue change (uses per-CPU allocations now), and various fixes.

Comments (2 posted)

Kernel development news

Quotes of the week

visionary [n]: onanist with strong exhibitionist tendencies; from "visions", the source of inspiration they refer to when it becomes obvious that they have lost both sight and capacity for rational thought.

-- Al Viro

A "spec" is close to useless. I have _never_ seen a spec that was both big enough to be useful _and_ accurate.

And I have seen _lots_ of total crap work that was based on specs. It's _the_ single worst way to write software, because it by definition means that the software was written to match theory, not reality.

-- Linus Torvalds

Comments (3 posted)

Nested classes

The device model/sysfs "class" subsystem is a mechanism which allows different kernel subsystems to export device-independent interfaces to user space. With a recent kernel, a number of interesting class hierarchies can be found. For example, /sys/class/net represents all of the network interfaces in the system, /sys/class/sound shows the audio devices, and /sys/class/graphics can be used to find frame buffers.

[Network class diagram] The class API has changed little since it was documented in this LWN driver porting article. Kernel code registers a class structure to create a directory in /sys/class, then populates it with class_device objects. This API has worked for some time, but it has its limitations; it forces a two-level class->device structure which is unable to represent all of the relevant data structures in the kernel. For many class hierarchies, such as the network device class tree shown in the diagram to the right, two levels is sufficient. Other subsystems, however, have had trouble with this limitation.

[Block class diagram] Consider, for example, the block subsystem, as represented by the simplified diagram to the left. The block subsystem deals in block devices, of course, and those are represented in the second layer of the diagram. Each block device, however, can contain partitions, which are (virtual) block devices in their own right. Putting all of those partitions in the top layer of the block class hierarchy would lose the relationship between those partitions and the physical devices where they live; the deeper hierarchy truly does make sense. There are also other objects, such as the request queue, which need to be present in the class tree. The fact that the class subsystem cannot represent this structure is one of the reasons why the block layer has its own sysfs subtree, under /sys/block, even though it logically belongs under /sys/class.

This issue recently came to a head when Dmitry Torokhov reworked the input subsystem to make use of sysfs. The input class tree also fails to fit neatly into the class subsystem, though for slightly different reasons. The input layer can export multiple interfaces to the same device; a touch screen can show up as a serial device, as an event generator, or as a mouse, for example. Even a straightforward mouse can appear by itself, or as part of the multiplexed "mice" device.

As a way of representing the structure of the input subsystem, Dmitry implemented a "subclass" mechanism. Various objections to the implementation were raised, however, and Greg Kroah-Hartman went off to design a solution he liked better. His patch has now been posted for review; it is also part of the -mm tree.

Greg's solution does not involve subclasses at all; instead, the class_device structure has acquired a new parent field. The function which creates class_device structures has a new prototype:

    struct class_device *class_device_create(struct class *cls,
					     struct class_device *parent,
					     dev_t devt,
					     struct device *device, 
					     char *fmt, ...);

The parent argument is new. If it is non-NULL, the new class_device will be placed under the parent class_device in sysfs, rather than directly under the class itself. Needless to say, this change breaks all users of the class subsystem; if it goes into the mainline, all out-of-tree code using classes will have to be updated.

This interface should work reasonably well in the block case, where partitions can truly be thought of as child devices. Dmitry is less pleased with it for the input subsystem, however. He would like to be able to set up different hotplug handlers for lower-level entries, but, since those handlers are set up at the class level, an implementation without subclasses does not provide that capability. There are other objections as well; the parent mechanism makes it a little harder to set up the sort of hierarchy Dmitry would like to create, for example.

As of this writing, there has been no further discussion of the interface. There is a distinct chance that it could change before it makes its way into the mainline. In one way or another, however, support for a deeper /sys/class is likely to be merged.

Comments (none posted)

On the value of EXPORT_SYMBOL_GPL

When a loadable module is inserted, any references it makes to kernel functions and data structures must be linked to the current running kernel. The module loader does not provide access to all kernel symbols, however; only those which have been explicitly exported are available. The export requirement narrows the API seen by modules, though not by all that much: there are over 6,000 symbols exported in the 2.6.13 kernel.

Exports come in two flavors: vanilla (EXPORT_SYMBOL) and GPL-only (EXPORT_SYMBOL_GPL). The former are available to any kernel module, while the latter cannot be used by any modules which do not carry a GPL-compatible license. The module loader will enforce this distinction by denying access to GPL-only symbols if the module's declared license does not pass muster. Currently, less that 10% of the kernel's symbols are GPL-only, but the number of GPL-only symbols is growing. There is a certain amount of pressure to make new exports GPL-only in many cases.

It has often been argued that there is no practical difference between the two types of exports. Those who believe that all kernel modules are required by the kernel license to be GPL-licensed see all symbols as being implicitly GPL-only in any case. Another camp, which sees the module interface as a boundary which the GPL cannot cross, does not believe that GPL-only restrictions can be made to stick. In any case, GPL-only symbols can be easily circumvented by patching the kernel, falsely declaring a GPL-compatible license, or by inserting a shim module which provides wider access to the symbols of interest.

Linus, however, believes that GPL-only exports are significant.

I've talked to a lawyer or two, and (a) there's an absolutely _huge_ difference and (b) they liked it.

The fact is, the law isn't a blind and mindless computer that takes what you say literally. Intent matters a LOT. And using the xxx_GPL() version to show that it's an internal interface is very meaningful indeed.

One of the lawyers said that it was a much better approach than trying to make the license explain all the details - codifying the intention in the code itself is not only more flexible, but a lot less likely to be misunderstood.

He also points out that circumventing a GPL-only export requires an explicit action, making it clear that the resulting copyright infringement was a deliberate act.

Regardless of any legal significance they may have, the GPL-only exports do succeed in communicating the will of the large subset of the kernel development community which wants to restrict the use of non-free kernel modules. The outright banning of such modules may not be on the agenda anytime soon, but the functionality available to them is not likely to grow much.

Comments (27 posted)

The Linux Kernel Primer

Your editor recently received a copy of The Linux Kernel Primer, by Claudia Salzberg Rodriguez, Gordon Fischer, and Steven Smolski, published by Prentice Hall. This volume describes itself as "the definitive guide to Linux kernel programming"; it has chapters on processes, scheduling, I/O, filesystems, memory management, and the bootstrap process. It appears to be a guide to internal kernel APIs for the 2.6 kernel.

Reviewing kernel-related books is a difficult task. Your editor could easily be seen as having a conflict of interest in such cases, with any criticism viewed as an attempt to steer purchasers toward his own, possibly competing work. So, in the interests of full disclosure, let it be said: the author of this review is an author of a different, kernel-related book, and anything found here should be viewed with suspicion.

Because the simple fact is that your editor cannot recommend this book. It shows every sign of having been put together in a hurry, with basic grammatical errors being a frequent occurrence. The material is disorganized, with no clear ordering of concepts. Factual errors are not hard to find. The sample code provided is visibly buggy.

The book does not say, anywhere, which version of the kernel is covered - something any serious reader will want to know. Various hints through the text suggest that the authors were working from the 2.6.7 kernel at the latest, however, making the book somewhat obsolete before it hits the shelves. The version of struct file shown in the book is from 2.6.1; struct page comes from 2.6.4. The list of I/O schedulers does not include CFQ - added in 2.6.6.

The fundamental fault in this book, however, is this: there is no mention, anywhere, of concurrency issues. Even the few pages devoted to interrupts fail to mention race conditions or the primitives used to control interrupt delivery. Spinlocks and semaphores do not merit coverage until page 409 - and, even then, the API for working with them is not discussed. There is no way to write code for the 2.6 kernel without taking concurrency into account. Your editor cannot understand why the authors felt that this topic could be passed over.

More documentation for the kernel is a good thing. The kernel is a complex program, and kernel hackers can certainly benefit from a variety of views of how the kernel API works. In this case, however, your editor would recommend staying with the other books in this field, including Linux Kernel Development by Robert Love, and Understanding The Linux Kernel by Bovet and Cesati (third edition due in November).

Comments (4 posted)

Patches and updates

Kernel trees