Kernel development [LWN.net]

Release status

Kernel release status

The current stable 2.6 kernel is 2.6.14.3, released on November 24. This release contains a fair number of patches with important fixes.

The current 2.6 prepatch is 2.6.15-rc3, released by Linus on November 28. It consists mostly of fixes, as is appropriate at this stage of the kernel process, but there is also the VM_UNPAGED work discussed on last week's Kernel Page (somewhat reworked by Linus since then). See the long-format changelog for the details.

The current -mm tree is 2.6.15-rc3-mm1. Recent changes to -mm include some architecture updates, "trusted computing" BIOS measurement support, an MD update, the dynamic USB ID patch, some memory management tweaks, and a device mapper update.

Comments (none posted)

Kernel development news

Future Driver core changes

November 30, 2005

This article was contributed by Greg Kroah-Hartman.

Now that the 2.6.15 kernel is starting to stabilize, and the class device nesting code is looking pretty stable, it is time to start working on the future of the Linux driver core, with regards to the class and device structures. A month ago, I wrote a short summary of what I thought the future was going to look like . In this short article, I'm going to try to explain more about exactly what each of the steps that I described is going to entail.

The future of `struct class_device`

Eventually, the structure class_device is going to be merged into the device structure, and go away. But as this can not happen all at once, here are the different steps that I see happening to achieve this goal:

Try to move all users that individually create a struct class on their own, to use the class_create() and class_destroy() functions instead. In the future, struct class will become an "opaque" pointer, and only known by the class core code. No subsystems will be able to create this structure, except by doing it dynamically. By doing this step, we will remove the custom hotplug/uevent and release handlers for the classes, making it easier to convert these structures over to struct device in the future.
There are only around 20 different places in the current kernel that this will affect. The toughest of these will probably be the input and the network subsystems.
Remove all exported class functions in the driver core that are not being used by existing code. This will prevent any new code that gets added during the conversion process from using these functions.
Allow struct device to show up as attached to a struct class. This will entail adding a pointer to struct device. When this is done, the different users of struct class_device can be moved over to using struct device easily.
As the movement of the different classes are converted to struct device, start adding fields that are present in the struct class_device structure to struct device. At first glance, it looks like only the dev_t field (which holds the major/minor pair for the device) will need to be moved. All of the other fields in struct class_device look like they will match up with existing fields, but only by converting the different subsystems will this be known for sure.
struct class_interface is going to be "difficult". As only CardBus/PCMCIA and SCSI is using these functions and structures, how they are used there will have to be evaluated for how they can be converted away. The first thing that comes to mind would be a virtual bus for these types of devices, much like how the virtual USB Serial bus works. That would allow the PCMCIA and SCSI cores to still get notification of when a device is added or removed, which is what struct class_interface is used for.
A number of the things in /sys/class/ do not correspond directly with "real" devices that are currently in the /sys/device/ directory tree. The items in /sys/class/mem/ are examples of this type of device. For them, a new place in the /sys/device/ hierarchy will have to be found. Current ideas have moving them to the /sys/device/system/ tree, but others have proposed /sys/device/virtual/ instead, as they are not really "system" devices. Still others have said that it would be better to just put them all in the /sys/devices/ directory. It is unclear what will be the final resting place for these devices.

After all of these steps are complete, the last few holdouts of the class_device structure can be removed, and replaced with struct device, and then the class device structures and functions can finally be deleted.

The main point of this process is that it is going to happen one subsystem at a time, hopefully in such a manner that no users ever notice the difference.

Trees of symlinks

After the previously mentioned steps have been complete, the /sys/class/ directories will only contain subdirectories of symlinks back to the /sys/device tree. For example, for the 2.6.15 kernel release, the /sys/class/usb_host tree looks like:

/sys/class/usb_host/
|-- usb_host1
|   |-- device -> ../../../devices/pci0000:00/0000:00:1d.0
|   `-- uevent
|-- usb_host2
|   |-- device -> ../../../devices/pci0000:00/0000:00:1d.1
|   `-- uevent
|-- usb_host3
|   |-- device -> ../../../devices/pci0000:00/0000:00:1d.2
|   `-- uevent
`-- usb_host4
    |-- device -> ../../../devices/pci0000:00/0000:00:1d.3
    `-- uevent

Once the conversion process is done, it will look like this:

/sys/class/usb_host/
|-- usb_host1 -> ../../devices/pci0000:00/0000:00:1d.0/usb_host:usb_host1
|-- usb_host2 -> ../../devices/pci0000:00/0000:00:1d.1/usb_host:usb_host2
|-- usb_host3 -> ../../devices/pci0000:00/0000:00:1d.2/usb_host:usb_host3
`-- usb_host4 -> ../../devices/pci0000:00/0000:00:1d.3/usb_host:usb_host4

With the usb_host1 class device moving to:

/sys/devices/pci0000:00/0000:00:1d.0/usb_host:usb_host1
|-- device -> ../../0000:00:1d.0
`-- uevent

Note how the existing symlink in the device directory that points back to the class device (usb_host:usbhost1 is for the first usb host class device) is now a subdirectory in the device tree. Hopefully this will prevent any userspace program that is relying on the current structure of sysfs from breaking.

kobject/kset/subsystem/attribute diet

As anyone who has tried to understand the tangled web of interlocking pointers and dependencies between the kobject, kset, and subsystem structures and helper functions knows, a lot of work could be done here to make it simpler and easier to use and understand. Along with this, the attribute system of how to create files for kobjects in sysfs is quite complex. Any rework that can be done in this area, while ensuring that all of the current users of these core structures still work properly, will be done. As of this writing, no concrete plans for exactly what needs to be done here have been finalized.

Easier and more APIs

The current driver model is very flexible and powerful. Unfortunately along with this power and flexibility comes the ability to use it in incorrect ways very easily. The driver core does try to warn if somethings are not set up properly (like the lack of a release function), but it is still quite easy to get around these limited checks. So, on the Rusty scale of good kernel api levels, the driver core is very low on the list.

Again, like the kset maze, no concrete plans for exactly what will be done in this area have been finalized, but an example of what things might look like would be the current class_device_create() and class_device_destroy() functions. These functions push all of the nasty reference counting logic and class handling code into the driver core, and let the driver author worry about getting their driver specific logic correct. The driver author is no longer forced to become intimate with the driver core inner workings.

Better documentation

Due to the complexity of the current driver model code, much better documentation is needed to help developers who do want to use the core functions figure out how things should be done. The documentation that is in the current kernel tree (in Documentation/driver-model/) is woefully out of date. Hopefully the majority of this documentation can be moved to sit next to the driver core code itself, in kerneldoc format, which will help prevent any future changes from going undocumented.

Other subsystems have converted over to this format, with very great success, USB being one good example of this.

Out of tree subsystems

All of these changes will hopefully be done without breaking any userspace utilities (although, there will probably be a few udev updates needed along the way.) Any in-kernel code will be fixed up along the way, preventing any build or usage breakage, and then the unused structures and functions will be removed from the kernel tree. For driver subsystems that live outside of the main kernel tree, this means that their individual authors will have to update them to handle the new changes that are happening, or they can just submit them for inclusion in the main kernel tree, so that their code will be converted for them.

Comments (11 posted)

Understanding the Linux Kernel, 3rd Edition

It's official: the third edition of Understanding the Linux Kernel, by Daniel P. Bovet and Marco Cesati, is out. Your editor was pleased to receive a copy of this 900-page monster, delivered by a company which specializes in other sorts of heavy loads, such as pianos. UTLK 3 was some time in coming, but it is a welcome arrival.

As one would expect, this version of UTLK covers the 2.6 kernel. Your editor would like to point out to kernel-oriented publishers, however, that simply saying "2.6" is not particularly informative. A wide variety of kernels have come out under the 2.6 name. Readers will want to know which 2.6 kernel is covered by a given book, and they would rather not have to dig for that information. As it turns out, the reader who gets far enough into the introduction will discover that UTLK 3 was written for the 2.6.11 kernel.

This version of the book, like its predecessor, is big on details, and is very firmly tied to the i386 architecture. So, after a brief (and, in your editor's opinion, not entirely necessary) introduction to how Unix-like systems work, the book delves directly into i386 segmentation registers, global descriptor tables, and related details. Many of the key control paths in the kernel are described, step by step; often, the book includes flow charts, like the example on the left (which describes the page fault handling algorithm). The level of detail sometimes makes it hard to get a sense for the big picture, but it does help somebody trying to figure out how a particular function works.

Indeed, that is perhaps the key feature which differentiates this book. It is very much a "how it works" book, designed to help people understand the code. It is not, however, a "how to hack it" book like Linux Device Drivers or Linux Kernel Development. It presents kernel functions and data structures, steps the reader through them, but does not, for example, emphasize the rules for using them. UTLK is a study guide, not a programming manual.

But it is an effective and useful study guide. It covers a wide range of topics, including memory management, process management, scheduling, signals, the virtual filesystem, timing, the I/O layers, and more. Even with its weight, this book cannot cover everything, however; omitted topics include networking, security (security modules, key management, etc.), specific device drivers, sound, video, the kernel build system, and all of the other architectures supported by Linux. (For what it's worth, O'Reilly is said to have an "understanding the Linux network stack" book in the works now).

Certainly, there are things your editor would have done differently. There are some minor technical glitches; for example, the book claims that acquiring a semaphore always involves putting the acquiring process to sleep first, which is very much not the case. The discussion of sleeping starts with sleep_on(), and only later mentions that sleep_on() is not a recommended interface. The discussion of some interesting topics (direct I/O, for example) is overly short. But, as a whole, the book is excellent, and the kernel function index at the end helps to make it a useful reference. There is a space on your editor's "L1 bookshelf" (the one reachable without moving the chair) for UTLK 3.

Comments (5 posted)

Patches and updates

Kernel trees