Kernel development

Brief items

Kernel release status

The 2.6.26 merge window remains open, so there is no released 2.6 development kernel. See the article below for a summary of patches merged over the last week.

No stable kernel releases have been made over the last week. As of this writing, the 2.6.24.6 and 2.6.25.1 stable updates are in the review process; if all goes well, these updates should be released on May 1.

Comments (none posted)

Kernel development news

Quotes of the week

Those who have been watching the linux-kernel list know that the 2.6.26 merge window has been a little rougher than some of those which came before. That has led to some fairly strong discussion over how changes find their way into the mainline. Here's a few selections.

-- James Bottomley

-- Ingo Molnar

-- David Miller

-- Alan Cox

-- Andrew Morton

-- Rafael Wysocki

-- Linus Torvalds

Comments (4 posted)

The 2.6.26 merge window, part 2

Since last week's summary was written, another 3700 changesets have found their way into the mainline git repository. The most significant user-visible changes include:

• New drivers have been merged for Wolfson WM9713 codecs, TI DAVINCI AC97 sound chips, Emagic Audiowerk 2 soundcards, x86 PC speakers (new driver which makes them look like sound cards), Asus AV100 (Xonar DX) sound cards, Micron MT9M001 and MT9V022 cameras, PXA27x Quick Capture cameras, Kworld ATSC 120 tuners, cx23417 MPEG encoders, Integrant ITD1000 tuners, Philips TDA10048HN-based demodulators, Philips SAA7171/3/4 audio/video decoders (the last out-of-tree IVTV driver), Auvitek AU8522 demodulators, Samsung S5H1411-based tuners, framebuffer, keyboard, and mouse virtual devices (for Xen), several Wolfson Microelectronics touchscreens, wireless Xbox 360 controllers, Zhen Hua PPM-4CH transmitters, SPCP8x5 USB to serial adaptors, NCR 53c9x SCSI controllers (replacement driver), Freescale 8610 and 5121 display interface units, Intel 965G/965GM integrated graphics controllers, TI OMAP sound controllers (including the one on the Nokia 810), Eee PC function keys, and Intel IXP4xx Ethernet devices.

• There is now "basic" support for braille screen readers.

• Support for the One Laptop Per Child XO architecture has been merged into the mainline.

• The new virtual files found in /proc/pid/mountinfo provide information on all filesystem mounts visible to the relevant process.

• The new virtual file /proc/vmallocinfo displays information on use of vmalloc space within the kernel.

• The SPARC Niagara architecture now has NUMA support.

• The Xen balloon driver (allowing memory to be added to or removed from virtual guests) has been merged.

• By default, /dev/mem can no longer be used to access RAM; Fedora and Red Hat have applied this patch for years, but now it has found its way into the mainline.

• The KVM paravirtualization subsystem now supports the S/390, PowerPC 440, and ia64 architectures.

• Per-process "securebits" are supported. These bits control how a process's capability bits are managed; the patch is intended to help those who would transition over to a fully capability-based system. See this article for a more detailed description of this feature.

• The getrusage() system call has a new RUSAGE_THREAD option which causes it to return information about the current thread only.

• The device whitelist control group patch (described briefly in this article) has been merged.

• It is now possible to create and use partitions with network block device (NBD) devices.

• The audit subsystem can now test events against the type of the file being operated upon.

• The VFS now makes backing device information available under /sys/class/bdi. Interested people can look at per-device readahead and writeback variables there.

• The FUSE filesystem now supports the creation of shared writable memory mappings.

Changes visible to kernel developers include:

• ioremap() on the x86 architecture will now always return an uncached mapping. Previously, it had taken a more relaxed approach, leaving the caching as the BIOS had set it up. The practical result was to almost always create uncached mappings, but with occasional exceptions. Drivers which depend on a cached mapping will now break; they will need to use ioremap_cache() instead.

• The Video4Linux2 API now defines a set of controls for camera devices; they allow user space to work with parameters like exposure type, tilt and pan, focus, and more.

• On the x86 architecture, there is a new configuration parameter which allows gcc to make its own decisions about the inlining of functions, even when functions are declared inline. In some cases, this option can reduce the size of the kernel's text segment by over 2%.

• The legacy IDE layer has gone through a lot of internal changes which will break any remaining IDE drivers.

• The nopage() virtual memory area operation has been removed; all in-tree code is now using fault() instead.

• The SLUB allocator supports a new sysfs file (/sys/kernel/slab/name/order) which allows system administrators to change the size of page allocations used by the named slab.

• A condition which triggers a warning from WARN_ON will now also taint the kernel.

• The get_info() interface for /proc files has been removed. There is also a new function for creating /proc files:

      struct proc_dir_entry *proc_create_data(const char *name, mode_t mode,
  					    struct proc_dir_entry *parent,
  					    const struct file_operations *proc_fops,
  					    void *data);
  

  This version adds the data pointer, ensuring that it will be set in the resulting proc_dir_entry structure before user space can try to access it.

• The object debugging infrastructure has been merged.

The merge window remains open; tune in next week for (what should be) the final set of changes merged for 2.6.26.

Comments (2 posted)

Restricting root with per-process securebits

Linux capabilities have had a long and somewhat tortuous journey as part of the Linux kernel. Slowly—and very carefully—functionality is being added to this security feature to get it to a point where it is a viable alternative to the all-or-nothing setuid(0) model. A recently merged patch adds a per-process securebits feature that will allow capabilities-based daemons or subsystems to coexist with existing setuid utilities.

Linux capabilities break up the privileged tasks normally associated with root (i.e. uid 0) into finer-grained abilities which can be individually granted or revoked for specific processes. The idea is to change the standard Unix model that root has all special privileges while all other users have none. The terminology is always a bit contentious, though, as Linux capabilities are derived from a POSIX proposal that was never adopted, but shares the name "capabilities" with an entirely different approach; this article is only concerned with capabilities of the Linux variety.

There has long been interest in creating a Linux system that did not rely upon a single root account. Capabilities are seen as the way to get there, but they have suffered from a bit of a chicken-and-egg problem. With the recent work to add file-based capabilities and restore CAP_SETPCAP to its original meaning, a true capabilities-based system is becoming possible. In the patch, which has been merged for 2.6.26, Andrew Morgan describes the new functionality:

The patch removes the global securebits variable, replacing it with an entry in struct task_struct, that can be manipulated by a process, but only for itself—and any children. Morgan envisions hybrid systems that have some utilities using capabilities to get their privileges along with some setuid(0) utilities. In that scenario, a capabilities-based utility or daemon may wish to limit what its children can do, even if they execute a setuid(0) binary. As part of the evolution, process trees can be created that cannot get root privileges.

Processes which have the CAP_SETPCAP capability can change their securebits setting via the prctl() system call. There are three separate bits that govern the interaction of capabilities and setuid:

• SECURE_NOROOT – enabling this gives no special privileges to uid 0
• SECURE_NO_SETUID_FIXUP – setting this bit disables capability fixes when transitioning from or to uid 0 via setuid. This might be done for compatibility with older programs that use setuid to reduce their privileges.
• SECURE_KEEP_CAPS – when set, a process can retain its capabilities even when transitioning to a normal (not uid 0) user. This bit is cleared by exec().

Each of these bits also has a companion *_LOCKED bit that, if set, will not allow any user program to alter the corresponding setting. As Morgan notes in the patch, a program that can set its capabilities (has CAP_SETPCAP) can drop all privileges for itself and any child process by doing:

    prctl(PR_SET_SECUREBITS, 0x2f);

This is the equivalent of setting SECURE_NOROOT, SECURE_NO_ROOT_LOCKED, SECURE_NO_SETUID_FIXUP, SECURE_NO_SETUID_FIXUP_LOCKED, and SECURE_KEEP_CAPS_LOCKED.

The memory of the sendmail-capabilities bug from 2000 makes some a bit queasy—or worse—about any patches that involve capabilities and setuid. Andrew Morton asks: "what was the bug which caused us to cripple capability inheritance back in the days of yore? (Some sendmail thing?)" That bug was caused because unprivileged users could take away the CAP_SETUID capability from setuid binaries like sendmail. When sendmail then used setuid to drop its privileges, it failed, but sendmail did not check, so it was still running with full privilege. This could be leveraged by a user to gain root privileges. It was a disconnect between capabilities and the longstanding behavior of Unix-like systems when dropping privileges.

Morgan has written a detailed description of the sendmail-capabilities bug in response to Morton's questions. He makes it clear that he wants to move toward full capability support without breaking existing code:

Morton seemed satisfied that his concerns had been addressed, but still wonders about the future for capabilities: "So how do we ever get to the stage where we can recommend that distributors turn these things on, and have them agree with us?" This was echoed by Ismail Dönmez, who was looking for concrete examples of how to use the per-process securebits feature. Morgan provides a pointer to some examples along with his belief that sometime soon the capabilities developers will become confident enough to recommend turning off the "experimental" flag for the SECURITY_FILE_CAPABILITIES kernel configuration. That flag governs both the file-based capabilities as well as the per-process securebits. In addition, Morgan says:

A developerWorks article on file-based capabilities by Serge Hallyn and a web page on POSIX capabilities by Chris Friedhoff were both mentioned in the thread as good references for the work being done to actually use capabilities in systems. Those pre-date the securebits work, so Dönmez was looking for use-cases for the new feature. Morgan replied that containers were one, deferring to Hallyn who has some ideas on using securebits:

Capabilities are an interesting, but complicated, security feature. For most of the ten years they have been part of the Linux kernel, they have either been broken, ignored, or both. With the latest work being done by Hallyn, Morgan, and others, capabilities are finally becoming a fully-working alternative to things like SELinux. It will be interesting to see if more user utilities will become capability-aware and whether distributions start using capabilities. Some day, root may just fade away.

Comments (4 posted)

Ksplice: kernel patches without reboots

The kernel developers are generally quite good about responding to security problems. Once a vulnerability in the kernel has been found, a patch comes out in short order; system administrators can then apply the patch (or get a patched kernel from their distributor), reboot the system, and get on with life knowing that the vulnerability has been fixed. It is a system which works pretty well.

One little problem remains, though: rebooting the system is a pain. At a minimum, it requires a few minutes of down time. In many situations, that down time cannot be tolerated. Reboots also disrupt any ongoing work, break existing network connections, and can cause the loss of results from long-running processes. And, most importantly of all, reboots prove traumatic for a certain subset of Linux administrators who prize a long uptime above almost all other things. Administrators currently have to choose between multi-year uptimes and security fixes; anything which frees them from a dilemma of this magnitude can only be welcome.

That "anything" might just be a recently-announced project called ksplice. With ksplice, system administrators can have the best of both worlds: security fixes without unsightly reboots.

An in-depth explanation of how ksplice works can be found in this document [PDF]. In short, ksplice requires as input the source tree for the running kernel and the security patch. It will then build two kernels, one with the patch and one without; the kernels are built with a special set of options which makes it easy to figure out which functions change as a result of the patch. The two kernels will be compared, with the purpose of finding those functions. Changes can propagate further than one might expect, especially if, for example, an inline function is modified.

Once a list of changed functions has been made, the updated code for those functions is packaged into a kernel module and loaded into the system. Then comes the tricky part: getting the running kernel to start using the new code. That requires patching the running code, which is a risky thing to do. Ksplice starts with a call to stop_machine_run(), which dumps a high-priority thread onto each processor, thus taking control of all processors in the system. It then examines all threads in the system to ensure that none of them are running in the functions to be replaced; if so, trampoline jumps are patched into the beginning of each replaced function (they "bounce" the call to the old code into the replacement code) and life continues. Otherwise ksplice will back off and try again later.

This method imposes a number of limitations. One is that only code changes can be patched in with ksplice; patches which make changes to data structures cannot be accommodated. Another comes from the retry-based approach to ensuring that no threads are running in the patched functions; what happens if one of those functions is never free? Kernel functions like schedule(), sys_poll(), or sys_waitid() are likely to always have processes running within them. In cases like this, ksplice will eventually give up and inform the user that the patch cannot be done; it is simply not possible to make changes to those particular functions.

These limitations mean that, out of 50 security patches examined by the ksplice developers, eight could not be applied with ksplice. So multi-year uptimes are probably still incompatible with the application of all security patches. Even so, ksplice certainly has the potential to reduce patch-related downtime considerably. Chances are good that there will be a fair amount of interest in ksplice in sites running high-uptime, mission-critical systems.

There are few things in the way of an immediate merge of this code into the mainline. One is a matter of coding quality and can be fixed. Then, there is the matter of the lead developer being unconvinced that merging this code makes sense since it is, essentially, a standalone feature. Andi Kleen's response made the (usual) reasons for merging the code clear:

So, presumably, the code will eventually be proposed for a mainline merge. But there is one other little difficulty pointed out by Tomasz Chmielewski: Microsoft holds a patent described this way:

Microsoft came up with this novel new technique in the distant past: 2002. The posting immediately brought out a crowd of surprised graybeards who distinctly remember using such techniques on their PDP-11 systems some decades before Microsoft "invented" hot-patching. The basic claim of the patent would thus appear to be invalidated by some decades' worth of prior art, but some of the dependent claims include features (such as capturing all other processors on the system) which were unlikely to be useful on PDP-11s.

Given that the kernel developers are now well aware of this patent, they must take it into account when deciding whether to accept this code into the mainline. It would not be surprising if they chose to avoid baiting the Microsoft FUD machine in this way, even if they all agreed that the patent lacked validity. So a promising technology risks being left out of the kernel as the result of a software patent which was filed at least 30 years too late.

Comments (64 posted)

Patches and updates

Core kernel code

• Ulrich Drepper: alternative^2 to sys_indirect: socket, socketpair. (April 25, 2008)

• Ulrich Drepper: paccept, socket, socketpair w/flags. (April 26, 2008)

• Ulrich Drepper: eventfd, signalfd, timerfd, epoll_create w/flags. (April 28, 2008)

• Ulrich Drepper: dup2 w/flags. (April 28, 2008)

• Ingo Molnar: generic bitops, take 3. (April 26, 2008)

• Steven Rostedt: New read/write locks for PI and multiple readers. (April 28, 2008)

• Nadia.Derbey@bull.net: Scalability requirements for sysv ipc - v2. (April 29, 2008)

Development tools

• Ingo Molnar: kmemcheck. (April 25, 2008)

• Vegard Nossum: [PATCH] headerdep: a tool for detecting inclusion cycles in header file. (April 26, 2008)

• Ingo Molnar: ftrace, v15. (April 27, 2008)

• Mathieu Desnoyers: Linux Kernel Markers instrumentation for sched-devel.git. (April 28, 2008)

• Mel Gorman: Verification and debugging of memory initialisation V4. (April 29, 2008)

• Pekka Paalanen: mmiotrace full patch, preview 3. (April 29, 2008)

Device drivers

• Dave Airlie: drm patches for 2.6.26-rc1. (April 26, 2008)

• Dave Airlie: agp patches for 2.6.26-rc1. (April 26, 2008)

• Len Brown: ACPICA patches for 2.6.26-rc0. (April 28, 2008)

• James Bottomley: another tranche of SCSI updates for 2.6.26. (April 28, 2008)

• Greg KH: USB patches for 2.6.25-git. (April 28, 2008)

• Bjorn Helgaas: PNP cleanup, v4. (April 28, 2008)

• Bartlomiej Zolnierkiewicz: IDE updates part 2. (April 28, 2008)

• Bartlomiej Zolnierkiewicz: IDE updates part 3. (April 28, 2008)

• Bartlomiej Zolnierkiewicz: IDE updates part 4. (April 28, 2008)

• Bartlomiej Zolnierkiewicz: IDE updates part 5. (April 29, 2008)

• Sascha Sommer: MMC/SD host driver for Ricoh Bay1Controllers. (April 28, 2008)

• Jean Delvare: i2c: Add support for device alias names. (April 28, 2008)

• David Brownell: gpio: sysfs interface. (April 29, 2008)

Documentation

• Kok, Auke: 82598 Software Developer Manual 2.2 released. (April 26, 2008)

Filesystems and block I/O

• Al Viro: VFS patches. (April 26, 2008)

• Bob Copeland: OMFS filesystem version 5. (April 26, 2008)

• Ryo Tsuruta: [PATCH 0/2] dm-ioband: I/O bandwidth controller v0.0.4: Introduction. (April 28, 2008)

• Miklos Szeredi: vfs: add helpers to check r/o bind mounts. (April 28, 2008)

• Evgeniy Polyakov: POHMELFS high performance network filesystem release.. (April 28, 2008)

• Josef Bacik: fiemap support for ext3. (April 28, 2008)

• FUJITA Tomonori: add large command support to the block layer. (April 28, 2008)

• Martin K. Petersen: Block/SCSI Data Integrity Support. (April 28, 2008)

• Aneesh Kumar K.V: ext4: Group meta-data blocks together.. (April 29, 2008)

• Theodore Ts'o: ext4 update. (April 30, 2008)

• Chris Mason: Btrfs v0.14 Released. (April 30, 2008)

Janitorial

• Matthew Wilcox: Announce: Semaphore-Removal tree. (April 26, 2008)

Memory management

• Andrea Arcangeli: mmu notifier #v14. (April 26, 2008)

• Rusty Russell: Minimal mmu notifiers for kvm. (April 28, 2008)

• Erez Zadok: VM: vm_ops->fault takes explicit struct file. (April 28, 2008)

• Pekka J Enberg: SLUB/SLOB updates for 2.6.26. (April 28, 2008)

• Rik van Riel: VM pageout scalability improvements (V6). (April 29, 2008)

Networking

• Shigeo N: XTP for 2.6.25. (April 28, 2008)

Architecture-specific

• Dean Nelson: allocate multiple contiguous pages via uncached allocator -v3. (April 26, 2008)

• Ingo Molnar: "big box" x86 changes. (April 26, 2008)

• Ingo Molnar: StackProtector enhancements and fixes. (April 28, 2008)

• Dave Airlie: x86: create array based interface to chagne page attribute. (April 28, 2008)

• Markus Metzger: x86, ptrace: in-kernel BTS interface. (April 30, 2008)

Security-related

• David P. Quigley: Introduce generalized hooks for getting and setting inode secctx.. (April 28, 2008)

• Al Viro: audit patches. (April 30, 2008)

Virtualization and containers

• Avi Kivity: kvm-67 release. (April 28, 2008)

• Matt Helsley: Container Freezer: Reuse Suspend Freezer. (April 28, 2008)

• Benjamin Thery: sysfs tagged directories. (April 29, 2008)

• YAMAMOTO Takashi: another swap controller for cgroup. (April 30, 2008)

Miscellaneous

• Karel Zak: util-linux-ng v2.14-rc2. (April 28, 2008)

• Sebastien Tricaud: Wolfotrack 1.0 released. (April 30, 2008)

Page editor: Jonathan Corbet
Next page: Distributions>>