Kernel development

Kernel release status

The current 2.6 prepatch is 2.6.12-rc4, announced by Linus on May 6. Changes this time around include more "sparse" annotations, a CIFS update, various architecture updates, resource limits for niceness and realtime scheduling (covered in last week's Kernel Page), a JFS update, some networking tweaks, and more. See the long-format changelog for the details.

Linus is currently on vacation, so no new patches have been added to his git repository since -rc4.

The latest -mm release is 2.6.12-rc3-mm3. Recent changes to -mm include a rework of the huge page code, a bunch of UML updates, a device mapper update, and more fixes.

Comments (3 posted)

The coding style enforcer

        if (condition) do_this;
          do_something_everytime;

Jesper Juhl recently found some code which evidently had something to hide, and submitted a patch to break the offending if statements onto two lines. Andrew Morton rejected it:

In further discussion, however, Andrew seemed to agree that, perhaps, cleaning up the kernel source to be more generally compliant with the coding style documentation might be a good thing. He just doesn't want to cope with hundreds of little patches to that end. He will, however, consider a small number of very large patches.

So a major coding style cleanup seems likely to happen, perhaps before 2.6.12 comes out. Applying this sort of patch so late in the cycle should be safe; the intent is to change the formatting, but to make no actual code changes. Andrew also plans to drop any changes which do not apply against the -mm tree, in the hopes of minimizing the effects of the changes on patches maintained by other developers.

If all goes according to this plan, the final 2.6.12 patch could be large indeed.

Comments (10 posted)

The mini_fo filesystem

At its core, mini_fo performs a simple fan-out operation. Each inode, dentry, and file structure associated with a mini_fo filesystem contains (via its private data) pointers to two other structures of the same type. One of them refers to the file or directory on the base filesystem; the other, instead, is for a local version of the file or directory on a local "storage filesystem." Both are hidden from user space, which thinks it is dealing directly with a file stored in the mini_fo filesystem.

When a mini_fo filesystem is first created, it appears as an exact copy of the underlying base filesystem. Any operation which reads files or directories is simply passed through to the base filesystem, with almost no additional overhead. In this mode, mini_fo functions as a sort of loopback filesystem.

Things change, however, when a file is opened for writing. In this case, mini_fo will create a copy of the file on the storage filesystem, with all of the data moved over. Any subsequent operations on that file will used the locally-stored version rather than the base version. So any changes made will appear locally, but they will not be propagated back to the base. Changes will be persistent across mounts as long as the storage directory used by mini_fo is not modified by anything except mini_fo.

Modified files are not the full story, of course; mini_fo must also cope with operations like deletes and renames. To that end, it maintains a set of lists of files which it knows about locally; there is one list for modified files, one for deleted files, one for files created locally, etc. These lists are stored in-kernel as standard linked lists. They are also written to the storage filesystem in a magic file (named META_dAfFgHE39ktF3HD2sr, for what it's worth) and reloaded from that file when the filesystem is mounted.

This release of mini_fo works with both the 2.4 and 2.6 kernels. Its author claims that it is intended for use with embedded systems, and thus has a small memory footprint. See the mini_fo web page for more information.

Comments (10 posted)

A system call for unsharing

In the current Linux kernel, the sharing decisions made at clone() time last for the lifetime of the processes involved. There is not usually a reason to change resource sharing, but recent discussions on supporting private mounts (with the filesystems in user space patch, or otherwise) have suggested that it would actually be useful for a process to be able to "unshare" resources after its creation. In particular, if a process could detach itself from the global filesystem namespace and create its own, it would be possible to set up that new namespace with whatever private mounts that process needs. If this functionality were used within a PAM module, it would be relatively easy for administrators to set up per-user views of the filesystem, complete with private mounts.

To that end, Jenak Desai has posted a patch adding a new unshare() system call. The interface is simple enough:

    long unshare(unsigned long flags);

The flags argument can be CLONE_NEWNS (to create a new filesystem namespace), CLONE_VM (to establish a private virtual address space) or CLONE_SIGHAND (to unshare signal handlers). If all goes well, when the call returns, the designated resource(s) will now be private to the calling process; otherwise the situation is unchanged.

This patch has not yet made it to the linux-kernel mailing list, and may see some changes before it is considered for inclusion.

Comments (none posted)

Execute-in-place

Carsten Otte (of IBM) has posted a set of patches adding XIP support to the 2.6 kernel. These patches, in addition, enable fast memory-to-memory block I/O for such devices, shorting out the page cache and most of the block layer. As a result, the XIP patches are useful in a number of situations, such as, as Carsten notes, for shared-memory block devices used to communicate between (virtual) systems.

The first step is to add support at the block driver level. To that end, a new method is added to the block_device_operations structure:

    int (*direct_access) (struct inode *inode, sector_t sector, 
                          unsigned long *data);

This method, if implemented, should come up with a kernel virtual address corresponding to the given sector on the block device represented by inode. That address, which must remain valid until the device is closed, is returned in *data. The return value is zero on success or a negative error code in case of problems.

The next step is a new method in the address_space_operations structure:

    struct page *(*get_xip_page)(struct address_space *space, 
                                 sector_t blockno, int create);

This method's job is to translate a specific block number within a filesystem to a page structure pointing to its directly-mapped data. It is a filesystem-specific function which will translate blockno to a sector number on the underlying device, then use that device's direct_access() method to get an address. Carsten has posted an implementation for ext2 which shows how this method can be put together.

So far, the XIP patches enable fast, memory-to-memory device access, but they do not yet implement true execute-in-place operation. The last step is to replace the usual nopage() VMA operation (filemap_nopage()) with a new version (filemap_xip_nopage()) when the underlying device and filesystem support XIP. The new nopage() method will (using get_xip_page()) handle page faults by causing a process's page tables to point directly to the on-"disk" pages, rather than reading those pages into RAM. Some other technique will be needed to run the kernel itself in an XIP mode, but anything that is invoked thereafter can be run directly from the memory device.

Put the above pieces together, and Linux has a complete execute-in-place implementation. Supporting XIP at the block level is not the only way it could be implemented; David Woodhouse pointed out that an alternative approach is to use a special-purpose filesystem. Carsten's patches, however, point out a way in which any filesystem could be made to work in an XIP mode.

Comments (10 posted)