Page clustering [LWN.net]

The object-based RMAP patch is one approach to reducing the overhead of the virtual memory subsystem. William Lee Irwin has posted another: page clustering. Much of the VM subsystem's overhead is per-page; each page requires a memory map entry, possibly RMAP chains, etc. One way of reducing that overhead, clearly, would be to have fewer pages. Since most users will react poorly to suggestions that they remove memory from their systems, the only feasible way of reducing the page count is to make the pages themselves bigger.

The page clustering patch (based on work originally done by Hugh Dickins) works by taking physical pages (as seen by the hardware) and grouping them into larger, virtual pages as seen by the kernel. x86 hardware works (normally) with 4K pages; with page clustering the kernel can work with pages as large as 32K (according to the comments in page.h or 64K (according to what the code is actually doing). Thus, the page count (and associated overhead) can be reduced by a factor of up to 16.

This idea is not particularly new; early versions of BSD clustered the 512-byte pages provided by VAX systems into 1024-byte internal pages. Still, it's a bit tricky to implement inside the Linux kernel. Much kernel code thinks it understands the concept of the "page size," but, with this patch, there are two different page sizes. Code dealing with the hardware memory management unit (MMU) must work on the MMU's terms, while code working with kernel pages should see the larger size. The result is a great deal of work trying to figure out whether each bit of code should be working with PAGE_SIZE units, or the new MMUPAGE_SIZE. It is not a job for the faint of heart.

This patch is, for now, not for casual users; by William's admission a number of things are still broken. But, fear not: "I've yet to encounter non-fsck-recoverable filesystem corruption with remotely current sources." Even when the problems are fixed, this patch looks fairly involved for 2.5 at this point. But, one never knows.