Multi-protection VMAs

The virtual memory area (VMA) structure (struct vm_area_struct) is one of the core building blocks of the Linux virtual memory code. Each VMA describes a piece of a process's address space; that piece is a (usually contiguous) series of pages from a single backing store (a file or, for anonymous memory, swap space) with a uniform set of access permissions. Each VMA maintains information on the address space covered, pointers to the backing store, permission information, a set of function pointers for operations on that VMA, and other housekeeping information.

Before the 2.6 kernel was released, all VMAs mapped a range of address space onto a contiguous range of pages in the backing store. Things got a bit more complicated with the addition of the remap_file_pages() system call, which allows applications to rearrange the mapping of memory pages to backing store pages within a VMA. That system call includes a parameter for setting the permissions of the remapped pages, but that parameter is currently ignored. For now, it is still true that all pages within a VMA carry the same page permissions. If an application tries to break that rule - by calling mprotect() on a subset of the pages within a VMA, for example - the VMA will be split into multiple VMAs, each of which imposes uniform permissions on its (reduced) part of the address space.

This behavior might just change however. Paolo Giarrusso has recently dusted off an old patch (developed with Ingo Molnar) which allows remap_file_pages() to change page permission as well. In theory, this change should be relatively straightforward. The page tables already hold the permissions for each page, so there is no need for any additional data structures to track the per-page permissions. The tricky part comes in when the page is swapped out. At that point, the kernel must take care to keep the permission information in the page table entry. A new VM_MANYPROTS VMA flag tells the kernel to use those saved permissions (instead of the permissions stored in the VMA itself) when the page is faulted back in.

To change page permissions, an application must pass the new MAP_CHGPROT flag to remap_file_pages(). Interestingly, the current patch does not support creating or operating on VM_MANYPROTS areas with mprotect(); there is, apparently, a disagreement over just what the semantics should be in that case.

The motivation behind this change is to improve performance for User-mode Linux. The UML code creates vast numbers (tens of thousands) of single-page mappings to simulate its own virtual memory environment. Each of those mappings creates a VMA. As the kernel works with all of those VMAs, memory-oriented operations slow down significantly. The memory overhead is also significant - each VMA requires at least 88 bytes of memory, 200 bytes on your editor's x86-64 system. Eliminating all of those VMAs can make UML much more efficient; Ingo Molnar reports that UML performance improves noticeably with the patch in place.

Ordinary Linux users could also benefit from this patch, however. Ulrich Drepper explained how the C library uses VMAs currently; it turns out that linking to a single shared library can create up to five separate VMAs. An application which brings in a large number of libraries - as many desktop applications do - can end up creating hundreds of VMAs for shared library mappings. That leads to many VMAs being created on the system; just how many can be seen by looking at the vm_area_struct line in /proc/slabinfo. Your editor's system currently has over 13,000 VMAs active, using about 2.5MB of memory.

Of the five VMAs potentially created by glibc for each shared library mapping, four are mappings into the same file with different permissions. The ability to have multiple permissions settings within a single VMA has the potential to collapse those four VMAs into one, leaving a single file mapping and an anonymous memory segment for each library. The result would be significantly reduced memory usage and faster kernel performance. Those benefits are likely to motivate the inclusion of this patch, sooner or later.

---

(Log in to post comments)