128-Bit page tables for Arm

The information of most interest in a PTE is a physical address — the address of the actual memory for bottom-level entries, or the address of the next table for higher levels. Not all of the bits in the PTE hold that address, though; if nothing else, the lowest bits, which correspond to the offset within the page in a virtual address, can be used for other purposes. Our article "On pages and folios" describes PTE entries in some detail; this diagram, showing how a 64-bit PTE might be divided, is taken from that article:

On a modern, 64-bit Arm system configured for 56-bit addresses and 4096-byte pages, a maximum of 44 bits of the PTE can be used to represent the physical address of a page in memory, while 12 bits are available for housekeeping purposes.

Expanding a PTE to 128 bits obviously makes it possible to store more information in that entry, but at a cost: doubling the size of PTEs will, clearly, double the size of the page tables. For some workloads, page tables already occupy more memory than users would like; every page occupied by a page table is unavailable for use by the system's workloads. Since one page can only hold half as many PTEs as before, there are implications for huge pages; a 2MB PMD-level huge page on a 64-bit-PTE system drops to 1MB when PTEs grow to 128 bits, and PUD-level huge pages shrink from 1GB to 256MB — a factor of four, since there are two levels of page tables that are eliminated for those pages. The sizes of multi-size transparent huge pages that can be coalesced for faster translation lookaside buffer (TLB) access also drop by similar factors. For workloads that are heavily dependent on huge pages for performance, those reductions could prove painful.

Given that there are clear costs involved with going to larger PTE sizes, one would expect that there would be gains in other areas; otherwise, there would be little point in making the change. To understand those gains, it is worth looking more closely at the new format. The details of what 128-bit Arm PTEs look like can be seen on this page; the bottom-level PTE format looks like this:

One thing that jumps out immediately is that the address portion of the PTE, stretching from bits 12 to 55, is only 44 bits in length. Add in the 12 offset bits, and the maximum address size is 56 bits, the same that is currently possible with 64-bit PTEs. So expanding the range of physical memory that can be addressed is not an immediate goal of this change. That said, it is worth noting all of the reserved bits — the bits shown in gray and without labels. There are 35 reserved bits above the most-significant address bit, leaving quite a bit of room for future expansion of the address field.

There are ten bits of the PTE that are not interpreted by the hardware at all; they are reserved for software use. That is double the five bits that are available on Arm systems with 64-bit PTEs. Five extra bits may not seem like a lot, but the kernel should eventually be able to make good use of them for memory management.

Bits 109 and 110 are the "skip level" (SKL) field. This field, which is present at all page-table levels, controls how the page-table-traversal process works. On most systems, the page-table hierarchy is rigid, with the expected number of levels (usually between three and five) always present. The one exception is for huge pages, where one or more levels are removed at the end of the search. Consider the PMD, which sits one level above the final page-table level; one entry in the PMD points to an entire page of PTEs, and covers (typically) 2MB of address space. By setting a bit in the PMD entry, though, the kernel can indicate that the entire 2MB has been assigned in a single page — a PMD-level huge page.

The SKL field generalizes this mechanism somewhat; it can indicate that the following step in the lookup process is a huge page, even if it is another level in the page-table hierarchy rather than the final step. That feature would appear to open the door to using huge pages for the page tables themselves, which would have the potential of speeding address translation and reducing TLB use.

The kernel changes required to support 128-bit PTEs are relatively small, consisting mostly of a set of macro definitions describing where the various fields and bits are. The most disruptive part has to do with page-table manipulation. When working through entries at various levels of the page-table hierarchy, it is important to read them atomically, so that the entire entry is coherent. That reading is done with READ_ONCE() in current kernels, but READ_ONCE() depends on atomic operations, and Arm CPUs do not have 128-bit atomic operations. Similar concerns apply to the modification of PTEs as well. The patch series wraps these accesses with a new pair of functions (ptval_get() and ptval_set()) that default to READ_ONCE() and WRITE_ONCE(), but which can be overridden by the architecture if need be. For Arm systems built for 128-bit PTEs, those functions are implemented with atomic ldp and stp (load-pair and set-pair) instructions.

In the current implementation, at least, the kernel must be specifically built for 128-bit PTEs, and kernels built that way will be unable to boot on CPUs that do not support that PTE size. That will make life harder for distributors; they have strong incentives to keep the number of kernels they build to a minimum. Fixing this problem will almost certainly involve a fair amount of boot-time code patching, but that may be required before distributors will consider enabling support for the larger PTEs.

This series has been through a few RFC iterations and appears to be stabilizing somewhat. Of course, relatively few people have hardware that supports 128-bit PTEs, so the amount of in-the-field testing of this work is necessarily limited. By the time that hardware becomes more widely available, though, Linux may well be ready to support it. But, in the short term, it is not at all clear that many systems will benefit from making use of this feature.