Are 1.7% really worth it? [LWN.net]

Are 1.7% really worth it?

Posted May 11, 2026 21:38 UTC (Mon) by PeeWee (subscriber, #175777)
Parent article: Providing 64KB base pages with 4KB kernels, two different ways

Using 64KB base pages, Shutsemau began, can provide a 1.7% performance improvement on "a very important workload" on Arm processors; he would like to bring that speedup to x86 systems as well.

Given all the ifs and buts that followed, according to this article, I cannot help but wonder if it's really worth it, especially when the performance improvement is only part of the equation; the other being internal fragmentation / memory waste. I think, at some point, one needs to ask, how many instances of such an "important workload" could be run with 4K pages vs. 64K pages. If one can run 100 instances with 4K pages without swapping but only 80 with 64K, due to underutilized pages, that's a net loss; assuming perfect scalability with instance count, of course. What about other workloads that need to coexist on such systems? If any of them need to make do with less available memory, that goes to the cost side of the balance sheet.

Also, why not aim lower? I've read that MacOS is using 16K pages, which seems like a reasonable compromise. It gives four times the page table coverage compared to 4K.

I'm also wondering if CPU vendors are simply cheaping out here. Maybe we need some new MMU hardware that's capable of dealing with modern memory demands. I mean, very major portions (> 30%) of silicon real estate is spent on Cache Memory, but so little on TLB?!

Are 1.7% really worth it?

Posted May 12, 2026 8:35 UTC (Tue) by linusw (subscriber, #40300) [Link]

At the same time dynamic stacks saving 1-2% of system memory is pretty much shot down. I'm not saying performance and RAM footprint are immediately comparable. But we are perhaps hitting a wall of diminishing returns.
https://lore.kernel.org/linux-mm/da9321ad-4198-494e-b9fa-...

Are 1.7% really worth it?

Posted May 13, 2026 8:20 UTC (Wed) by joib (subscriber, #8541) [Link]

I'm not a kernel developer, but seems to me a better approach would be to finish the mthp/folio/memdesc work, and then one could see to which extent this kind of "virtual" larger base page size would still be useful or needed?

As for the size of this new base page, IIRC some newer x86 CPU's have "contiguous PTE" support for 8 consecutive pages, or 32kB, so maybe that would be a sweet spot?

Internal fragmentation

Posted May 15, 2026 17:24 UTC (Fri) by anton (subscriber, #25547) [Link] (1 responses)

I made some measurements of how much internal fragmentation would grow with various page sizes and posted it on Usenet <2020Oct9.190337@mips.complang.tuwien.ac.at> Given that I no longer know a working Usenet archive, I just repost this work here:

I use the following script:

cat /proc/[1-9]*/maps|
awk '$2~/.w.p/||!$2":"$6 in m {m[$2":"$6]=1; print $1" swap - doit"}'|
sed 's/-/ /'|
gforth -e 'variable mem create ps $2000 , $4000 , $8000 , $10000 , create sums 0 , 0 , 0 , 0 , : doit 4 0 do dup ps i th @ naligned over - sums i th +! loop mem +! ; hex stdin include-file decimal : printit mem @ $400 / 8 .r 4 0 do sums i th @ $400 / 8 .r loop ; printit cr bye'

On my desktop this outputs the second of the following lines:

 total       8KB    16KB    32KB    64KB
 1040972    6676   22244   56148  124788

The 8KB column tells how many extra KB would be used if the page size
was 8KB.  Comparing total memory as determined by this script to the
output of free (555900 without buffers/cache) indicates that, on
average, only about half of the address space in a VMA is consumes
memory; so there is also quite a bit of uncertainty about my estimates
for extra memory.

For the three machines above [desktop (2h uptime, spartan user interface);
 laptop (54d uptime, Ubuntu, Gnome, 2 users); server (89d uptime, Debian,
 Gitlab and Jitsi in Docker)], I get today (numbers in KB):

VMAs unique    used     total      8KB    16KB    32KB    64KB   
 7552  2333    555964  1033320    6704   22344   56344  125144 desktop
82836 25276   5346060 15707448   76072  223000  514472 1113672 laptop
47017 15425 105490636 60186068   40804  134492  319852  708588 server

I have no explanation why used is higher than total on the server.

If you want to get an average VMA size, just divide total by unique.

In any case, even with the considerable uncertainty from partially
populated VMAs, the extra cost of 8KB or 16KB pages does not appear to
be large, even on the laptop.

At current RAM prices, not everybody will want to spend an extra 1.1GB on 64KB pages for internal fragmentation, but having the option would be nice, and having some in-between option, too.

L1 TLBs are as small as they are because making them larger also means that they are slower, and at some point the L1 cache access speed is limited by TLB speed. For highly or fully associative structures (as is often the case for L1 TLBs), power consumption is also an issue.

Internal fragmentation

Posted Jul 18, 2026 21:30 UTC (Sat) by nyc (guest, #91222) [Link]

There are ways to adapt tail packing, incremental faulting, incremental zeroing, and more to address some of the major internal fragmentation issues. Plus converting things to slab allocations explicitly sized in other terms seamlessly avoids a lot of issues.

Are 1.7% really worth it?

Posted Jun 9, 2026 1:05 UTC (Tue) by nyc (guest, #91222) [Link]

There is a notion of TLB reach, which is basically how much memory is being mapped by the translations a TLB is caching. Where this is relevant is in TLBs whose entries' sizes may vary. Fill the TLB with base pages and it won't get far; the larger the page sizes used in the translations, the further the TLB can reach.

Some consumer-oriented vendors defeat the entire point of superpages by having fixed numbers of TLB entries for each translation size. Clearly such TLBs have the amount of memory they can map set in stone regardless of any kernel or userspace behaviour, apart from deliberately pessimally refusing to use some translation sizes.

So it's not just that TLBs are too small. They're also often not organised in ways that would allow kernels and userspace to make the most effective use of them.