Allocator optimizations for transparent huge pages

Posted May 27, 2024 10:04 UTC (Mon) by david.hildenbrand (subscriber, #108299)
In reply to: Allocator optimizations for transparent huge pages by DemiMarie
Parent article: Allocator optimizations for transparent huge pages

Especially page tables (and many more things) are unmovable and 4k. Instead of refusing to allocate them, we should do a better job at grouping them to restrict fragmentation locally (what pageblocks already do, and there is work in progress to improve the situation).

Allocator optimizations for transparent huge pages

Posted May 27, 2024 16:18 UTC (Mon) by DemiMarie (subscriber, #164188) [Link] (3 responses)

Could page tables be made movable?

Allocator optimizations for transparent huge pages

Posted May 27, 2024 19:32 UTC (Mon) by david.hildenbrand (subscriber, #108299) [Link] (2 responses)

I looked at this once (migration of leaf page tables only) and it turned into complexity. It can be done I think, given sufficient brain power and time.

But it‘s only the tip of the iceberg I’m afraid. Many/most allocations in the kernel are unmovable, only selected (pagecache, anonymous memory, zsmallloc) are movable. We see a lot more unmovable allocations lately, the trend is going into the wrong direction: secretmem and guest_memfd don‘t support page migration at all.

Allocator optimizations for transparent huge pages

Posted May 29, 2024 14:17 UTC (Wed) by page_walker (subscriber, #99801) [Link] (1 responses)

We can try to vmalloc everything in kernel and not use direct mapping at all. That means a kernel data access can trigger a page fault, when the page is under migration. That could cause recursive page faults, if kernel is handling a user page fault or a kernel page fault.

Allocator optimizations for transparent huge pages

Posted May 30, 2024 1:51 UTC (Thu) by willy (subscriber, #9762) [Link]

We don't support migrating the memory which underlies a vmalloc allocation, for exactly this reason.

Allocator optimizations for transparent huge pages

Posted May 28, 2024 7:31 UTC (Tue) by taladar (subscriber, #68407) [Link] (1 responses)

Assuming no memory hot swapping, aren't page tables basically a fixed size for the entire lifetime of a system (as in between two reboots)? Wouldn't they be ideally allocated as huge pages themselves?

Allocator optimizations for transparent huge pages

Posted May 28, 2024 11:10 UTC (Tue) by david.hildenbrand (subscriber, #108299) [Link]

Page tables used for user space process are very dynamic and are the problematic bit. Page tables used for the direct map are (besides memory hot(un)plug) static and we try to make use of huge mappings if possible.