Beware fragmentation

Posted Sep 12, 2022 8:25 UTC (Mon) by WolfWings (subscriber, #56790)
In reply to: Beware fragmentation by jreiser
Parent article: The transparent huge page shrinker

Sorry, minor oversight on my part, since you do need to fault the RAM in before it exists. The program was literally typed from memory as an example, sorry if I missed an include. :P

#ifdef MADV_HUGEPAGE
    if ( x != NULL ) {
        madvise( x, size, MADV_HUGEPAGE )
        for ( int i = 0; i < size; i += alignment ) {
            ((char *)x)[i] = 0;
        }
    }
#endif

If you fault the pages in before the madvise then you have to wait for the THP scanner to circle back around and find things, but if you madvise before you fault the pages in then each fault is generated as a THP directly so you also need a lot less page faults to actually commit all the RAM you allocated, since Linux is (too) aggressive about overcommit until pages are actually accessed.