ARC, anyone? [LWN.net]

ARC, anyone?

Posted Mar 5, 2026 16:06 UTC (Thu) by intelfx (subscriber, #130118)
Parent article: Reconsidering the multi-generational LRU

So, it would appear we still do not have anything remotely competitive with ZFS' ARC. Sigh.

to post comments

ARC, anyone?

Posted Mar 6, 2026 13:42 UTC (Fri) by PeeWee (subscriber, #175777) [Link] (2 responses)

IIUC, we sort of already do have, or had(?), an approximation of that with "classic" LRU. The active/inactive lists track frequency/recency respectively, kind of. And there is also refault distance tracking, though it seems to be limited to file pages, which should make no difference for a comparison to ARC.

Thus Linux already has ARC-but-not-ARC. File pages start life on the inactive list, after being initially faulted in. That's almost identical to what happens in ZFS ARC [PDF] (p. 13f); total misses - the page wasn't even on one of the ghost lists - are put at the head of the LRU list, Linux's inactive list being the equivalent. Only after being accessed again will they be promoted to the active list, hence they get promoted due to frequent access, otherwise they'd already be evicted again, possible still in some shadow entry. But a promotion to the active list implies a minor/soft page fault - inactive pages get unmapped but not evicted -, so it's not exactly the same as in ARC, where pages are simply promoted to the LFU list, no fault having happened, so slight advantage for ARC, maybe, depending on the LFU list overhead, see below. Arguably, that makes Linux's active list an LFU list in disguise, sans the access counter. The only way to eviction is by being less frequently accessed than other active pages; and to get on the active list, file pages need to be accessed a second time. And that's where the equivalency breaks, because in ARC it would be evicted but placed at the head of the ghost LFU list, whereas in Linux it gets unmapped and placed at the head of the inactive list for being the most recent visitor in purgatory.

Reiterating, in Linux mm terms, file pages age out of the active list because they are not accessed frequently (enough) onto the inactive list and then onto the shadow "list" (entries in the radix tree, according to above article), after being evicted for not being accessed recently. If they get refauted in and there is still a shadow entry, one page gets demoted from the active list right next to the refaulted one, thus shortening the active list and lengthening the inactive list by one, respectively, if the refault distance (how many page faults were handled in its absence) is less than the length of the active list. There is no mention of what happens when refault distance is equal or greater than the length of the active list, so I am assuming no additional action takes place in that case. Also, since all of the above is in context of file-backed pages, I just want to add that anon pages only differ in that they are immediately placed on the active list after faulting them in, which might have implications regarding their exact refault behavior and which list to adjust, but I don't even know if it even applies there; couldn't find anything on anon refault distance tracking.

There may also be other reasons for not having an ARC clone in Linux. As that presentation above states, LFU lists are of O(log(n)) complexity. Maybe that implies, it only makes sense for file pages, the overhead being small enough compared to the cost of a major fault, hitting spinning rust I/O. But it might be too much for tracking in-memory pages for frequency, if all that buys us is avoiding a minor fault - still considering the active list as LFU. Just assuming that active pages are frequently accessed might be the cheaper option, since aging out does not mean eviction, only demotion to a lower "tier", with tolerable (soft) refault cost.

It's also hard to tell if it is ("remotely") competitive with ARC, at least for me. ZFS (ARC) has great tools and stats, but I wouldn't know where to find their equivalents in Linux, e.g. hit and miss rates on said active/inactive/shadow lists. Plus, ZFS ARC can optionally store compressed blocks, which need to be decompressed on access, just to name one more variable; I am almost sure that the compressed version stays in ARC and the decompressed one lives in the page cache, at least in ZFS on Linux (ZoL). That, if true, may have interesting effects, because really hot read-only pages, i.e. always clean (not dirtied, which would invalidate the ARC cache entry), never to be evicted from the page cache, their compressed siblings may get evicted from ARC, because, for all it knows, they are cold; I'd call that a temperature paradox, for lack of a better term, if there isn't already a name for it. But maybe I am totally misunderstanding how ZoL integrates with vanilla Linux.

One more question just popped into my head; if ARC is so superior, then why was it only implemented for ZFS and not also in the swap path of Solaris? Did Sun Microsystems simply not think of that, or the Illumos developers, for that matter? Maybe that does speak to the relative cost compared to simpler solutions.

ARC, anyone?

Posted Mar 6, 2026 17:39 UTC (Fri) by hnaz (subscriber, #67104) [Link] (1 responses)

> And there is also refault distance tracking, though it seems to be limited to file pages

There actually is shadow tracking for swap-backed pages. On classic LRU, swapbacked memory starts out on inactive list, just like file. It provides the same workingset protection against bulk accesses with low locality (e.g. a scan through a large, cold, mostly swapped out anon segment).

You can observe the shadow list hits through /proc/vmstat::workingset_*.

ARC, anyone?

Posted Mar 6, 2026 20:29 UTC (Fri) by PeeWee (subscriber, #175777) [Link]

> And there is also refault distance tracking, though it seems to be limited to file pages There actually is shadow tracking for swap-backed pages. On classic LRU, swapbacked memory starts out on inactive list, just like file. It provides the same workingset protection against bulk accesses with low locality (e.g. a scan through a large, cold, mostly swapped out anon segment).

Thanks for pointing that out. Thinking about it some more, given this info, it makes perfect sense, because they are just file-backed pages as well, right? I've also had a glance at workset.c, which is exemplary well documented by comments, in the meantime, and it does contain MGLRU code, as well. Not that I read the actual code, or would be able to understand it, but I think it's just the same as in the classic active/inactive lists, the only difference being that there is no "inactive" list, only gen 0, the oldest generation LRU list, on which file-backed pages are placed.

You can observe the shadow list hits through /proc/vmstat::workingset_*.

TIL where to find the stats. Thanks! And it looks like it's just the same with MGLRU:

workingset_nodes 4530
workingset_refault_anon 4
workingset_refault_file 13627
workingset_activate_anon 4
workingset_activate_file 9981
workingset_restore_anon 4
workingset_restore_file 4655
workingset_nodereclaim 5396

There hasn't been much memory churn yet since last boot. But I do have two 16K mTHPs, as it seems (pool_total_size=32768), already in zswap, while still at 8.8 GiB committed, with ~7 GiB page cache. That's with vm.swappiness=180 and 14.5 GiB RAM total.