The multi-generational LRU

Posted Apr 3, 2021 11:27 UTC (Sat) by linusw (subscriber, #40300)
Parent article: The multi-generational LRU

This is exactly the kind of content I want to see in LWN. Excellent article.

My first thought is that as engineers we are good at looking at one thing in isolation and optimize it, as the economists say "ceteris paribus" (all other things being equal).

What I'm asking myself is how other parts of the memory hierarchy affect and play with the LRU mechanism.

Especially the block layer. How does the previous code and this code perform depending on whether we use none, deadline, BFQ or kyber as block scheduler?

The multi-generational LRU

Posted Apr 3, 2021 11:39 UTC (Sat) by Sesse (subscriber, #53779) [Link] (1 responses)

I would assume it doesn't matter, since they work on entirely different timescales. I/O schedulers work on times from microseconds up to milliseconds (or perhaps seconds on rotating rust), evicting pages works on times of seconds to hours. So what one does is unlikely to impact the other one much.

The multi-generational LRU

Posted Apr 3, 2021 11:59 UTC (Sat) by matthias (subscriber, #94967) [Link]

In the case of memory pressure, the time scales might be much closer to each other. It is entirely possible, that a page is evicted almost immediately after usage. But I would think that the nature of the storage device would play a much bigger role than the I/O scheduler. But this is just a guess.

At the end these are all heuristics and it is impossible to test all possible combinations. Optimizing one system on its own is hard enough. The more systems you bring in to that equation, the more combinations you will get. And the combinatoric explosion will prohibit to test all possible combinations. This is why it is important that such patches are tested on real life workloads and real life configurations. This way, at least the most important combinations of configuration of subsystems are covered.