LWN: Comments on "Top-tier memory management"

Top-tier memory management

krakensden — Fri, 04 Jun 2021 04:53:22 +0000

I'm pretty excited about that patch set, the abstract makes it sound like a huge win.

Top-tier memory management

Paf — Sun, 30 May 2021 14:54:41 +0000

I’m not sure of the precise cost of the mechanisms being proposed here, but the page cache LRU is kind of expensive (at least to me as an FS developer optimizing throughput). I’m not sure that kind of management is appropriate for all of memory. (But I might be off in terms of scale.)

Top-tier memory management

flussence — Sun, 30 May 2021 12:29:52 +0000

It's *possible*, but at present it also sounds like an incredibly niche feature for a few extra MHz that'd get shot down by manufacturers just telling users to not use mismatched RAM sticks.

This idea isn't too far from how old CPUs had independent functional units for x87/SSE/3DNow. I've heard urban legends about clever asm programmers wringing a few percent more speed out of those, but it wasn't a party trick worth spending silicon on in the long run.

Top-tier memory management

willy — Sun, 30 May 2021 10:30:06 +0000

While that sounds appealing, you're really just giving up a lot of bandwidth. If you have N sticks of RAM, you want to let the CPU put cache line L on stick L % N.

What you're proposing would put pages A-B on stick 0, B-C on stick 1, etc. It's like choosing concatenation instead of RAID 0.

Top-tier memory management

Cyberax — Sun, 30 May 2021 08:47:09 +0000

That's actually not at all a problem. CPUs talk with DRAM through a memory controller, and it can already change the DRAM frequency.

Top-tier memory management

flussence — Sun, 30 May 2021 08:06:27 +0000

That sounds like a nice thing to have, but it seems more likely we'll have practical nuclear fusion before mainboards that support independent memory channel clocks.

Top-tier memory management

pabs — Sun, 30 May 2021 07:11:25 +0000

I wonder if this sort of thing could allow driving memory sticks of different speeds at different rates and preferring the faster ones.

Swap

MattBBaker — Fri, 28 May 2021 22:31:49 +0000

The system takes care of that feature too. It's fair to say that if a page was being evicted from DRAM, then on a system with just DRAM it would have been swapped instead. So why not assume that everything in slow space is going to be swapped too. Especially if the slow bank is an HBM space then you can migrate DRAM pages into the HBM banks and once you get HBM_LINE_SIZE pages then page them all out in bulk.

Top-tier memory management

droundy — Fri, 28 May 2021 19:06:39 +0000

I wonder why something closer to the LRU used for the page cache wouldn't be used for determining which tier a page belongs in? It kind of seems like the same problem.

Swap

faramir — Fri, 28 May 2021 18:41:23 +0000

While virtual memory is transparent to the software which is using it, it is not transparent to the system. It requires the system to find some real memory that can be (re)used. This can sometimes be a problem. With NUMA or persistent memory, that isn't a problem. Memory access may still be slower, but at least it will still work without any immediate work by the system.

Swap

corbet — Fri, 28 May 2021 17:51:21 +0000

In a sense what you're asking for already exists, it's called "virtual memory". In that sense it is directly addressible and the kernel will automatically move data in and out.

Swap

calumapplepie — Fri, 28 May 2021 17:35:11 +0000

How hard would it be to rework swap into just another tier, from the perspective of memory load distribution?

Obviously, since swap can't be mapped for direct access by processes, it'd never be quite like the others. But duplicating the code for "move between fast and slow storage" seems like a worse call. Obviously swap needs a fair amount of unique logic: but so would many other different tiers. Further, generalizing a single system to allow for arbitrary tiers that can be as different as DRAM and spinning rust swapfiles would mean that the next weird memory system that is dreamed up by the hardware designers can be added without much effort.

Swap

kay — Fri, 28 May 2021 16:35:48 +0000

tecnically yes, its no direct access memory

I remember computers using magnetic tape as memory ;-), so you can see it as veeery slow to access memory ;-)

Swap

corbet — Fri, 28 May 2021 16:28:21 +0000

Swap is not directly addressible, though, so it doesn't really qualify as a "tier" in the sense being discussed here.

Top-tier memory management

kay — Fri, 28 May 2021 16:26:02 +0000

Indeed and to extend your remark: this sounds very similar to the LRU page reclaim idea described in https://lwn.net/Articles/856931/

Top-tier memory management

NHO — Fri, 28 May 2021 16:13:47 +0000

What is swap but a very slow memory tier?