How much would it cost for every NVMe drive to have a non-volatile cache? [LWN.net]

How much would it cost for every NVMe drive to have a non-volatile cache?

Posted Mar 9, 2026 14:55 UTC (Mon) by farnz (subscriber, #17727)
In reply to: How much would it cost for every NVMe drive to have a non-volatile cache? by DemiMarie
Parent article: The ongoing quest for atomic buffered writes

What difference does it make? The notifications (in kernel caches, sent to device but not yet guaranteed on non-volatile storage, guaranteed as safe as the device can make it) are the same whether the cache is volatile, or non-volatile.

And the BOM cost for being able to save the volatile cache to non-volatile store is not small - maybe $10 for the capacitance to power the DRAM, flash and MCU until the cache is saved, plus however much extra flash you need so that you have a "safe space" to write the volatile cache to. You can't just make the device only have non-volatile cache, because the performance characteristics of non-volatile memory aren't what you need when you're doing things like "read 1 MiB from the main flash, replace 4 KiB with this new change, write 1 MiB to a new location, mark the original 1 MiB as safe to erase".

Note, too, that depending on the NVMe design, a write in non-volatile cache may not be considered "safe", because the chances of the non-volatile cache being corrupted are too high. At least one device I've looked at with non-volatile cache has a clear statement that the non-volatile cache is only safe during a commanded shutdown of the device, and not a power loss event, because the microcontroller may scribble over non-volatile cache during power loss.

to post comments

How much would it cost for every NVMe drive to have a non-volatile cache?

Posted Mar 9, 2026 16:20 UTC (Mon) by Wol (subscriber, #4433) [Link] (1 responses)

> You can't just make the device only have non-volatile cache, because the performance characteristics of non-volatile memory aren't what you need when you're doing things like "read 1 MiB from the main flash, replace 4 KiB with this new change, write 1 MiB to a new location, mark the original 1 MiB as safe to erase".

Did NVRAM use to be 4KiB blocks? Can you still get it?

Okay, I don't know the cost implications, but couldn't you write your 4K updates to a 4K NVRAM cache, and then use normal DRAM as your cache for actually doing your "read 1MiB in, update 4K, write 1 MiB out"? And then you just need to be able to flush your incoming data to the 4K NVRAM on a power fail, which is an easier problem than flushing everything to the permanent backing store?

(And have a "here is a 1MiB write" command, so if the OS is streaming data to disk it can still send data in chunks that match the NVMe block size for efficiency).

Cheers,
Wol

How much would it cost for every NVMe drive to have a non-volatile cache?

Posted Mar 9, 2026 16:32 UTC (Mon) by farnz (subscriber, #17727) [Link]

You can get various forms of flash with different block sizes; the smaller the blocks, the more expensive it becomes per byte, and the faster you want it, the more expensive it gets.

But the problem is that you don't have just 4K of updates at a time to handle - a high performance NVMe drive is handling gigabytes of unwritten data in its volatile cache, and having gigabytes of very fast flash is expensive, as compared to gigabytes of RAM, gigabytes of reasonably fast flash, and a supercapacitor to let you flush the RAM to the fast flash before you lose power completely. However, even that's going to be significant - doing it with pSLC flash will add maybe $30 to your $100/TiB NVMe SSD, more if you want a bigger cache (useful for performance).