Non-coherent shared memory?

jagalactic — Tue, 13 May 2025 14:00:42 +0000

Greg is correct that write order is usually not guaranteed by hardware currently. But when we think about cache coherency it's tempting to always think about the fine-grained multi-writer cases, which are hard - and already have a high performance cost on existing systems. So without sharing disaggregated memory, we already try to avoid these use cases due to the performance cost.

But a lot of work flows do what you might call "publish and share", where the publisher needs to flush the processor cache as a last step of publishing, and then there are no hard cache problems because data is not mutated in place while it's shared. This is compatible with a lot of data analytics flows as well as sharing in-memory databases read-only. In fact, it's consistent with how an important subset of databases - the log-structured merge tree (LSM) variants - work, even in writable mode.

LSMs write out searchable indexes (often called SST files) which - after the initial write - are read-only until they eventually get merged into new SSTs and get unlinked. Putting SSTs in shared memory (e.g. via famfs) is a pretty nice use case.

Mind you, today there are not yet any databases that can directly create files in famfs, due to its pre-allocation requirement - but 1) files can be copied into famfs for sharing, and 2) there are multiple ways famfs and/or apps could adapt to relax famfs' create procedure in future work...

Non-coherent shared memory?

gmprice — Fri, 09 May 2025 00:15:57 +0000

It's hard because write order isn't guaranteed and explicit flushes that go all the way to the backing media (not just buffers attached to one head) presently don't exist.

Non-coherent shared memory?

DemiMarie — Thu, 08 May 2025 23:36:00 +0000

Would it make sense to use shared memory in a non-cache-coherent mode with explicit flushes?

LWN: Comments on "A FUSE implementation for famfs"

Non-coherent shared memory?

Non-coherent shared memory?

Non-coherent shared memory?