Rust and UB

> Within a single process it is UB on the read side if the other side does a non-atomic write,

Yes, I am well aware. I was talking about the multi-process situation.

I don't think we have a formal (in the mathematical sense) model that accurately captures the multi-process situation. For the specific case of read-write races, we do have memory models that make read-write races return "poison"/"undef" on the reader side, without causing full UB (https://dl.acm.org/doi/10.1145/3591297), so if the kernel side uses atomics and also "freezes" all its reads (as in https://github.com/rust-lang/rfcs/pull/3605 -- so unfortunately not yet available in Rust), it would be protected against UB even in a theoretical sense. In practice, even without "freeze" it seems hard to imagine how an atomic read racing with a non-atomic write returns anything worse than a strange bit pattern. Obviously if the relevant data is a pointer you blindly trust, things go wrong, but in the situation we were talking about where there's privilege separation between the two parties, there cannot be any pointers that the privileged side blindly trusts, so I was implicitly excluding that case.

If this is a write-write race, then we don't have any theoretical model I am aware of that would help us. But in practice, if you are the side doing the atomic write, I don't see how you could possibly suffer from UB if there is a racing non-atomic write, provided that you never trust the data you read from this memory (which indeed the kernel cannot trust anyway).

> In QEMU (written in C though Rust is coming), we use Linux-like wrappers for the C memory model. It turns out to be usable and just as optimized as the Linux primitives.

Oh, interesting. I was not aware the C memory model is already used in the Linux kernel. So to make sure we are talking about the same thing, you are using the operations from the C "atomics" library (https://en.cppreference.com/w/c/atomic), and no volatile accesses or inline assembly?

Are you following some sort of discipline where every location in memory can be used either with the LKMM or the C model, or are you mixing both of them on the same location? Mixing both on the same location seems extremely risky to me. Even if memory is "partitioned" into regions governed by the C memory model and regions governed by the LKMM, there are some very non-trivial interactions going on -- the happens-before relationships of the two models must be sufficiently compatible with each other. Theoretically speaking, this is a really hard problem. In practice, it probably works because there are not so many different lowering schemes one can use for the basic atomic operations.

> However, we do have a couple tricks under our sleeve, where we "know" that some reorderings cannot happen at the processor level, and therefore use compiler fences instead of thread fences. For example instead, of the ill-fated Consume load, we use a Relaxed load followed by an Acquire compiler barrier. And before/after any RMW atomic, on x86 we use SeqCst compiler fences instead of a thread fence, not unlike smp_mb__before_atomic() in Linux. Those probably would *also* be technically UB...

Yeah okay that's not the C memory model. ;) Even if the processor can't do such reorderings, the compiler can. So I see no way to ever justify using compiler fences instead of thread fences in a principled way. I would expect that on strongly ordered hardware like x86, most thread fences anyway compile to NOPs?