LWN: Comments on "Suppressing SIGBUS signals"

Suppressing SIGBUS signals

ssmith32 — Wed, 07 Jul 2021 06:48:18 +0000

Yes, I imagine redundancy would be a better solution. Better to have the fly-by-wire fail over to a backup, then try to limp along in some weird state.

Of course, time and again, cost-cutting works against that, and you have cars that share data buses between control systems & the entertainment system :(

Suppressing SIGBUS signals

immibis — Mon, 05 Jul 2021 08:54:51 +0000

I'm sure there are many ways to bring things down if you can arbitrarily truncate files I'm using.

Suppressing SIGBUS signals

daniels — Fri, 02 Jul 2021 19:28:32 +0000

> Create and open a file on tmpfs, in directory not accessible to anyone else. mmap() it. Pass the descriptor over SCM_RIGHTS datagram. And unlink() the damn thing, to keep the clutter down.

That’s literally what we do, yeah.

Suppressing SIGBUS signals

ilammy — Thu, 01 Jul 2021 02:16:43 +0000

After doing some research, this idea is implemented in some systems as a MAP_COPY flag for mmap() [1]. That's basically the semantics of MAP_PRIVATE mapping, but with a “snapshot” guarantee that the data won't change, making your copy genuinely private, but avoiding the actual copy if the data does not change.

While thinking of how this could be implemented, I realized that it could be quite expensive, complicated, and full of “spooky action at a distance”. If some process grabs or released a MAP_COPY mapping of a file, then all existing processes must be made aware of it (e.g., by turning all mappings for everyone RO and catching page faults). Any change by the other process forces said process to expend some kernel time doing the copy of the page for the benefit of some other process, which is not particularly fair.

Turns out, adding MAP_COPY into Linux was discussed several times [2][3], but it's still considered a pretty stupid idea.

[1]: https://www.gnu.org/software/hurd/glibc/mmap.html
[2]: https://yarchive.net/comp/linux/map_copy.html
[3]: https://www.spinics.net/lists/linux-mm/msg119339.html

Suppressing SIGBUS signals

mathstuf — Wed, 30 Jun 2021 18:03:24 +0000

I've said this elsewhere, but "safe" is a lot like security: it depends on your environment (threat model in security). I don't care much about cosmic rays flipping bits (in the "I should consider this case" sense) and count that as outside of the safety model of my code. NASA can't work with such reckless abandon and have *far* higher concerns about such things. But languages don't save you there: redundancy does.

Aircraft have similarly high standards and I imagine the solution there is along the lines of rip out (or somehow poison) the `panic` function and let the linker tell you that you have a case that isn't *explicitly* handled.

Suppressing SIGBUS signals

Wol — Wed, 30 Jun 2021 17:59:35 +0000

> A process that panic()s or crashes on invalid memory accesses is inherently safe, isn't it?

And if you're in the aircraft where that software is controlling the plane's fly-by-wire system ... ?

Cheers,
Wol

Suppressing SIGBUS signals

miquels — Wed, 30 Jun 2021 15:52:59 +0000

A process that panic()s or crashes on invalid memory accesses is inherently safe, isn't it?

The problem with Rust and mmap is that it just doesn't work if you write from one thread or process and read from another - that's instant UB (Undefined Behaviour) and that is _really_ unsafe.

I think the only safe way to read from or write to mmap'ed memory in Rust is to use atomics - mmap it as AtomicU<whatever> and only read/write using the load/store operations on the atomic values.

Suppressing SIGBUS signals

hmh — Wed, 30 Jun 2021 11:57:37 +0000

That looks like it would have been a much more generally useful (and expensive, complicated, etc) feature to implement than suppressing sigbus or zero-extending...

Mmap snapshot (maybe with a read-only result if that would be much easier or cheaper to implement and still cover the use cases).

But until someone offers to do that work...

Suppressing SIGBUS signals

matthias — Tue, 29 Jun 2021 10:34:20 +0000

> You appear to be conflating two very different issues here. It's true the compositor has never really be able to tell if the client has been displaying garbage. However the source of SIGBUS is essentially a protocol error. The client has promised the compositor that there was buffer here, and instead there was a hand grande. SIGBUS always allowed the compositor to detect and handle such protocol violations.

The client has promised that there is a buffer that does not change while the compositor is using it. Any change would be a protocol error. Only in the very special case that the change is truncate, this protocol error could be detected by the compositor. With the proposed change of semantics, this special case just looks like other cases where the client changes the buffer at the wrong time.

> [Finding out which program displays garbage] is probably not impossible, but it surely wouldn't be easy.

For debugging purposes there should be a possibility to ask the compositor to which program some window belongs. After all, you also want to debug cases where a program just opens a window with garbage in it. Probably a much more common case than a program calling truncate on the buffer.

Suppressing SIGBUS signals

dullfire — Mon, 28 Jun 2021 18:54:55 +0000

I would not imagine "mmap" could ever be fully async-signal-safe (after all you are screwing with the page table). However for the every narrow case of replacing a no-longer valid mapping, it should be fine.

Seeing as mmap is typically a thin wrapper around a syscall (AFAIK there is no currently known way for this to not be a kernel task) most of this must naturally be done in kernel, which negates most of the issues.

However since you brought it up, I'm assuming you are implying being portable/standards conformant is important. So for that we would need a change of standards. But that's probably more of a 'paper' change (while it's possible some implementations that are posix.1 2008 compliant implement mmap(2) in a way that would be unsafe for this proposed usage, I kind of doubt it).

A linux-only change, that just papers over the issue seems like a poor solution.

Suppressing SIGBUS signals

NYKevin — Mon, 28 Jun 2021 17:29:41 +0000

> As I said in my original post: I'm pretty sure the compositors SIGBUS handler could mmap over the faulting region, and then set a flag that the given client is miss behaving.

signal-safety(7) does not list mmap as async-signal-safe, at least on my system. But that just means that POSIX doesn't require it to be safe. I suppose it's possible that Linux does implement an async-signal-safe mmap as an extension?

(Unfortunately, we can't just use the usual trick of "set a flag and return, then do the real work from the main event loop" because we need to fix the memory problem *before* we return from the SIGBUS handler, or else we'd need to pause the offending thread, call mmap from a different thread, etc.)

Suppressing SIGBUS signals

excors — Mon, 28 Jun 2021 14:53:35 +0000

> The compositor is just doing a memory lookup, at assembly level there is no such thing as a "failed read". The kernel can generate a signal but it can't fail the original instruction, the MOV has to return something.

I don't think that's really true. At the assembly level, the outcome of the MOV instruction is that it either loads a value into the register *or* triggers an exception. E.g. if you look in the ARMv8-A Architecture Reference Manual, it explicitly defines the LDR instruction in terms of the "AArch64.MemSingle" operation which can call the "AArch64.TakeException" operation, which sets up the exception state then calls "EndOfInstruction" to stop any further processing of the LDR instruction, so it won't write anything to the destination register.

Once the exception is triggered, the kernel is free to do whatever it wants - update the page tables then jump back to the MOV instruction to retry it, manually update the register state then jump back to the instruction after the MOV, call a signal handler, etc. Anyone writing user-space assembly code has to be aware of that, e.g. there are often ABI rules about stack pointers that are specifically there to allow the kernel to interrupt your thread at any point and run a signal handler on its current stack. So that's not something you can safely ignore when working at assembly level.

Suppressing SIGBUS signals

dullfire — Mon, 28 Jun 2021 14:52:16 +0000

> Isn't the real problem here the use of signal? The compositor is just doing a memory lookup, at assembly level there is no such thing as a "failed read".

As I said in my original post: I'm pretty sure the compositors SIGBUS handler could mmap over the faulting region, and then set a flag that the given client is miss behaving.

When the compositor returns from the signal handler, it would finish it's turn function/call stack (using the newly mmaped region... possibly zero filled, possibly full of kitten pictures.), eventually get high enough to notice that that connection had been marked as bad, then either discard the processing it had done, or perhaps show one frame with a garbage window.

That way you resume from the error AND get notification of the buggy/malicious client. And you can have meaningful error messages.

Suppressing SIGBUS signals

kleptog — Mon, 28 Jun 2021 13:18:08 +0000

Isn't the real problem here the use of signal? The compositor is just doing a memory lookup, at assembly level there is no such thing as a "failed read". The kernel can generate a signal but it can't fail the original instruction, the MOV has to return something. The compositor would have to do a siglongjmp() because otherwise it's going to generate a SIGBUS for every single instruction accessing the missing pages. And siglongjmp() is pretty tricky at the best of times.

I can understand that developers would prefer if the kernel would just return zeros and the compositor recheck the size of the file after the copy completes. It's just less moving parts that way.

Someone else here noted userfaultfd() is doing something similar, so maybe there's an answer there. The compositor thread would be blocked and the handler could remap the pages to avoid the SIGBUS retriggerring.

Suppressing SIGBUS signals

dullfire — Mon, 28 Jun 2021 08:35:41 +0000

> This was always the case. If the client changes the contents of the buffer to garbage, the compositor will not notice and the window will display garbage. Nothing new here. The change is that the client truncating the buffer now will look exactly the same as the client clearing the buffer at an inappropriate time.

You appear to be conflating two very different issues here. It's true the compositor has never really be able to tell if the client has been displaying garbage. However the source of SIGBUS is essentially a protocol error. The client has promised the compositor that there was buffer here, and instead there was a hand grande. SIGBUS always allowed the compositor to detect and handle such protocol violations. The proposed changes remove that ability (for the case of compositors that use it, so I guess it being "opt-in" is better than being "opt-out").

> Probably the program whose window displays garbage.

Except the compositor doesn't know there's even a problem, so it's can't say "hey it's connection XXX", or it registered with string "YYY". Not all programs have server side decoration (and IIRC the wayland standard is client side decoration). So if the user wasn't looking at that window, or doesn't remember which one it is (or never new in the case of dialogs, and a few other things), then it becomes difficult to figure out. If the window is on a "task list" like WM UI element, that might help, but not all windows are placed there.

It's probably not impossible, but it surely wouldn't be easy.

To sum it up (again): The approach here appears to be "make it work 'safely' by sweeping all the problems under the rug". I don't think that's a good long term solution.

Suppressing SIGBUS signals

ilammy — Mon, 28 Jun 2021 02:50:59 +0000

I wonder if some sort of copy-on-write private file mappings could help with that by avoiding copying the entire range (since mapped files tend to be huge). Like, you map a file a get a snapshot of its contents. Your process writing to that memory copies a page just for you and never syncs that page with the actual file. If any other process touches the file in any way via non-cow mapping or normal file ops, then the original data is copied and other process cow-mappings are dissociated from the file.

Suppressing SIGBUS signals

viro — Mon, 28 Jun 2021 01:25:02 +0000

Create and open a file on tmpfs, in directory not accessible to anyone else. mmap() it. Pass the descriptor over SCM_RIGHTS datagram. And unlink() the damn thing, to keep the clutter down. Or use O_TMPFILE, for that matter, since that's no more Linux-specific than new mmap flags would be.

Sure, somebody malicious and controlling a process with your credentials will be able to screw you over, but then they could attach gdb to your client and have their merry way with it.

Suppressing SIGBUS signals

izbyshev — Mon, 28 Jun 2021 00:07:45 +0000

Yes (the man page says that behavior of MAP_PRIVATE is unspecified if the underlying file changes, but this is what happens on Linux in practice). The ftruncate()-based scenario is only somewhat interesting because it might seem like it only changes the file metadata, but in fact the data in the truncated tail also changes (as observed by mmap() users).

To be clear, the same behavior within the last page of the file is already possible with current kernels: ftruncate() that doesn't remove the last page completely will look like filling its remainder with zeros (though it's also formally unspecified).

So, overall, I don't see how MAP_NOSIGBUS would help with unsafety of Rust's &[u8] referring to mmap()'ed range.

Suppressing SIGBUS signals

NYKevin — Sun, 27 Jun 2021 21:26:51 +0000

> Not every system is an overcommit system. Windows isn't. Linux with vm.overcommit_memory=2 isn't either.

Yes, that's why I included the footnote about the OOM killer being out of scope. It's not part of Rust in the first place.

Suppressing SIGBUS signals

dancol — Sun, 27 Jun 2021 20:11:37 +0000

It occurs to me that one way to address the original problem with SIGBUS is to wire up userfaultfd to the mapping somehow. Then the compositor could arrange for the zero substitution --- or any other policy --- on its own without hard-coding zero-fill policy into the kernel.

Suppressing SIGBUS signals

jayalane — Sun, 27 Jun 2021 19:35:19 +0000

You can’t change signal handling semantics much at all without breaking a ton of old stuff. Pretty sure Linus would not approve.

Suppressing SIGBUS signals

NYKevin — Sun, 27 Jun 2021 17:17:33 +0000

I don't think it is semantically possible to avoid making a copy. The length of the mmap is equal to the length on disk, and because Linux went with the no-mandatory-file-locking model, that length can change at any time, regardless of whether you're using Rust, C, or a magnetic needle and a steady hand. If you want to end up with a "safe" object, you need it to not get suddenly truncated/zero-filled while you're halfway through processing it.

Making a copy also resolves issues of the form "What if someone decides to scribble all over the file without changing its length, while I'm halfway through processing it?" More generally, this falls into the "validate, then process" model of doing things - you can't validate something if it can change out from under you!

Suppressing SIGBUS signals

matthias — Sun, 27 Jun 2021 14:46:46 +0000

> Further more it would actually only make the intended us case situation worse. Now clients can behave badly, and the compositor won't even be able to notice. No longs, no disconnected the miss behaving client, just users with "ugly windows",

This was always the case. If the client changes the contents of the buffer to garbage, the compositor will not notice and the window will display garbage. Nothing new here. The change is that the client truncating the buffer now will look exactly the same as the client clearing the buffer at an inappropriate time.

> and they may not even know what program it was. Sounds like a nightmare to debug.

Probably the program whose window displays garbage.

Suppressing SIGBUS signals

ocrete — Sun, 27 Jun 2021 14:24:25 +0000

Isn't the content of a memory range chaning silently always possible if another program is modifying the file?

Suppressing SIGBUS signals

anton — Sun, 27 Jun 2021 13:52:10 +0000

I would have thought about System V shared memory stuff (shmget and friends), and I was wondering about that myself when I read the article. Not that I have experience with it, but it seems that it was made for this. I now see that there is also POSIX shared memory (see man shm_overview), but it supports ftrucate(), so that probably does not help.

Suppressing SIGBUS signals

dullfire — Sun, 27 Jun 2021 13:49:56 +0000

Having now thought about it for a while, this seems like a pretty bad approach to me.

It seems to be targeted at exactly one use case (as others have mentioned).
Further more it would actually only make the intended us case situation worse. Now clients can behave badly, and the compositor won't even be able to notice. No longs, no disconnected the miss behaving client, just users with "ugly windows", and they may not even know what program it was. Sounds like a nightmare to debug.

Further more, for non RGB buffer layouts, I'm not even sure what that would end up looking like on the screen.

Again having thought about it for a while, it seems a sane thing to do would be: handle the SIGBUS by noting where(address) it's failing, and then mapping in a private page of all zero (or ones, or w/e) over the now invalid mapping. Then the compositor can log something like "client XXXX has given malformed buffer" or what ever. It can even take corrective action, like dropping the client connection (and thus implicitly destroying the bad display content), or maybe even popping up an error message for the user (depending on settings,use case ,etc).

I don't think papering over the problem (and then closing your eyes to it ever existing) is going to make better user experiences, or more stable software. However that seems to be the approach this patch series wants to take.

Suppressing SIGBUS signals

mpr22 — Sun, 27 Jun 2021 12:14:15 +0000

> Anyway, Rust wouldn't be having this problem at all if it had just adopted exceptions for error handling.

I dare say it would be having others, though.

Suppressing SIGBUS signals

eru — Sun, 27 Jun 2021 09:42:40 +0000

responded with an implementation of MAP_NOSIGBUS, which differs from __MAP_NOFAULT in a number of ways

I wonder why? Would it not be useful if *ix -style systems implemented similar extensions similarly? That would help porting and also they would probably eventually get standardized.

Suppressing SIGBUS signals

randomguy3 — Sun, 27 Jun 2021 08:13:23 +0000

I went and looked up the link:

The reason is that there will always exist clients which are either old (and predate file sealing) or refuse to use Linux-only APIs (they don't use memfd and file sealing, instead they use e.g. shm_open). Requiring sealed memfds in compositors would break these clients.
I don't believe the situation is about to change.
Rather than requiring changes in all compositors *and* clients, can we maybe only require changes in compositors? For instance, OpenBSD has a __MAP_NOFAULT flag. When passed to mmap, it means that out-of-bound accesses will read as zeroes instead of triggering SIGBUS. Such a flag would be very helpful to unblock the annoying SIGBUS situation.

Suppressing SIGBUS signals

randomguy3 — Sun, 27 Jun 2021 08:09:59 +0000

Are you referring to MAP_SHARED + MAP_ANONYMOUS memory? As I understand it, you can only share such memory with your child processes.

Or perhaps you mean using mem_fd? That possibility was mentioned in the article, which also notes "no compositor requires the use of sealed memfds because there are clients that are unwilling or unable to use them".

I am interested in the reasons behind this "unable or unwilling", though.

Suppressing SIGBUS signals

dancol — Sun, 27 Jun 2021 05:57:39 +0000

> why OOMing needs to crash, in a language with manual memory management. You could instead have fallible allocation and require the application to handle an OOM condition explicitly.

Not every system is an overcommit system. Windows isn't. Linux with vm.overcommit_memory=2 isn't either.

Anyway, Rust wouldn't be having this problem at all if it had just adopted exceptions for error handling.

Suppressing SIGBUS signals

dancol — Sun, 27 Jun 2021 05:51:16 +0000

Or the language could just do the right thing and install a signal handler and report the access failure as a language-specific exception or panic --- but the entire Linux ecosystem has been inexplicably reluctant to change or improve anything whatsoever relating to signals, so it's very hard for a general-purpose library to use a signal handler in a way that doesn't stomp over someone else's desired use.

Suppressing SIGBUS signals

roc — Sun, 27 Jun 2021 04:14:28 +0000

That's a good point.

Suppressing SIGBUS signals

roc — Sun, 27 Jun 2021 04:12:43 +0000

> `&[u8]` might be fine, but if I mmap'd something like `&[RawSpriteData]`, I think I'd *prefer* to crash because all-zeros isn't very useful to me.

You can have a safe mmap function that returns &[u8]. Then you could use something like https://docs.rs/safe-transmute/0.11.2/safe_transmute to transmute to a different kind of reference if that's safe.

You're right that in extremes, safety gets a bit fuzzy. It's nice to be able to push the boundaries pretty far out though.

Suppressing SIGBUS signals

ju3Ceemi — Sat, 26 Jun 2021 23:10:51 +0000

Why are they using a memory-baked file as shared memory ?
Why not use shared memory as shared memory ?

Suppressing SIGBUS signals

mathstuf — Sat, 26 Jun 2021 19:21:39 +0000

Agreed. It's the same reason commit messages are so important for diffs: it helps to introduce others to what the change does as a whole, how it is used, and to (ideally) say why it is useful. Documentation is even more important because it should be what people look for first when trying to figure out how to do something. Commit messages are just harder to dig up via manpages…

Suppressing SIGBUS signals

izbyshev — Sat, 26 Jun 2021 18:02:39 +0000

> Rust safety only requires that &[u8] reference a range of memory that doesn't change and doesn't cause a crash if you access it.

Consider the following sequence of events:
1. Program A maps the file with MAP_PRIVATE | MAP_NOSIGBUS.
2. Program A accesses page P from that file.
3. Program B truncates the file to zero.
4. Page P is evicted from RAM due to memory pressure.
5. Program A accesses page P again.

How is the resulting page fault handled? If a zero-filled page is mapped, then the contents of the range of memory does change silently.

Suppressing SIGBUS signals

mathstuf — Sat, 26 Jun 2021 17:16:58 +0000

I don't know. `&[u8]` might be fine, but if I mmap'd something like `&[RawSpriteData]`, I think I'd *prefer* to crash because all-zeros isn't very useful to me.

> I know, at compile time, that dereferencing a given pointer will never cause the program to crash, corrupt the heap, or otherwise misbehave.

Rust (or any language for that matter) can only do so much. If, say, the hypervisor ends up fiddling with the VM's page tables, programs can certainly crash no matter how "safe" the language or runtime is. If the RAM gets a bit twiddled by a cosmic ray, all bets are similarly off in most cases. These kinds of things are outside of a language's control and are, IMO, purely in the realm of "just how paranoid do you need to be?" when coding any project in any language. JoeSchmo webapp? Users are probably trained to refresh on weird errors these days and such things are fine with systemd's restart logic. Sending a rover to Mars? Random RAM flips are very relevant, better have redundant hardware.

> I know, at compile time, that my program will handle all error cases within a given subroutine. The subroutine is "safe."

Not sure how you plan to handle the "power lost" error condition, but I'd be interested :) .

Suppressing SIGBUS signals

Bigos — Sat, 26 Jun 2021 14:45:11 +0000

What I meant is to fix all legitimate applications (doing a change in mesa might solve the majority of it) and then eventually require sealed memfds instead of supporting the mechanism that is, as you mentioned, prone to abuse.

However, it seems the ship has already sailed and no one wants to change the protocol to accommodate this safer way of passing buffers around. Which is sad.

Suppressing SIGBUS signals

re:fi.64 — Sat, 26 Jun 2021 14:08:44 +0000

Afaik this isn't really a solvable issue for any protocol without just not supporting sending these at all, which would be a performance hit.