Suppressing SIGBUS signals

Posted Jun 26, 2021 0:57 UTC (Sat) by NYKevin (subscriber, #129325)
In reply to: Suppressing SIGBUS signals by zlynx
Parent article: Suppressing SIGBUS signals

The word "safe" has two different meanings in this context:

1. I know, at compile time, that dereferencing a given pointer will never cause the program to crash, corrupt the heap, or otherwise misbehave. The pointer is "safe."
2. I know, at compile time, that my program will handle all error cases within a given subroutine. The subroutine is "safe."

The problem is that mmap basically can't support (2) if the map is backed by a real file, unless you want to catch and handle SIGBUS. But nobody *wants* to handle SIGBUS, especially in a "let's try to recover and keep executing" way (as opposed to a "let's print a cryptic error message and call abort" way). (1), on the other hand, is solved quite effectively with this patch, and anyone who really does want (2) can use something other than mmap to read the file (which is no worse than the status quo).

Suppressing SIGBUS signals

Posted Jun 26, 2021 1:21 UTC (Sat) by sbaugh (guest, #103291) [Link] (4 responses)

I think Rust considers crashing the program to be "safe" (certainly I do); that's what it does on OOM, after all.

So I think this is misguided: I think SIGBUS signals on bad memory accesses are already safe, in Rust terms.

Suppressing SIGBUS signals

Posted Jun 26, 2021 2:04 UTC (Sat) by NYKevin (subscriber, #129325) [Link] (3 responses)

Meh, that's a matter of perspective. Crashing is only the default behavior, and they are working to make it easier to avoid crashing on OOM. There is no technical reason* why OOMing needs to crash, in a language with manual memory management. You could instead have fallible allocation and require the application to handle an OOM condition explicitly.

* You cannot control what the OOM killer does (although the sysadmin can, to some extent). But the OOM killer could even kill a completely unrelated process, and is not part of Rust itself, so let's ignore it. Besides, since Rust is a systems language, it can be used to write kernelspace code that is exempt from the OOM killer altogether.

Suppressing SIGBUS signals

Posted Jun 27, 2021 5:57 UTC (Sun) by dancol (guest, #142293) [Link] (2 responses)

> why OOMing needs to crash, in a language with manual memory management. You could instead have fallible allocation and require the application to handle an OOM condition explicitly.

Not every system is an overcommit system. Windows isn't. Linux with vm.overcommit_memory=2 isn't either.

Anyway, Rust wouldn't be having this problem at all if it had just adopted exceptions for error handling.

Suppressing SIGBUS signals

Posted Jun 27, 2021 12:14 UTC (Sun) by mpr22 (subscriber, #60784) [Link]

> Anyway, Rust wouldn't be having this problem at all if it had just adopted exceptions for error handling.

I dare say it would be having others, though.

Suppressing SIGBUS signals

Posted Jun 27, 2021 21:26 UTC (Sun) by NYKevin (subscriber, #129325) [Link]

> Not every system is an overcommit system. Windows isn't. Linux with vm.overcommit_memory=2 isn't either.

Yes, that's why I included the footnote about the OOM killer being out of scope. It's not part of Rust in the first place.

Suppressing SIGBUS signals

Posted Jun 26, 2021 17:16 UTC (Sat) by mathstuf (subscriber, #69389) [Link] (5 responses)

I don't know. `&[u8]` might be fine, but if I mmap'd something like `&[RawSpriteData]`, I think I'd *prefer* to crash because all-zeros isn't very useful to me.

> I know, at compile time, that dereferencing a given pointer will never cause the program to crash, corrupt the heap, or otherwise misbehave.

Rust (or any language for that matter) can only do so much. If, say, the hypervisor ends up fiddling with the VM's page tables, programs can certainly crash no matter how "safe" the language or runtime is. If the RAM gets a bit twiddled by a cosmic ray, all bets are similarly off in most cases. These kinds of things are outside of a language's control and are, IMO, purely in the realm of "just how paranoid do you need to be?" when coding any project in any language. JoeSchmo webapp? Users are probably trained to refresh on weird errors these days and such things are fine with systemd's restart logic. Sending a rover to Mars? Random RAM flips are very relevant, better have redundant hardware.

> I know, at compile time, that my program will handle all error cases within a given subroutine. The subroutine is "safe."

Not sure how you plan to handle the "power lost" error condition, but I'd be interested :) .

Suppressing SIGBUS signals

Posted Jun 27, 2021 4:12 UTC (Sun) by roc (subscriber, #30627) [Link] (4 responses)

> `&[u8]` might be fine, but if I mmap'd something like `&[RawSpriteData]`, I think I'd *prefer* to crash because all-zeros isn't very useful to me.

You can have a safe mmap function that returns &[u8]. Then you could use something like https://docs.rs/safe-transmute/0.11.2/safe_transmute to transmute to a different kind of reference if that's safe.

You're right that in extremes, safety gets a bit fuzzy. It's nice to be able to push the boundaries pretty far out though.

Suppressing SIGBUS signals

Posted Jun 27, 2021 17:17 UTC (Sun) by NYKevin (subscriber, #129325) [Link] (3 responses)

I don't think it is semantically possible to avoid making a copy. The length of the mmap is equal to the length on disk, and because Linux went with the no-mandatory-file-locking model, that length can change at any time, regardless of whether you're using Rust, C, or a magnetic needle and a steady hand. If you want to end up with a "safe" object, you need it to not get suddenly truncated/zero-filled while you're halfway through processing it.

Making a copy also resolves issues of the form "What if someone decides to scribble all over the file without changing its length, while I'm halfway through processing it?" More generally, this falls into the "validate, then process" model of doing things - you can't validate something if it can change out from under you!

Suppressing SIGBUS signals

Posted Jun 28, 2021 2:50 UTC (Mon) by ilammy (subscriber, #145312) [Link] (2 responses)

I wonder if some sort of copy-on-write private file mappings could help with that by avoiding copying the entire range (since mapped files tend to be huge). Like, you map a file a get a snapshot of its contents. Your process writing to that memory copies a page just for you and never syncs that page with the actual file. If any other process touches the file in any way via non-cow mapping or normal file ops, then the original data is copied and other process cow-mappings are dissociated from the file.

Suppressing SIGBUS signals

Posted Jun 30, 2021 11:57 UTC (Wed) by hmh (subscriber, #3838) [Link] (1 responses)

That looks like it would have been a much more generally useful (and expensive, complicated, etc) feature to implement than suppressing sigbus or zero-extending...

Mmap snapshot (maybe with a read-only result if that would be much easier or cheaper to implement and still cover the use cases).

But until someone offers to do that work...

Suppressing SIGBUS signals

Posted Jul 1, 2021 2:16 UTC (Thu) by ilammy (subscriber, #145312) [Link]

After doing some research, this idea is implemented in some systems as a MAP_COPY flag for mmap() [1]. That's basically the semantics of MAP_PRIVATE mapping, but with a “snapshot” guarantee that the data won't change, making your copy genuinely private, but avoiding the actual copy if the data does not change.

While thinking of how this could be implemented, I realized that it could be quite expensive, complicated, and full of “spooky action at a distance”. If some process grabs or released a MAP_COPY mapping of a file, then all existing processes must be made aware of it (e.g., by turning all mappings for everyone RO and catching page faults). Any change by the other process forces said process to expend some kernel time doing the copy of the page for the benefit of some other process, which is not particularly fair.

Turns out, adding MAP_COPY into Linux was discussed several times [2][3], but it's still considered a pretty stupid idea.

[1]: https://www.gnu.org/software/hurd/glibc/mmap.html
[2]: https://yarchive.net/comp/linux/map_copy.html
[3]: https://www.spinics.net/lists/linux-mm/msg119339.html