Beingessner: Rust's Unsafe Pointer Types Need An Overhaul

Interesting, thanks for the link, I wasn't aware that Ada did that. It would be interesting to compare evolutions of all such languages and the classes of bugs or the complexities in developing certain classes of programs.

>> But if we could say "this never aliases anything" some constructs could be more easily optimized.
Which is why (safe) Rust gets to go very fast. But attempting to retro-fit this to a language like C is impractical.

To be fair, C does have the restrict keyword. But that's more or less the opposite of register or Rust's borrow checker (i.e. instead of the type system preventing aliasing from happening and promising the programmer that it has done so, the programmer prevents aliasing from happening and promises the type system that they have done so), and this arguably makes it less useful in more complicated cases.

int foo[16];
int * foo_ptr = &foo[0];
int * restrict foo_restrict = foo;

> Meanwhile volatile serves only one clear purpose, you can use it to perform explicit stores and loads from a memory address, likely because you are doing MMIO. This is tricky to get right but since C doesn't provide any intrinsics for this purpose it's the only way that's even a little bit portable. All other uses of volatile are platform specific (in the good cases) or just voodoo / cargo cult C, sprinkled on by people who are hoping maybe the bug goes away if they write volatile in more places.

When accessing/modifying shared memory between processes/threads, volatile is some time the right thing to do to ensure stores/loads to/from memory. Thus it's not limited to MMIO.

Actually volatile must be used for any global variable modified from a signal handler. The most common pattern is to have volatile sig_atomic_t global flag set by the signal handler. I bet this is the most common use for volatile in userspace.

> The standard says that register is merely a hint [which today your compiler almost certainly ignores], that it be a good idea to put this variable in a register and serves no other purpose despite the restriction.

I know but there are few cases where it's still used. Trying to get the pointer from a variable declared as register will be instantly refused (which is great). Declaring a global variable with register (you're forced to indicate what register) will allow the compiler to optimize some operations because it knows the variable cannot change.
But I agree these are almost exceptions to the general rule that the compiler doesn't care much anymore.

> Meanwhile volatile serves only one clear purpose, you can use it to perform explicit stores and loads from a memory address, likely because you are doing MMIO.

The primary usage is for signals, even before MMIO. Userland code needs to use volatile and is certainly not fiddling with MMIO in general.

Avoiding this problem as much as possible is the goal of Rust's Editions system, even for Unsafe, and that's called out in this article. Rust has seen the last of big sweeping changes like garbage collection, but Editions enable it to be more agile than languages like C or C++ without the trauma associated with something like Python 3.x

Importantly Editions respect time's arrow. You write new code the new way, and your old code is unchanged. 10M lines or 10B lines doesn't matter, you aren't required to touch any of it. But Editions change how people think about what's possible and that means both that more adventurous changes are considered (knowing Editions might make the change practicable), and so often changes which wouldn't have been conceived at all without editions, ultimately turn out not to be incompatible and so the benefits accrue to everybody, not just on new editions.

The point which you claim has been contradicted is that it's hard to imagine - "The likes of C pointers being reworked to give them better formal properties"

But you don't present any evidence of such a reworking, only that people have managed to run some C software on CHERI which of course you'd expect since CHERI has been under development for some time explicitly to run C software. Here's an excerpt from Cambridge's description of CHERI, "The CHERI memory-protection features allow historically memory-unsafe programming languages such as C and C++ to be adapted to provide strong, compatible, and efficient protection against many currently widely exploited vulnerabilities". Nothing in there about formal properties, no proposals to the ISO committee, instead they are being pragmatic, what choice do they have after decades of C programmers resolute disinterest.

> If common platforms implemented CHERI or something like it, Rust probably would have never been conceived in the first place.

If "something like it" managed to make the vague semantics of C's logical machine better match the reality of a modern computer by adjusting the computer instead, perhaps you'd even be right. Maybe if this had happened in the 1990s, the elevator Graydon was annoyed by in 2005 would have actually worked.

CHERI is a long way from this fantasy, many grave C problems are orthogonal to CHERI but are completely solved in (safe) Rust. Which doesn't make CHERI a bad idea, it just highlights that Graydon's problem wasn't something a lot the lines of "there's this one thing about C I don't like, so I guess I will write an entirely new programming language" but rather that systematically none of the useful lessons of past decades of programming language theory had been adopted into systems programming languages people actually use.

C arguably solves this problem by ignoring it. The standard already says it's illegal to even evaluate an invalid not-null not-one-past-the-end pointer, regardless of whether you dereference it, and integer types are similarly allowed to have trap representations, so you just declare "all the weird CHERI stuff" to be UB or implementation details that "portable" code can't rely on, and then you've "ported" C to CHERI by making CHERI look like just another architecture. The most you get out of this is "If your program otherwise would have performed certain very specific kinds of UB, on CHERI it will *probably* trap at runtime instead." "Probably" because who knows what the optimizer will do. Contrast this with the much stronger guarantees you get out of safe Rust, where it will fail at compile time, every time.

That paper does indeed acknowledge the necessity of provenance in C (and C++). But what they're doing is patching the hole in these standards - a hole made about twenty years ago, whereas the Rust article is about something a little different - that hole never existed in Rust. In _unsafe_ Rust the nomicon tells you already that if you violate the provenance rules that's undefined behaviour. Unlike safe Rust, but just like C, unsafe Rust doesn't _prevent_ you from picking some random bytes and insisting they're definitely a pointer to something, however you now have incoherent nonsense, don't do that (on CHERI you fault immediately in unsafe Rust or C if you do this).

Aria proposes that usize - Rust's built in unsigned integer type that's typically 64-bit on a modern computer - should formally be the same size as an address _not_ the same size as a pointer as it is defined today. As a side effect, something desirable (to me at least, but I believe others too) falls out, while we're acknowledging that a pointer isn't just an integer with intent, we abolish the (ab)use of as casts to turn one into the other. Instead the programmer is expected to write what they meant, e.g. ptr.with_addr(address) gets you a pointer (maybe 129-bits) made from an address (maybe 64-bits) plus your promise that what you are doing is OK. Did you lie? Same rules as before, now your program is meaningless.

The C proposal can't go around adding methods to pointers, not least because C doesn't have methods and if it did it wouldn't have them on pointers, it just changes the formal semantics of the language to acknowledge the practical need for provenance. Existing correct C will remain correct, the TS just says why it's correct (or rather, why other seemingly reasonable C that doesn't work is not correct).

Also I expect that the committee will nod wisely and say that they don't have time to take this up right now, but please bring it back again next time, which is roughly what it has been doing since at least 2016, if your plan is to wait for them to fix C rather than learn a new language, don't figure on that happening any time soon.

I think it would help to distinguish between safe and unsafe Rust here.

Safe Rust already has a notion of "some bit patterns are not used by this type" and can therefore do "clever" things like packing different types together to make a type-safe untagged union. This has all of the advantages of a tagged union or Option/Either type, including compile-time type checking, but with no (or very little) overhead. Most of the time, you can and should try to make do with that, because you want the compiler to yell at you if you forget a case. From the perspective of unsafe Rust, even this might not be flexible enough, and so there does need to be an escape hatch where you can smuggle pointers around in other values, to some extent. But as the article acknowledges, the rules for this are a great deal more restrictive than just "make sure you end up with the same physical bit pattern you started with" - CHERI will trap if you try to do that, and Rust isn't really in a position to prevent CHERI from doing that in the general case.