LWN: Comments on "Toward safe transmutation in Rust"

Transmute to user-defined types with all public fields?

riking — Tue, 29 Oct 2024 23:21:51 +0000

Yes, Assume::SAFETY will do nothing to a type whose fields are all public (recursively).

Constructor checking

NYKevin — Mon, 28 Oct 2024 07:43:18 +0000

In this case the type would want to be able to mark itself as able to be transmuted to, but only when the type itself is doing the transmuting.

You can sort of do that now:

// Simple one-field struct as an example, can be replaced with a more complicated struct.
#[repr(C)] // But it does need to be repr(C) or repr(transparent)
struct MyType(u64);

use std::convert::TryFrom;

impl<'a> TryFrom<&'a [u8]> for &'a MyType {
    type Error = &'static str;  // XXX: In a real codebase, use a proper Error type and not a string.
    fn try_from(x: &'a [u8]) -> Result<Self, &'static str> {
        if x.len() != std::mem::size_of::<MyType>() {
            return Err("Wrong size!");
        }
        let ptr: *const MyType = unsafe { std::mem::transmute(x.as_ptr()) };
        if ptr.is_aligned() {
            Ok(unsafe { &*ptr })
        } else {
            Err("Not aligned!")
        }
    }
}

// Implementation for &'a mut [u8] omitted because it's nearly identical.

// Now safe code can call try_from() and try_into() for this conversion.

But this is just a thin wrapper around transmute. It's sound, in this case, but if MyType is more complicated, then you have all the problems that the article describes. The compiler is not going to do very much for you here (the only check that the compiler performs in the above example code is to make sure that a pointer-to-u8 is the same size as a pointer-to-MyType, which is rather obvious anyway).

(If this looks like a lot of boilerplate code, bear in mind that Rust has macros, so you don't have to write all of this out repeatedly if you're doing it a lot.)

garbage in, garbage out

amarao — Sun, 27 Oct 2024 21:16:29 +0000

This completely breaks the foundation of modern cves, garbage in, garbage out.

Transmute to user-defined types with all public fields?

asahilina — Sat, 26 Oct 2024 15:27:37 +0000

That is the simplified version and what is already being done with macros in certain crates: you can transmute into types where all bit patterns are valid. But what is being researched here seems to be more general, e.g. you would in theory be able to transmute [NonZeroU8; 4] into NonZeroU32, though not vice versa, despite neither type having all bit patterns valid.

Transmute to user-defined types with all public fields?

jpab — Fri, 25 Oct 2024 17:25:14 +0000

I guess I'll extend this and say that I do see value in the more complicated safer-but-still-unsafe API, but having a subset of the capability available through a fully-safe API could be quite neat.

Transmute to user-defined types with all public fields?

jpab — Fri, 25 Oct 2024 15:54:09 +0000

I agree, and would suggest that a possible simpler API here would be a safe transmute that applies the validity, alignment, and lifetime requirements, but is only available to transmute to types that *could be constructed directly* in the scope you're in.

That is, if I'm in a scope where I can write a literal `Foo { x, y, z }` then I should also be able to transmute from an appropriate source to a Foo, without using `unsafe`, but relying on the compiler to apply all the checks.

If I'm in a scope where I can't directly construct a Foo (ie, to get one then I need to call some function like Foo::new()) then I clearly need unsafe. Though I'm not sure when I would want to do that anyway.

Visibility of direct construction is already an essential element of enforcing invariants in values. And since safe transmute already requires compiler magic to enforce the various structural safety constraints it could presumably check visibility as well.

There is some care needed because you need to have visibility not only to directly construct a value of the target type but also to directly construct values of each field type (and of their fields recursively).

Transmute to user-defined types with all public fields?

roc — Fri, 25 Oct 2024 00:49:31 +0000

Fine, a user-defined type with all public fields and no padding should be a valid transmutation target if its fields are

Transmute to user-defined types with all public fields?

heftig — Thu, 24 Oct 2024 20:30:38 +0000

If the structure has padding bits, they generally must all be zero.

Constructor checking

ringerc — Thu, 24 Oct 2024 19:01:21 +0000

This can also be done by giving the type another constructor or helper method that constructs an instance of the type by transmutation, right?

In this case the type would want to be able to mark itself as able to be transmuted to, but only when the type itself is doing the transmuting. Essentially a private access marker trait. I don't know enough Rust to know if this is possible.

Transmute to user-defined types with all public fields?

roc — Thu, 24 Oct 2024 11:17:34 +0000

Seems to me that a user-defined type whose fields are all public can't have any invariants so you should be able to transmute to it.

Should aim to deprecate std::mem::transmute()

fishface60 — Thu, 24 Oct 2024 09:52:15 +0000

> It may seem as though requiring the use of unsafe to do transmutation
> represents a lack of progress. But this design has the advantage that the
> programmer only needs to assert the safety of the specific invariant that
> the compiler is unable to prove — the above code still uses the
> compile-time checks for bit-validity, alignment, and lifetime problems.

I hope this gets used more. You've still got to recheck the safety invariants when things change, but this should help when a type you're transmuting changes underneath you.

I think there would have to be an eventual goal of deprecating std::mem::transmute() though, since TransmuteFrom::<_, Assume::SAFETY>::transmute(src) is more work to use and people will tend towards using the simplest option, so the language should aim to make the best option the simplest to use.

Constructor checking

epa — Thu, 24 Oct 2024 07:53:52 +0000

A user-defined type may have its own constructor that does real work, like opening a network connection. Or maybe the constructor just checks some invariant, like requiring an even number in the above example. Often you could check that invariant after assigning the object’s fields. You can express it as taking an already-populated object and validating it before allowing it to be used.

In that case the language could let you define a ‘check’ method as an extra step which runs after the constructor. When casting (‘transmuting’) an area of memory to an instance of the type, the check method is run. Then the requirement like ‘must be even’ can be expressed in code and the programmer doesn’t have to promise the compiler he or she has already checked it.

zerocopy

gspr — Thu, 24 Oct 2024 07:43:15 +0000

Indeed, zerocopy figures prominently in the talk and in this LWN article.

zerocopy

philipptoelke — Thu, 24 Oct 2024 06:26:43 +0000

I have used https://docs.rs/zerocopy/latest/zerocopy/index.html successfully to parse packet data from buffers into structs. I think it comes from the Google Fuchsia team.

Cast

intelfx — Wed, 23 Oct 2024 21:01:40 +0000

> As far as I lnow, you can cast your char* buffer into a struct whatever* and then access the struct's fields with no copy.

Yes, and that's exactly the kind of UB-laden thing that is much harder to do (properly) than to talk about it. It is absolutely invalid to "just" cast an arbitrary buffer into a "struct whatever", unless multiple very specific conditions are met. It has to be done with utter attention to detail (*if* it even can be done under specific circumstances), and the article is precisely about offloading some of that utter care and attention to the compiler.

Cast

daroc — Wed, 23 Oct 2024 20:41:26 +0000

Yes, certainly you can. That is an example of using type casting — transmutation, in the Rust lingo.

The traditional parsers I was trying to distinguish are libraries that parse structures using actual parsing techniques — parser combinators, grammars, etc.

Cast

ju3Ceemi — Wed, 23 Oct 2024 20:37:58 +0000

I do not understand this : "In a traditional parser, the programmer would have to copy all of the data in the UDP header at least once in order to move it from the incoming buffer into a structure"

As far as I lnow, you can cast your char* buffer into a struct whatever* and then access the struct's fields with no copy.