Function Pointer cast

Posted Jan 29, 2024 20:09 UTC (Mon) by tialaramex (subscriber, #21167)
Parent article: Defining the Rust 2024 edition

I am known to not like 'as' casts anyway, but I am particularly enthusiastic about banning people from writing this naive function pointer 'as' cast if we're not (at least for now) banning the others.

I think even after you discount the people who'll go "I didn't want a function pointer anyway, WTF?" because they've made a typo - the remaining people mostly don't realise what a bad idea this actually is and abolishing the 'as' cast helps clarify that.

A pointer has three components, an address (which is an integer), an address space (Rust doesn't care about this, good luck), and a validity context (nebulous but in hardware sometimes represented as some sort of address range)

The 'as' cast discards both the address space and context, leaving you just that integer. If what you needed was literally an integer to distinguish function A from function B, which you are *very certain* are literally different machine code (because if they are not you lose) and they live in the same address space (Rust doesn't guarantee that) and your integer is big enough (this either) then this will work. Otherwise it's just some integer.

If you intend to reconstitute the integer into a pointer, you need the address space and context or you're entirely relying on the compiler to happen to decide what you're doing is OK. Maybe it is OK. Maybe it isn't, we don't know. First Bad news, Rust provides *no way* to specify that address space, I hope the compiler blesses you. Second Bad news, the only mechanism Rust provides to supply context is that you can hand over another pointer you have and say "The same context as this". If you're lying or wrong, that's Undefined Behaviour.

Function Pointer cast

Posted Jan 29, 2024 21:13 UTC (Mon) by mb (subscriber, #50428) [Link] (6 responses)

>an address space (Rust doesn't care about this, good luck),

This is known and being worked on

https://doc.rust-lang.org/std/ptr/index.html#strict-prove...

>Rust provides *no way* to specify that address space

https://doc.rust-lang.org/std/primitive.pointer.html#meth...

Function Pointer cast

Posted Jan 30, 2024 12:36 UTC (Tue) by khim (subscriber, #9252) [Link] (5 responses)

> > Rust provides *no way* to specify that address space

https://doc.rust-lang.org/std/primitive.pointer.html#meth...

That method returns *const T which is not fn. And I couldn't find a way to turn it into fn that Miri would like.

Can you show the full example:
1. Convert fn function pointer to usize
2. Store it into array of bytes (to_ne_bytes) then reverse it twice.
3. Convert back into interger (from_ne_bytes).
4. Turn that into function which can be called.

From what I'm seeing you may only ever turn integer that points to function foo with the use of another pointer to that exact function foo which kinda defeats the purpose: if I have an integer that's in reality address of function then it's probably address of function bar or baz… otherwise what's the point of all that excercise?

Function Pointer cast

Posted Jan 30, 2024 13:25 UTC (Tue) by tialaramex (subscriber, #21167) [Link] (4 responses)

I'm pretty sure Miri cannot accept the shenanigans proposed. For the reason I gave, your function pointer knew it was a function pointer while your integer is just an address. On the computer you're probably using to write this comment it turns out, de facto, that all your addresses are in the same flat address space, but Miri doesn't know that. Your function pointers also point into R+X memory where the machine code to implement them exists, which live "forever", so just knowing the address is enough. But again Miri isn't comfortable with this claim.

You can have a *mut T instead of a *const T if you want, they both exist in this API, so that's nice. But you can't safely make a function pointer this way AFAIK.

Also, you definitely can't have an actual function. Unlike C or C++, Rust's functions have unique unnameable zero size types, more similar to a C++ lambda. So you definitely can't make those, they're figments of the compiler's imagination.

let mut why_not = core::mem::needs_drop::<u8>; // Sure. Value of a Zero Size Type representing the abstract idea of this predicate
why_not = core::mem::needs_drop::<i8>; // Won't compile, they're both predicates with identical signatures and the exact same implementation but in theory they're different functions, thus it's a type mismatch.

... You can make function pointers though, which are the thing you'd get in C or C++ when you try to put the function in a variable. You can undoubtedly make a function pointer from an integer and have it work. But no, you probably cannot convince Miri that's OK.

Function Pointer cast

Posted Jan 30, 2024 13:50 UTC (Tue) by khim (subscriber, #9252) [Link] (3 responses)

> So you definitely can't make those, they're figments of the compiler's imagination.

No, they are not. You may write something like this:

fn main() {
    let add_or_sub = if std::env::args().len() % 2 == 0 { add } else { sub };
    println!("{}", add_or_sub(42, 2));
}

fn add(x: i32, y: i32) -> i32 {
    x + y
}

fn sub(x: i32, y: i32) -> i32 {
    x - y
}

How may add_or_sub variable even exist if function pointers are just a figments of the compiler's imagination?

> You can undoubtedly make a function pointer from an integer and have it work. But no, you probably cannot convince Miri that's OK.

I would consider it a defect in Miri and a pretty serious one. While Miri is not supposed to handle all possible Rust programs there are way to many APIs that assume that you may create function pointers from integers. But yeah, ultimately Miri is just a model of Rust, not the full Rust.

Function Pointer cast

Posted Jan 30, 2024 15:52 UTC (Tue) by tialaramex (subscriber, #21167) [Link] (2 responses)

What you've got there is just a function pointer, of type fn(i32, i32) -> i32 Although Rust doesn't have the uh, exciting variety of integer coercions of C, it does have coercions, and in particular what your snippet does is coerce the two ZSTs (which are distinct types) into merely function pointers (which are compatible if the functions have the same signature). Watch what happens if we modify your code a little:

fn main() {
    let mut add_or_sub = add;
    add_or_sub = if std::env::args().len() % 2 == 0 { add } else { sub };
    println!("{}", add_or_sub(42, 2));
}

error[E0308]: mismatched types
 --> src/main.rs:3:68
  |
3 |     add_or_sub = if std::env::args().len() % 2 == 0 { add } else { sub };
  |                                                                    ^^^ expected fn item, found a different fn item
  |
  = note: expected fn item `fn(_, _) -> _ {add}`
             found fn item `fn(_, _) -> _ {sub}`
  = note: different fn items have unique types, even if their signatures are the same
  = help: consider casting both fn items to fn pointers using `as fn(i32, i32) -> i32`

Function Pointer cast

Posted Jan 30, 2024 16:59 UTC (Tue) by khim (subscriber, #9252) [Link] (1 responses)

But isn't the fact that you now have add_or_sub which is not “a mere function pointer” means that you actually can make these (with transmute) e.g.?

Function Pointer cast

Posted Jan 30, 2024 18:18 UTC (Tue) by tialaramex (subscriber, #21167) [Link]

In your program add_or_sub was a function pointer, specifically it was a function pointer to some function which takes two i32s and returns just one, fn(i32, i32) -> i32. The function pointer is probably (on modern computers) a 64-bit value which in practice is just the address in memory of some machine code.

But in my response add_or_sub is now a function item, a Zero Size variable which is always (in this particular case) our add function. ZSTs have a singular value, which is why we don't need storage space for them, the value of the (now mis-named) add_or_sub is always just add, a specific function we wrote to add numbers together. Even another function with not just the same signature, but exactly the same body and resulting machine code is a different function and so cannot be assigned to the (again, now misnamed) add_or_sub variable.

Transmute is a red herring, transmuting things into a ZST is silly. If you can *spell* the ZST then you're welcome to have one, since it has only a single value the compiler knows from the type what its value is, we don't need to "transmute" it. However like a lambda type these function items are unnameable, so you can't do that.

Function pointers are very nameable, and I'd guess you've been thinking about function pointers all along, but I wanted to emphasise that Rust does have (and make good use of) types signifying specifically the function, not just a pointer.

Yes I imagine you can take a magic integer (with some well chosen value) and transmute it into a function pointer, and (on a typical modern computer) you could call the function via that pointer and it'd work. I am not surprised that MIRI cannot justify this, and I think there aren't a lot cases where anybody has a good reason to do it, so if "Miri doesn't like it" dissuades someone from doing this that's probably good.

Function Pointer cast

Posted Jan 30, 2024 12:23 UTC (Tue) by khim (subscriber, #9252) [Link] (5 responses)

> Second Bad news, the only mechanism Rust provides to supply context is that you can hand over another pointer you have and say "The same context as this".

Interesting. How do you do that with functions pointers? Heck, why do you do that with function pointers? They all are part of the same address space, your binary, why do they need that?

Function Pointer cast

Posted Jan 30, 2024 19:15 UTC (Tue) by tialaramex (subscriber, #21167) [Link] (4 responses)

I don't know why anybody would want to do this (build a function pointer from an integer) either except in some (already deeply unsafe) dynamic plugin linking type situation.

In that case ultimately, since they could just lie I'm in their hands anyway - maybe this *is* a valid pointer to an implementation of the function I wanted... but alas that implementation is in Motorola 68000 machine code and this an Intel PC so I'm screwed anyway.

My guess is that pragmatically it will just work to transmute the magic numbers into the function pointer, but that's just a guess and I can't begin to imagine how you could convince the machine that this definitely works.

Function Pointer cast

Posted Feb 2, 2024 17:58 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (3 responses)

> I can't begin to imagine how you could convince the machine that this definitely works.

Do you mean abstract machine here? Or perhaps the model (like MIRI)? The compiler? Because the hardware machine doesn't care and will try whatever you point the instruction pointer at (well, I suppose there are some security mechanisms that can say "no" these days).

Function Pointer cast

Posted Feb 7, 2024 1:51 UTC (Wed) by tialaramex (subscriber, #21167) [Link] (2 responses)

Miri. I can convince you, or some committee, but I can't imagine how I'd persuade Miri.

Function Pointer cast

Posted Feb 8, 2024 0:34 UTC (Thu) by milesrout (subscriber, #126894) [Link] (1 responses)

Surely the solution to that problem is to fix miri so it corresponds to what actual machines do, rather than try to "fix" code that works fine on real machines but doesn't work on miri? The point of an abstract machine is to abstract away the irrelevant differences in the details between different concrete machines. But all actual targets for Rust have a single address space, at least for code (and most have a single address space for code and data). So there is no actual reason why this should be undefined.

Then again I would say the same thing for the "undefined behaviour" that is dereferencing an invalid pointer. It's possible to define it perfectly well: you get the result of dereferencing it. And if that means that your system faults, then it faults. If it means you read garbage, you get garbage. But it doesn't give the compiler permission to compile out your later pointer validity checks.

Function Pointer cast

Posted Feb 8, 2024 6:50 UTC (Thu) by mb (subscriber, #50428) [Link]

> dereferencing an invalid pointer. It's possible to define it perfectly well

Yes. But that probably costs you most optimization opportunities and all parts of the code.
You can have that behavior today by disabling optimization in C.
Not sure, if it's possible in Rust. Probably not. Rust is quite strict about assuming no-UB.

"Coding to the machine" and "getting what the hardware does" is impossible in real programs.