Rearranging across the interface [LWN.net]

Rearranging across the interface

Posted Feb 26, 2025 15:45 UTC (Wed) by Wol (subscriber, #4433)
In reply to: Rearranging across the interface by farnz
Parent article: Rewriting essential Linux packages in Rust

So what you're saying is, if you the programmer move the checks across the interface boundary, the compiler has no way of knowing you've done it?

Hmmm ...

That is an edge case, but equally, you do want the compiler to catch it, and I can see why it wouldn't ... but if you're building a library I find it hard to see why you the programmer would want to do it - surely you'd either have both sides of the interface in a single crate, or you're explicitly moving stuff between a library and an application ... not good ...

Cheers,
Wol

to post comments

Rearranging across the interface

Posted Feb 26, 2025 16:32 UTC (Wed) by farnz (subscriber, #17727) [Link] (4 responses)

No; I'm saying that if the compiler doesn't even know that this is an interface boundary, why would it bother detecting that you've moved code across the boundary in a fashion that's safe when statically linked, but not when dynamically linked?

Put concretely, in the private module raw_vec.rs (none of which is exposed as an interface boundary), I move a check from shrink_unchecked to shrink; how is the compiler supposed to know that this is not a safe movement to make, given that shrink is the only caller of shrink_unchecked? Further, how it is supposed to know that moving a check from shrink to shrink_unchecked is safe? And, just to make it lovely and hard, how is it supposed to distinguish "this check is safe to move freely" from "this check must not move"?

And note that "checks" and "security fixes" look exactly the same to the compiler; some code has changed. How is the compiler supposed to distinguish a "good" change from a "bad" change?

Rearranging across the interface

Posted Feb 26, 2025 21:43 UTC (Wed) by Wol (subscriber, #4433) [Link] (3 responses)

> No; I'm saying that if the compiler doesn't even know that this is an interface boundary, why would it bother detecting that you've moved code across the boundary in a fashion that's safe when statically linked, but not when dynamically linked?

Because if the whole aim of this is to create a dynamic library, the compiler NEEDS to know this is an interface boundary, no?

Cheers,
Wol

Rearranging across the interface

Posted Feb 27, 2025 10:44 UTC (Thu) by farnz (subscriber, #17727) [Link] (2 responses)

But then you're getting into a mess around defining what is, and is not, a safe code change inside the ABI boundary. If you do make the internals of a crate (not the exported interface) the ABI boundary, you're now in a position where the compiler has to make a judgement call - "is this change inside the internals of a library a bad change, or a good change?".

Note that when making this judgement call, it can't just look at things like "is this moving a check across an internal boundary", since some moves across an internal boundary are safe, nor can you condition it on removing a check from inside the boundary (since I may remove an internal check that is guaranteed to be true since all the inline functions that can call this have always done an equivalent check, and I'm no longer expecting more inline functions without the check).

Rearranging across the interface

Posted Feb 27, 2025 14:07 UTC (Thu) by Wol (subscriber, #4433) [Link] (1 responses)

but if the crate IS the compilation object (as it would be if it's a library, no?) then surely the external boundary is the external declaration - what the library provides to all and sundry - then there's no problem with any internal moves?

If an external application cannot see the boundary, then it's not a boundary! So you'd need to include the definition of all the Ts in Vec<T> you wanted to export, but the idea is that the crate presents a frozen interface to the outside world, and what goes on inside the crate is none of the caller's business. So internal boundaries aren't boundaries.

Cheers,
Wol

Rearranging across the interface

Posted Feb 27, 2025 14:12 UTC (Thu) by farnz (subscriber, #17727) [Link]

No, for performance reasons. We inline parts of our libraries (even in C, where the inlined parts go in the .h file) into their callers because the result of doing so is a massive performance boost from the optimizer - which can do things like reason "hey, len can't be zero here, so I can eliminate the code that handles the empty list case completely".

To get the sort of boundary you're describing, we do static linking and carefully hand-crafted interfaces for plugins. That's the state of play today, for everything from assembly through C to Agda and Idrs; the goal, however, is to dynamically link, which means that we need to go deeper. And then we have a problem, because the moment you go deeper, your boundaries stop applying, thanks to inlining.

Rearranging across the interface

Posted Feb 26, 2025 18:04 UTC (Wed) by excors (subscriber, #95769) [Link] (2 responses)

As I understand it, the issue is that "interface boundary" can mean either "API boundary" or "ABI boundary". `Vec<T, A>::shrink_to_fit` is an API boundary; it's publicly documented and safe and has stability guarantees. But in a hypothetical Rust ABI, that API couldn't be an ABI boundary, because it depends on generic type parameters that aren't known when the .so is compiled.

`RawVecInner<A>::shrink_to_fit` could be an ABI boundary, because that doesn't depend on `T` (and we'll ignore `A`), but it's currently not an API boundary. It can't be made into a public API because its safety depends on non-trivial preconditions (like being told the correct alignment of `T`) and that'd be terrible API design - preconditions should be as tightly scoped as possible, within a function or module or crate. So you'd have to invent a new category of interface boundary, which is both an internal API and an ABI, with stability guarantees (including for the non-machine-checkable safety preconditions) and with tooling to help you fulfil those guarantees, which sounds really hard.

Rearranging across the interface

Posted Feb 26, 2025 22:46 UTC (Wed) by Wol (subscriber, #4433) [Link] (1 responses)

> So you'd have to invent a new category of interface boundary, which is both an internal API and an ABI, with stability guarantees (including for the non-machine-checkable safety preconditions) and with tooling to help you fulfil those guarantees, which sounds really hard.

Like putting the equivalent of a C .h in the crate?

But I would have thought if the compiler can prove the preconditions as part of a monolithic compilation, surely it must be able to encode them in some sort of .h interface in a library crate?

Of course, if you get two libraries calling each other, then the compiler might have to inject glue code to rearrange the structures passed bwtween the two :-)

Cheers,
Wol

Rearranging across the interface

Posted Feb 27, 2025 12:43 UTC (Thu) by farnz (subscriber, #17727) [Link]

The compiler doesn't prove anything for the danger cases; it relies on the human assertion that they've checked that this unsafe block is safe, given the code that they can see today.

The challenge is that we're talking about separating the unsafe block (in an inline function) from the unsafe fn it calls (in the shared object); this means that the human not only has to consider the unsafe code as it stands today, but all possible future and past variants on the unsafe code, otherwise Rust's safety promise is not upheld.

That's clearly an intractable problem; the question is about reducing it down to a tractable problem. There's three basic routes to make it tractable:

Ignore the problem, and rely on the humans being infallible and remembering to change a "version identifier" when making a change to both the unsafe fn and its inlined callers. This makes it far too likely that you'll breach Rust's safety promises for Rust to adopt this.
Ensure that there's an ABI change whenever anything in the body of the unsafe fn changes. This is problematic, because security fixes are likely to change the body, and those are the cases where you most want to be able to swap in a new shared object.
Build a reverse tree of inline functions in this library that call the unsafe fn, and ensure that if any of them changes, the unsafe fn's ABI changes. This means that you can change the unsafe fn freely as needed for a security fix, but if you change any inlined caller, or add a new inlined caller, you change the ABI of the unsafe fn and require a new shared object, even if this change was harmless.

There's room to be sophisticated with symbol versioning in all cases; for example, you can have a human assert that this version of the unsafe fn is compatible with the inlined callers from older versions (thus allowing a swap of a shared object), or in case 3 you can use it to allow new inlined callers to use a new shared object, while allowing the existing ones to use either old or new shared objects.

In all cases, though, the trouble is preventing the human proofs of correctness being invalidated by creating new combinations of inline functions and out-of-line unsafe code that weren't present in any source version; you want the combinations to be ones that a human has approved.