Shared libraries [LWN.net]

Shared libraries

Posted Nov 26, 2025 14:01 UTC (Wed) by farnz (subscriber, #17727)
In reply to: Shared libraries by khim
Parent article: APT Rust requirement raises questions

You continue to misrepresent my position; I'm saying that we need the tooling to enable library developers to safely inline parts of their library into the caller, otherwise we're going to see C/C++ "you're holding it wrong" hacks because they're needed for performance.

The compiler cannot do this unaided - how does the compiler know whether a given small function is "hot path" in your users (and therefore needs to be inlined, with tooling assistance to prevent you from breaking the inlined paths), or "cold path" (and therefore can be out-of-line as a symbol in the shared binary)?

If you don't offer this as part of shared library support, then developers are going to hack around it in ways that tooling cannot check, because they're lazy - fixing the compiler to support this is a lot harder than hacking around it in interesting and unsupportable ways.

And the bit you keep ignoring is that I'm saying that we need tooling that's aware of this, and that can error out if you accidentally break your users. That is table stakes for this - otherwise you end up in the C++ situation.

to post comments

Shared libraries

Posted Nov 26, 2025 14:15 UTC (Wed) by khim (subscriber, #9252) [Link] (10 responses)

> You continue to misrepresent my position; I'm saying that we need the tooling to enable library developers to safely inline parts of their library into the caller,

Why?

> otherwise we're going to see C/C++ "you're holding it wrong" hacks because they're needed for performance.

Why that doesn't happen with bazillion other languages that offer similar capabilities?

> If you don't offer this as part of shared library support, then developers are going to hack around it in ways that tooling cannot check, because they're lazy - fixing the compiler to support this is a lot harder than hacking around it in interesting and unsupportable ways.

Not exposing “lightweight” objects in way that requires such hacks is even easier.

We are talking shared libraries that are used to offer some kind of stable ABI here. Supporting stable ABI is hard. People wouldn't add crazy hack where they couldn't guarantee if they would work or.

Look on Vulkan as an example of stable ABI that have to be performant, too: no crazy hacks, no inline function, just a bunch of non-opaque structures. With accessors to these structures in static libraries (that you may or may not want to use). Zero support from the compiler, zero magic, guaranteed to work, can be easily replicated in Rust, if needed.

> That is table stakes for this - otherwise you end up in the C++ situation.

Nope, you wouldn't. C++ situation arose because C++ already had that difference between .h and .c from the day one, because it already offered a way to have functions partially inlined in .so and partially in .h.

Don't give developers that capability — and they wouldn't use it. They would work with dynamic dispatch and limitations of dynamic dispatch. It's as simple as that.

Sure, optional availability of such capability wouldn't hurt… but lack of these are not show-stoppers at all. Developers can tolerate slow, they wouldn't tolerate “you need an entirely different API for shared libraries”, they would just link everything statically.

Shared libraries

Posted Nov 26, 2025 15:28 UTC (Wed) by farnz (subscriber, #17727) [Link] (8 responses)

Because these hacks do happen with other languages - the exceptions are those with a JIT, where it doesn't matter. And compiler developers just have to get used to this - like they've been forced to with C and C++. Either you provide the tooling to do a good job, or people come up with hacks.

And if you don't allow exposure of lightweight objects from a library, then you're ruling out such things as Option<&str> in the library's ABI surface. That's a lightweight object being exposed - and thus needs a stable ABI.

Vulkan's a really good example, because it does expose all sorts of things that get inlined into the caller - there's inline functions, for example, and those struct definitions are inlined into the caller, such that Vulkan cannot redefine the structures to a different size and still work with older libraries.

In an ideal world, there would be tooling that told people working on Vulkan "hey, you've changed this structure in such a way that it now has different size/alignment. You can't do that!", so that they couldn't make that mistake - but as it is, the normal C rules apply of "you're holding it wrong".

Shared libraries

Posted Nov 26, 2025 15:40 UTC (Wed) by khim (subscriber, #9252) [Link] (7 responses)

> In an ideal world, there would be tooling that told people working on Vulkan "hey, you've changed this structure in such a way that it now has different size/alignment. You can't do that!"

That tool is called Libabigail. And yes, it's used when Vulkan is upgraded.

> That's a lightweight object being exposed - and thus needs a stable ABI.

It just needs stable layout. But yes, it's an interesting dilemma there: if you just freeze the layout someone still needs to ensure that said Option<&str> would be valid when it crosses library boundary.

Whether this needs to be exposed to developers as some kind of universally accessible tooling or kept to the standard library is an interesting question. Today there are a lot of things that standard library does yet regular programs can not do. Providing inlining for methods of Option<str> without giving such ability to third-party library could be a good first step.

After all it's not possible to take self: Foo<Bar<Self>> today as target for the method, but only self: Rc<Self> or self: Arc<Self>… and sky haven't fallen on earth.

Shared libraries

Posted Nov 26, 2025 16:01 UTC (Wed) by Wol (subscriber, #4433) [Link] (2 responses)

> It just needs stable layout. But yes, it's an interesting dilemma there: if you just freeze the layout someone still needs to ensure that said Option<&str> would be valid when it crosses library boundary.

So you allow Rust to use the "extern" keyword. Which freezes the layout according to certain rules.

One likely mechanism is to require (a) that extern tells the compiler to use a simplistic layout as declared (no optimisation) so it's guaranteed to be consistent across programs. And then (b) (copying Rust "Editions") you allow "extern "version"" so you CAN update the layout, and more importantly, you can tell the Rust compiler about both layouts so it can auto-generate a shim, or you can choose not to, so it generates a linking error.

But either way, you do have to say "extern" is similar to "unsafe", in that you are relying on the programmer to enforce invariants, but that the compiler will do its job properly if the programmer does theirs (primarily ensures version is updated correctly).

Cheers,
Wol

Shared libraries

Posted Nov 26, 2025 16:11 UTC (Wed) by khim (subscriber, #9252) [Link] (1 responses)

> So you allow Rust to use the "extern" keyword. Which freezes the layout according to certain rules.

It's not enough to just freeze a layout. Option<&str> contains either None or reference to a valid UTF-8 string. Nothing else is allowed. If handling of it is inlined into both binary and library then they should agree about upholding such invariants.

> But either way, you do have to say "extern" is similar to "unsafe", in that you are relying on the programmer to enforce invariants

This would never work. The big advantage of Rust is the fact that compiler (and, most notably, not a developer) upholds many such invariants.

We need this guarantee kept across shared libraries boundary or it'll never work.

Shared libraries

Posted Nov 26, 2025 16:55 UTC (Wed) by Wol (subscriber, #4433) [Link]

> We need this guarantee kept across shared libraries boundary or it'll never work.

Which is farnz' point. You need tooling.

Which is the point of extern. It says this is where it's likely to go wrong. It tells the compiler "here is a boundary where abstractions mustn't leak". It tells the programmer "here is a boundary where you need to be careful what goes across".

It's a major improvement on C / C++ where the header file is part of the library, but contains loads of stuff that ends up in the application binary - a major abstraction leak that C / C++ sprinkles heavily with "here be dragons" pixie dust. A Rust compiler could cleanly enforce most of that, for example by ensuring structs passed through an extern have to be declared as extern, and have the same version as the function they are passing through.

Yes it's not perfect. But it's a damn sight better than what we have in the C ecosystem. Given Rust's concern about pointer provenance and stuff, you could use the same approach to argument provenance across an extern. Doesn't stop a programmer cheating and feeding two different versions of the same header file into the two different halves - application and library - in order to fool the compiler - but that's a clear breach of his obligation to enforce integrity across that boundary.

You might even be able to get the compiler (when compiling a library) to output an Intermediate Language Description of the call interface, which the compiler (when compiling an application) imports to guarantee compatibility. Again, you're then heavily dependent on versioning, and the programmer doing it properly, but it'll only take a few crashes on linking as attempts to do so cause problems, and the programmers will learn to "get it right".

Cheers,
Wol

Shared libraries

Posted Nov 26, 2025 16:50 UTC (Wed) by farnz (subscriber, #17727) [Link] (3 responses)

Libabigail is part of the tooling I'd like to see in proper shared library tooling - it covers size and alignment, but not field ordering within the structure (which also ought to be checked by tools). And, of course, if you want something that "just works", you need the size and alignment of structures to not be hard-coded in the user, but rather filled in by the dynamic linker via a COPY or SIZE relocation (causing you to have to do arithmetic all over the place when a structure from the dynamic library is embedded in a user-supplied structure).

Basically, what I want is tooling that tells me if I've made any change that breaks either my API (which currently exists) or my ABI, and that I can have in my build process to tell me when I've made a mistake. And, as Vulkan shows, this is not just about having a way to express "this item has a fixed layout regardless of compiler", but also "you said this was supposed to be unchanging, but you've changed it in a way that's significant. Did you mean to do that?", complete with the ability to reserve space for future expansion (and not in the C-like hacky way of unused fields, but a very deliberate and well-thought through way to reserve sufficient space with good enough alignment for what you might want to fill in that space with, allowing for some fields being fixed in location because they're exposed ABI, and others being mobile because they're opaque to callers).

And yes, this tooling needs co-operation with the compiler - that's table stakes for doing a stable ABI properly. But it also needs a bunch of other things, so that I have to tell my tools that yes, I mean to break the stable ABI here, just as I have to tell my tools (via SemVer changes) that yes, I mean to break the stable API.

Shared libraries

Posted Nov 26, 2025 17:04 UTC (Wed) by khim (subscriber, #9252) [Link] (2 responses)

> And, of course, if you want something that "just works", you need the size and alignment of structures to not be hard-coded in the user, but rather filled in by the dynamic linker via a COPY or SIZE relocation (causing you to have to do arithmetic all over the place when a structure from the dynamic library is embedded in a user-supplied structure).

It's not “of course”, but “do we really need it?” If structure has a potential of changing size then it's highly unlikely then it's “trivial” structure that would be used somewhere in the critical path. More likely it's some descriptor that needs some non-trivial work to handle it. Swift handles such structures with runtime descriptors and it may be enough to do that.

I would say that if you would try to do that you'll end up with “success” of OSI protocols: lots of hype, lots of talks, zero working implementations.

I would rather see that being marked as “maybe in 10 years if we would have the resources we might attempt that” rather then “that's something that we would use as justification not to do anything”.

> But it also needs a bunch of other things

It needs some things, yes. You are trying to stuff it full of things that are not, strictly speaking, needed, but only desired. That's willful ignorance of RFC1925: It is always possible to aglutenate multiple separate problems into a single complex interdependent solution. In most cases this is a bad idea.

I would say that starting with dynamic dispatch everywhere but some “blessed” types from the standard library (which is essentially what Swift did) would get us 90% there (if not 99% there) and the remaining 1% may be discussed later (or handled with hacky solutions, if needed).

But yes, basic library types like Option<&str> needs to be covered, somehow.

Shared libraries

Posted Nov 26, 2025 17:14 UTC (Wed) by farnz (subscriber, #17727) [Link] (1 responses)

Runtime descriptors get filled in by the dynamic linker - and all of the things I'm describing are things that are done semi-manually to maintain a stable ABI by Vulkan and glibc developers today.

I'm not arguing that these are required for a minimal solution - but rather that all of these problems have to be solved if you're going to have a stable ABI, and given that Rust needs tooling updates anyway to have a stable ABI, it would be better to have tooling that prevents accidental ABI breakage from day 1 of the Rust stable ABI for dynamic linking, rather than (as you seem to be suggesting) relying on "programmers don't make that mistake".

After all, the lesson of C++ UB is that "programmers don't make that mistake" is virtually always false.

Shared libraries

Posted Nov 26, 2025 17:23 UTC (Wed) by khim (subscriber, #9252) [Link]

> Runtime descriptors get filled in by the dynamic linker

Nope. Not even remotely close. Rather it's extension of current dyn Trait scheme: in the vtable there are size of the object (so it can be allocated on stack, alloca-style), reference to the constructor (so you can create a vector) and so on. Then generic may have one, common code for that thing and work — even without any dynamic linkage involved.

Very natural extension of the Rust language, not special code for the dynamic libraries. It would be well-received even without dynamic libraries, the ability to have true polymorphic callback is something people desire often.

> After all, the lesson of C++ UB is that "programmers don't make that mistake" is virtually always false.

Yes, but that doesn't mean we have to invent something grandiose, that would require next 10 or 20 years to develop. Simple solution may be better, because it's actually implementable… and we know, from Swift example, what can be implemented in the limited time.

Hint: anything that requires changes to the dynamic linker automatically makes the whole thing DOA. Simply because on Windows dynamic linker is part of the OS and if this scheme wouldn't support Windows then nobody serious would use it.

Shared libraries

Posted Nov 26, 2025 16:09 UTC (Wed) by excors (subscriber, #95769) [Link]

> Look on Vulkan as an example of stable ABI that have to be performant, too

Vulkan's ABI is stable and extensible and C-based (allowing FFI from many languages), with the tradeoff that it's painful and bug-prone to use directly.

That's only acceptable because most application developers won't use it directly. They'll use something like https://github.com/KhronosGroup/Vulkan-Hpp which provides an easier-to-use and safer C++ API, but gives no promises about API stability. That's a header-only library, so effectively it's statically linked into every application. (Header-only libraries are popular because C++'s ABI support is so poor that even static linking of separately-compiled libraries is unreliable). Or they'll use some other wrapper or engine, or write their own, with the same issues.

So now the problem is: how could you implement something like Vulkan-Hpp as a shared library with a stable ABI? And the current answer is, you can't. Vulkan isn't solving the problem of safe ABIs; it's just shifting it to another layer which doesn't solve it either.

If you want shared libraries and safety (which I think was the premise of this thread), it seems worth trying to come up with a better solution than that.

(Vulkan is also a peculiar example because it has a complex architecture where applications link (dynamically or sometimes statically) to a Vulkan Loader which dynamically links to multiple independent driver implementations, and drivers export the whole Vulkan ABI through a GetProcAddr interface, and applications call a trampoline function in the Loader that dispatches to the appropriate driver. (If you have multiple GPUs, the application might use multiple drivers at once). Applications can avoid the trampoline cost by using a GetProcAddr to get a function pointer directly into the driver, though it may actually be a function pointer into a separate Layer library that can intercept and modify each call before the driver sees it (for debugging etc). I wouldn't call that "no crazy hacks". (https://github.com/KhronosGroup/Vulkan-Loader/blob/main/d...))