Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 2:34 UTC (Tue) by Conan_Kudo (subscriber, #103240)
In reply to: Filesystem-oriented flags: sad, messy and not going away by tux3
Parent article: Filesystem-oriented flags: sad, messy and not going away

Afaik, Debian ships rust programs, and if I know anything about Debian packaging, static linking is not even close to being an option =]

Unfortunately, you'd be wrong. Debian ships packages with piles of source code, just as Fedora does. Applications statically link everything, because otherwise every rebuild of the compiler would necessitate rebuilding everything. It's just not practical. Maybe one day, the Rust community will care about us and work toward defining a native stable ABI. But I won't hold my breath. The Rust community thinks it's okay to have to constantly build everything for every change, despite the huge downsides.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 10:25 UTC (Tue) by roc (subscriber, #30627) [Link] (4 responses)

The stability of the C++ ABI also has massive downsides: https://cor3ntin.github.io/posts/abi/

In theory we could escape the dilemma by creating a stable ABI for shared libraries that you opt into at build time that would mostly be only for Linux distros. But even that would constrain language and library evolution as well as being a ton of work that no-one is really motivated to do.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 11:53 UTC (Tue) by Wol (subscriber, #4433) [Link]

The problem is too many developers only develop to solve the immediate problem. Spend a bit of time to define the *general* problem, define a state table and design the API to solve said state table, and then by all means just solve your bit of it.

That way, you can extend the function to fill the state table as and when, without having to redesign the interface.

Cheers,
Wol

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 21:21 UTC (Tue) by quotemstr (subscriber, #45331) [Link] (2 responses)

> In theory we could escape the dilemma by creating a stable ABI for shared libraries

COM solved that problem decades ago. We should seriously consider adopting something a lot like it. A stable object ABI that allows for both efficient intraprocess calling and extensible interprocess remoting is extremely powerful.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 18, 2020 8:53 UTC (Wed) by roc (subscriber, #30627) [Link] (1 responses)

Lowest-common-denominator ABIs like COM are awful to work with.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 18, 2020 11:07 UTC (Wed) by k3ninho (subscriber, #50375) [Link]

We could manage versioning with a request broker*.
"Do you speak the ABI of versions in this range?"
"Not all of them, I can fall back to v.A.B.C as most recent. Is that OK?"
"Confirmed OK."

*: common object request broker isn't a model, it's an architecture ;-)

K3n.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 21:33 UTC (Tue) by rvolgers (guest, #63218) [Link] (4 responses)

That "huge downsides" article is not entirely fair perhaps.

Rust does link dynamically to libc, and many Rust programs link to e.g. OpenSSL because Rust has good support for using dynamically linked C libraries. In fact, there are dynamic libraries with a C ABI that are implemented in Rust (librsvg comes to mind).

Rust has really good support for dynamic linking! It just doesn't have good support for dynamic linking using its *native ABI*. You could look at this as discouraging dynamic linking, but you can also look at it as encouraging dynamic linking that integrates well with the rest of the open source ecosystem by using the C ABI as a universal interface.

Also, a ton of Rust code is just not desirable to dynamically link, ever. We could do a cute experiment and compile some popular Rust programs while absolutely forbidding the compiler to inline functions between different crates (i.e. "libraries"). Pretty sure that will cause a code size explosion and speed reduction that will make people scream a lot louder than using a couple more kb of disk space.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 21:44 UTC (Tue) by rvolgers (guest, #63218) [Link] (2 responses)

To clarify my point a bit more: the way native Rust APIs are written is just very different from how C APIs are written. Due to the rich type system you can have a lot more back-and-forth between a library and its consumers than you would have in a C API.

Consider for example the Iterator trait in Rust. People expect code written using iterators to compile down to something that you would find hard to distinguish from a C for loop in disassembly, which requires the compiler to inline a whole bunch of calls to tiny functions and remove some intermediate values. And not all those tiny functions have to come from the same library, they can come from many different ones, and many will have generic arguments or callbacks with generic arguments from still other libraries.

And it's not just Iterator, the same goes for asynchronous I/O using Futures, and probably more absolutely core functionality that I'm forgetting about right now. As soon as parts of that become dynamically linked, you start having to make some really tough calls about what the compiler can statically assume and optimize out.

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 21:47 UTC (Tue) by areilly (subscriber, #87829) [Link]

Agree completely. You just beat me to it!

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 20, 2020 10:15 UTC (Fri) by jezuch (subscriber, #52988) [Link]

The rich type system also means that there are a lot of generic functions taking traits as inputs, which are not "real" types. So either you have a lot of monomorphization ("instantiation" in terms of C++ templates) where you substitute real types for the parameters, or you have dynamic dispatch, which is, well, akin to uncoditional surrender. Forcing dynamic linking also forces the latter, and this has huge impact on performance, maybe as much as impeded inlining has.

(Disclaimer: I speak from theory, not practice, so I may be more than a little wrong :) )

Filesystem-oriented flags: sad, messy and not going away

Posted Mar 17, 2020 21:45 UTC (Tue) by areilly (subscriber, #87829) [Link]

In these days of flatpack and image-based application distribution, where applications ship with private versions of all of the shared libraries that they use, it's easy to argue that the days of shared libraries being particularly useful, at least for supposed benefits of disk space or memory space savings, are long gone. From a language point of view, the model of separately compiled object files is too restrictive, and too much of a barrier to efficient abstraction. Most modern languages have a whole-program compilation model, and that includes C++, except for the cases where modules effectively isolate themselves behind a C API. (Go, rust, julia, haskell, all of the lisps...)
I view this trend as a good thing, btw. The modern languages have a lot going for them, and shared libraries really don't.