Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
Posted Jan 29, 2025 15:04 UTC (Wed) by bluca (subscriber, #118303)In reply to: Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ... by farnz
Parent article: Vendoring Go packages by default in Fedora
That's most definitely not true - some libraries are bad, but most are good on that front, and ABI transitions are relatively easy, with few exceptions especially compared to the grand total of number of libraries shipped.
And of course - in case there's a library that is badly maintained and doesn't bother with keeping ABI stable, that is simply a bad library, and not a candidate for being included in the first place
> But the original reason for dynamic linking is simply that consumers cannot recompile proprietary binaries, and want to receive binaries they can run without a "prepare" step. Good tooling can already rebuild all dependent packages on a version change (e.g. Open Build Service does this), and the other reason that people cared about dynamic linking back in the day is that downloading updates over dial-up was already painfully slow with statically linked packages.
That might surely have been the issue originally, but it's not today. Constantly rebuilding everything all the time costs a lot of compute power (hello, climate change anyone?), and while we in the western countries are blessed with symmetric fiber connections with no data caps, that is most definitely not the case everywhere in the world, and in many many countries metered 2g/3g/4g cellurar connections are the only option. And on top of that, disk space still matters in some cases: when building an initrd, you want that as small as possible even on a modern powerful laptop. And you want to save memory and latency by loading the DSO _once_ instead of every time for every program, which matters a lot of low-end/embedded devices.
Posted Jan 29, 2025 15:35 UTC (Wed)
by farnz (subscriber, #17727)
[Link] (14 responses)
On top of that, you only have to follow a distribution development list to see both announced and unannounced soname bumps, where a package has deliberately broken its ABI, along with bug reports because a package has silently broken its ABI. Distros have even set up language-specific tooling for C to generate ABI change reports, so that they can catch broken ABIs before users notice.
All of this is a lot of work, and it's largely based on tooling distros have built for C. It's not hugely surprising that other languages don't have the tooling yet - it took literal decades to get it built for C, and integrated so transparently into distros that it's now possible for package maintainers to not even be aware that it exists, right up until the package they maintain does a bad thing and the tooling reports bugs to them.
Posted Jan 29, 2025 18:37 UTC (Wed)
by bluca (subscriber, #118303)
[Link] (13 responses)
The most common C libraries that are used in any base system are very stable. glibc last broke compat what, 20 years ago? A very commonly used one with the most recent change was OpenSSL with version 3.0.
It does happen of course, and there are processes for it, well-regulated and understood. And slow.
Posted Jan 30, 2025 14:41 UTC (Thu)
by taladar (subscriber, #68407)
[Link] (10 responses)
That isn't really a solution, it just means people avoid change because it has proven to be so painful.
Posted Jan 30, 2025 14:46 UTC (Thu)
by josh (subscriber, #17465)
[Link]
Posted Jan 30, 2025 15:05 UTC (Thu)
by bluca (subscriber, #118303)
[Link] (8 responses)
Posted Jan 30, 2025 15:34 UTC (Thu)
by Wol (subscriber, #4433)
[Link] (1 responses)
Because they don't wear the same rose-tinted glasses as you? Because in their world view this particular problem just doesn't exist? (After all, isn't that one of the selling points of Rust - whole classes of major problems in C just disappear because the concept doesn't make sense in the Rust world.)
It's like a bird trying to explain to a fish how you spread your wings to soar in the wind - the fish (one or two excepted) will have no idea what you're talking about.
Cheers,
Posted Jan 30, 2025 16:12 UTC (Thu)
by bluca (subscriber, #118303)
[Link]
Posted Jan 30, 2025 17:34 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link]
Posted Jan 30, 2025 17:59 UTC (Thu)
by mb (subscriber, #50428)
[Link] (4 responses)
Just because Rust chose different solutions than you prefer for the problem of backward compatibility, doesn't mean that they don't care.
If you think backwards compatibility in the C ecosystem is so far superior, then where are C Editions? Where is the common practice in the C community to use Semantic Versioning?
Except from very few examples like glibc there are few projects in the C community that really care about backward compatibility to the extent possible with Rust Editions and Semantic Versioning. Most projects are stable because they are not changed much on the interface level anymore. glibc being a notable exception.
Posted Jan 30, 2025 18:36 UTC (Thu)
by bluca (subscriber, #118303)
[Link] (3 responses)
This is absolute nonsense. Again, there is data on this, just look at it. Compare symbols exports vs SONAME revision of common system libraries. libsystemd0 alone added 213 new APIs just in the very latest release. SONAME revision is still 0.
Posted Jan 30, 2025 18:48 UTC (Thu)
by mb (subscriber, #50428)
[Link] (2 responses)
Saying that Rust people don't maintain backward compatibility is:
>This is absolute nonsense
They do it in a very different way.
There are more solutions to a problem than the Luca-way, you know?
I do understand that you either don't like or don't know what Rust people do.
If you want Rust to have a stable soname/ABI, please come up with a proposal about *how* to do that.
Posted Jan 30, 2025 19:41 UTC (Thu)
by bluca (subscriber, #118303)
[Link] (1 responses)
Because it's true, whether or not you like it. The standard library changes soname on every build. Backward incompatibility is programmatically baked in the build process itself.
Posted Jan 30, 2025 19:44 UTC (Thu)
by corbet (editor, #1)
[Link]
Posted Feb 7, 2025 18:17 UTC (Fri)
by TheBicPen (guest, #169067)
[Link] (1 responses)
Posted Feb 7, 2025 18:34 UTC (Fri)
by bluca (subscriber, #118303)
[Link]
And yet, despite all the costs you claim the distros are trying to avoid, they still do regular complete archive rebuilds to find FTBFS bugs caused by a dependency change (thus spending the compute time anyway), the size of a minimal install has skyrocketed (you used to be able to install Debian from 7 floppies, or 10 MiB, the "minimal" install image they offer is now 650 MiB or 65x larger), and the use of ASLR means that the DSO is no longer loaded "just once", but multiple times, because distros don't (based on looking at a Debian 12 system) carefully arrange their DSOs to minimise the number of pages with relocations in them.
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
libacl, libpam, various util-linux libs, compression libs, selinux libs - those are all at soname revision 0, 1 or at most 2, and they are decades old.
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
Wol
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
>but apparently this looks like an alien concept in the golang/rustlang ecosystems.
Just because somebody else does something different from what you think would be the correct solution doesn't mean they are doing it wrong. It's quite possible that you are wrong. Not everybody except you is an idiot. Do you get that?
These things work _really_ well in the Rust ecosystem. And they are either completely absent (Editions) or used extremely rarely (SV) in C.
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
Just because rustlang/golang library maintainers can't or won't maintain backward compatibility, it doesn't mean nobody else will.
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
Like it or not.
But I'm just asking you politely to stop spreading this "they don't have/care about backwards compatibility" FUD.
I have ignored you for over half a year here in LWN comments and you are still busy spreading that nonsense. Why? This annoys me just as much as anti-systemd-FUD annoys you.
Be constructive rather than destructive.
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
So are we sure that this conversation is going anywhere but in circles? Might it be time to conclude that there isn't more to be said (again)?
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
Don't just vendor - rebuild the ecosystem and persuade the vendor to work on software management ...
