Shared libraries [LWN.net]

Shared libraries

Posted Nov 24, 2025 22:25 UTC (Mon) by Cyberax (✭ supporter ✭, #52523)
In reply to: Shared libraries by carlosrodfern
Parent article: APT Rust requirement raises questions

Why would this be complicated? You just recompile stuff and ship it. It might force you to invest in more efficient packaging system, if anything. There's no need to transmit 150Mb of data if you need to change 20 bytes of code.

Android uses this for the OTA system updates.

to post comments

Shared libraries

Posted Nov 24, 2025 23:39 UTC (Mon) by carlosrodfern (subscriber, #166486) [Link] (3 responses)

I'm actually pointing out the *problems* of doing all the things statically linked, not advocating for it.

The fact that statically linked programs are a good solution in containers doesn't mean that it can be extrapolated to an Linux distro. A slightly change in the nature of a problem, or in the size of the problem, can justifies a very different solution. It is a typical mistake that people make as they get excited about one technology or approach and want to apply it to all the things that like like a nail. Statically linked programs written in golang or Rust for containers make a lot of sense since the pros are weighty and the cons are not that significant in the context of that use case, but it is not a good approach for all the programs in Linux distros.

Shared libraries

Posted Nov 25, 2025 0:49 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

> I'm actually pointing out the *problems* of doing all the things statically linked, not advocating for it.

But it's not really a problem, is it? Binary diffs for patch update can negate the advantages of shared libraries.

> The fact that statically linked programs are a good solution in containers doesn't mean that it can be extrapolated to an Linux distro.

But maybe it can? I actually tried a fully static distro a while ago ( https://github.com/oasislinux/oasis ), and it objectively felt _better_ than regular Debian.

I'm not at all convinced that shared libraries are worth all the hassle.

Shared libraries

Posted Nov 25, 2025 20:27 UTC (Tue) by Whyte (subscriber, #161914) [Link] (1 responses)

Did your test also include KDE or GNOME?

That's the part I don't ever see happening, since the libraries supporting those are huge.

Shared libraries

Posted Nov 26, 2025 2:38 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

No, it used its own display manager. That was pretty snappy but limited.

I think that these days, graphics library problem can be solved by moving the UI into Electron-like shells, with a shared browser layer. And custom native code can be called via IPC. This in theory can provide even more optimizations, like pre-forked initialized shell that can be forked up without waiting for it to load (like the 'zygote' process in Android).

BTW, the browser doesn't have to be Chromium. Servo would be great for this purpose.

Shared libraries

Posted Nov 25, 2025 7:54 UTC (Tue) by joib (subscriber, #8541) [Link]

I think Ubuntu optionally supports using zsync for distribution changes efficiently. Also some other non-distro packaged software I use, use it for updating themselves. https://zsync.moria.org.uk/ I'm sure there are other things that do something similar as well.

So the tech to do this efficiently already exists in open source, it just needs to be integrated more deeply into distro package distribution tooling.

Shared libraries

Posted Nov 25, 2025 21:02 UTC (Tue) by jhoblitt (subscriber, #77733) [Link] (4 responses)

I'd also argue that 150MiB or even 1.5GiB package updates would not be a deal breaker. Those sizes are equivalent to streaming only a few minutes of 4K video.

Shared libraries

Posted Nov 26, 2025 5:46 UTC (Wed) by Heretic_Blacksheep (subscriber, #169992) [Link] (3 responses)

Cumulative. Imagine having to replace your entire distro 10-15 GB installation every couple of weeks. Then imagine multiplying that by hundreds of thousands and see how fast most distribution infrastructure volunteers throw in their cards. We're not talking about streaming a movie from Netflix, a billion dollar company with distributed media cache servers on nearly every ISP internal WAN. We're talking about Debian, which is volunteer run with volunteered mirror resources. Backbone interconnect data service is a metered service for all ISPs. You want to waste your bandwidth, fine, but don't be so selfish as to volunteer others that are graciously sharing their resources - or don't have resources to waste. Not every end user is so blessed as to having effectively unlimited 200+ Mbit service. Outside of the developed world, and even in some areas of the developed world, that's pretty rare. Even in the US, most residential ISP services have a soft 1 TB/month data cap regardless of bandwidth which has to be shared across all family members. The low cost subsidized plans are usually capped at 25 Mb/sec with much lower data caps - the very people that Linux distributions would benefit the most using recycled hardware because they can't afford fancy gear!

Also people forget the flash drives the vast majority of people use these days for storage have a finite lifetime and that lifetime is steadily getting lower as the number of bits per cell inflates. A typical drive rated for 300 TB write/erase cycles (Samsung 970 EVO rated lifetime) won't last nearly as long if everything were staticly built as it would with a distro with dynamic linking that only had to write a couple of gigabytes of updates a month. I've seen plenty of drives with even lower lifetime ratings on the consumer/enthusiast market. There's no telling the rating on rebranded OEM units (Lenovo, Dell, etc). Dynamic linking leaves a lot more write cycles for user data and browser-like-programs disk cache/system logging/file indexing (the bigger problems with SSD cell burn than rarely changed media files and such).

Shared libraries

Posted Nov 26, 2025 7:23 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

> Cumulative. Imagine having to replace your entire distro 10-15 GB installation every couple of weeks.

This is a simply ridiculously overblown comparison. The total size of non-kernel-related binaries on my desktop Linux computer is 1Gb. And they don't need to _all_ change every time.

Assuming that ALL of them need to be replaced, that's already an order of magnitude less. But let's roll with it. An unmetered 10GB port is around $200 a month, and an M2 SSD can easily saturate it. If you want to serve static files, that's probably around $300 a month expense.

A 10GB port can serve the 1Gb in about 1 second. So 86400 people daily, and over a month that's 2.5 million people. This is more than the number of users for most small distros.

Shared libraries

Posted Nov 26, 2025 7:33 UTC (Wed) by mb (subscriber, #50428) [Link] (1 responses)

>10-15 GB installation every couple of weeks
>rated for 300 TB write/erase cycles

That's over 400 years of steady updates.

I've yet to see my first SSD failure due to the "small" lifetimes that one can read about everywhere all the time.

I use them since about 15 years ago as system disk and as disks that are continuously being written to since 8 years ago. Not a single failure so far, except for one that was bad out of the box.

I don't say that failures can't happen, but so far SSDs are much *more* reliable than spinning rust for me.

Shared libraries

Posted Dec 4, 2025 17:00 UTC (Thu) by nye (guest, #51576) [Link]

Only actually ~500 full drive writes, but I've been impressed by Sandforce. Shame they went out of business. I've never seen an HDD last nearly this long.

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Retired_Block_Count     0x0033   093   093   003    Pre-fail  Always       -       608
  9 Power_On_Hours_and_Msec 0x0032   099   099   000    Old_age   Always       -       132110h+35m+45.600s
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       175
174 Unexpect_Power_Loss_Ct  0x0030   000   000   000    Old_age   Offline      -       81
231 SSD_Life_Left           0x0013   089   089   010    Pre-fail  Always       -       1
241 Lifetime_Writes_GiB     0x0032   000   000   000    Old_age   Always       -       32128
242 Lifetime_Reads_GiB      0x0032   000   000   000    Old_age   Always       -       79296

Shared libraries

Posted Dec 5, 2025 12:06 UTC (Fri) by ras (subscriber, #33059) [Link] (6 responses)

> Why would this be complicated? You just recompile stuff and ship it.

It's complicated because the upstream stuff isn't the same.

Debian Stable has one feature sysadmin of smaller places love: it's stable. If there is some security issue Debian doesn't ship the new upstream with it's new features and depreciations, they ship the same version with a security fix backported. That effectively means you can let unattended upgrades do it's thing in the background for the four year lifetime a Debian of a typical release without too many concerns. (You will get the occasional email about something that might require manual intervention, like a reboot for a kernel upgrade, but it's rare.)

Big shops probably don't see much value in this, as they have teams of programmers releasing new static binaries or docker images every few weeks. But for small shops it's live saver. For example it makes home mail/web/ssh/router server as hobby feasible, because I only have to touch it once every 4 years. If you work in place that has less than 10 people handling IT functions, that way of operating is the norm. I've don't know much about LWN for instance, but I'd bet that's how they manage their infrastructure. It's either that, or god help you, you use micro services and outsource it all to a very well paid provider.

To pull that off Debian uses a small army of security volunteers watching incoming CVE's, and doing the backporting work. I'd kinda amazed it works at all, but it does better than that. In my experience Microsoft's security patches break far more servers than Debian does.

One of the tricks Debian uses to make it is Debian insisting there is only one version of every library in use. Imagine what it would be like if you have 20 versions of glibc, gtk, libm and every other library. The work load of the backporters would explode. Yet when I look in the Cargo.lock of your typical largish Rust program that is exactly what I see. Worse, it's not just a case of different program uses different versions of the same library, it's often different versions of the same library used by one program! Kudo's for Rust for making that work I guess - but that wonderful experience of having "cargo build" just work most of the time looks to me to have laid the foundations for creating a security nightmare.

All that said, I'm not a huge fan of the current package model myself. I'd much prefer an immutable core, rather like MacOS does (but much smaller), and default to complete isolation of the apps running on top of it like Android does. It would be far more secure by default. But it's decades of work away, and in the mean time Rust's monomorphization model doesn't seem to be a good fit for how distro's work today.

Shared libraries

Posted Dec 5, 2025 12:48 UTC (Fri) by Wol (subscriber, #4433) [Link] (1 responses)

> One of the tricks Debian uses to make it is Debian insisting there is only one version of every library in use. Imagine what it would be like if you have 20 versions of glibc, gtk, libm and every other library. The work load of the backporters would explode. Yet when I look in the Cargo.lock of your typical largish Rust program that is exactly what I see. Worse, it's not just a case of different program uses different versions of the same library, it's often different versions of the same library used by one program! Kudo's for Rust for making that work I guess - but that wonderful experience of having "cargo build" just work most of the time looks to me to have laid the foundations for creating a security nightmare.

Can/does Cargo complain when told to use several different versions of the same library?

Given the general "If it compiles it will work" nature of Rust, a warning/error trace telling you "this library is loaded using several different versions pulled in in all these places", surely it would just take a bit of discipline to delete all the older references, a quick QA, and your "typical largish Rust program" would suddenly be rather smaller?

Cheers,
Wol

Shared libraries

Posted Dec 5, 2025 14:04 UTC (Fri) by farnz (subscriber, #17727) [Link]

Cargo itself will let you use multiple different versions of the same library in one program, without errors.

You can, however, use cargo-deny, which will tell you about multiple versions in use.

The issue, however, remains developer time. We all agree that using undermaintained versions of code is bad, but nobody is putting in the time to make sure that we're all on maintained versions. And the only difference in that regard between the C ecosystem and the Rust ecosystem is that in the Rust ecosystem, the use of large numbers of small dependencies means that it's easy to spot, where in the C ecosystem, spotting which parts of a big library (like glib) are undermaintained, and which ones are cared for is hidden behind the fact that glib overall is cared for.

Shared libraries

Posted Dec 5, 2025 13:08 UTC (Fri) by Wol (subscriber, #4433) [Link] (1 responses)

> All that said, I'm not a huge fan of the current package model myself. I'd much prefer an immutable core, rather like MacOS does (but much smaller), and default to complete isolation of the apps running on top of it like Android does. It would be far more secure by default. But it's decades of work away, and in the mean time Rust's monomorphization model doesn't seem to be a good fit for how distro's work today.

Unfortunately, I don't think that's a good fit with how linux development actually happens. Because everything happens independently, it's hard to do big releases. But yes, I agree with you.

The base OS - linux the kernel, systemd, basic utilities (similar in concept to MS-Dos).
The daemon set - sendmail and friends, MySQL and friends, etc etc
The windowing system - KDE/Plasma, Gnome, whatever
User applications.

And if each part were as self-contained as possible it would make life a whole lot easier. But it would require a "distro BDFL", and I don't think that's going to happen! It would probably take someone like Dell, or Red Hat, and the network effects are against it.

Cheers,
Wol

Shared libraries

Posted Dec 5, 2025 13:42 UTC (Fri) by pizza (subscriber, #46) [Link]

> Unfortunately, I don't think that's a good fit with how linux development actually happens. Because everything happens independently,

This point cannot be emphasized enough.

Shared libraries

Posted Dec 5, 2025 14:11 UTC (Fri) by daroc (editor, #160859) [Link]

As our HTTP headers state, LWN is currently hosted on CentOS Stream. But, as it happens, we've been looking into whether we need to change the site's hosting infrastructure due to some recent outages at our hosting provider. All the important configuration is theoretically kept in an ansible playbook that should allow us to change underlying systems without too much fuss — and if we do, Debian would certainly be a fine choice. But I believe the site has been on CentOS for the last several decades, and keeping to what is known to work also makes sense.

Shared libraries

Posted Dec 6, 2025 17:14 UTC (Sat) by ggreen199 (subscriber, #53396) [Link]

> All that said, I'm not a huge fan of the current package model myself. I'd much prefer an immutable core, rather like > MacOS does (but much smaller), and default to complete isolation of the apps running on top of it like Android does. > It would be far more secure by default. But it's decades of work away, and in the mean time Rust's
> monomorphization model doesn't seem to be a good fit for how distro's work today.

There are the new immutable distributions that work like this. For example, Kinoite, in Fedora, which is what I use now. It has an immutable core, and you use flatpaks or containers for everything else. Still early days, but I've been using it all year with no problems.