LWN: Comments on "Packaging Rust for Fedora"

Packaging Rust for Fedora

linuxrocks123 — Thu, 14 Sep 2023 02:12:15 +0000

Sorry for the late reply; I don't get email notifications. I believe from my browser history that this is the one I found advertised as working with that model: https://github.com/ErnyTech/nspire-tools/

Packaging Rust for Fedora

ghane — Sat, 03 Dec 2022 06:38:30 +0000

> “Standard business programmer” just does motions which were designed to achieve things he doesn't know about and then tries to fix issue which not need fixing while simultaneously missing things that do need fixing.

This. Precisely this.

Developers keep opening issues that make no business impact, and keep trying to solve every issue with a combination of StackOverflow and copy/paste.

This week, I have been trying to explain to a team why they should remove an "UpdatedAt" database field from a table that is an immutable append-only log. "But every other table has a CreatedAt and UpdatedAt field!"

Meanwhile, users see "Oops an Error happened" popups. "OK" button.

Shared Libraries Reduce RAM Usage

XERC — Fri, 11 Nov 2022 19:09:36 +0000

I just wanted to point out that one of the main benefits of shared libraries is that the same library instance in RAM can be used by multiple processes. May be I'm mistaken here, but that's my current, 2022_11, understanding. From speed point of view, the less RAM is needed, the greater the probability that what the CPU needs, is already at some CPU cache. With modern CPU caches that are multiple MiB in size, that "something" might be a whole small shared library or some part of it. Form speed point of view there's quite a bottle neck between CPU and RAM.

Thank You for reading my comment.

Packaging Rust for Fedora

jezuch — Fri, 11 Nov 2022 12:16:45 +0000

> Rebuilding all of them and sending the result to the mirrors ain't going to happen quickly.

And yet, I'm pretty sure Debian does it regularly and pretty routinely. Though maybe not with glibc (it's a small nightmare of its own).

Packaging Rust for Fedora

Cyberax — Wed, 09 Nov 2022 21:20:54 +0000

> It would probably give us huge saving on time and effort too.

It unfortunately doesn't scale. People have to make sure that multiple applications actually do work with the single version of the library. We tried that, and it failed. Never mind that this approach still doesn't cover everything. E.g. templates in C++, macros and header changes in C, etc.

Thus simply treating updates as blobs, and computing diffs automatically will help to free humans to do actually useful work.

Packaging Rust for Fedora

ras — Wed, 09 Nov 2022 21:13:45 +0000

> This is mostly because the current package update infrastructure is horribly wasteful. It forces users to re-download the whole package if only a handful of bytes are changed.

Here is an idea. Why don't we just split the common parts of every program binary into independent libraries. Pre-do the delta in other words. It wouldn't just save on distribution bandwidth, it would save on storage space too, and hell we could even share the RAM when it is executed. It would probably be more efficient bandwidth wise than doing the delta later, because while we have to update all of libc6 - we only have to do it once, not 26,000 times.

It would probably give us huge saving on time and effort too. Imagine - lets say the library foo has a CVE. Instead of finding every program that uses library "foo" and backporting the CVE fix to each and every copy of the "foo" source those programs included (which would probably be a different version because programmers seem to just love their .lock files, pinning the version), we would have one instance of "foo" (because it is a now a shared library) and fix just that. To take the libc6 example, instead of fixing 26,000 instances of libc6 source we just fix it once! It would take a bit of upfront effort to ensure every program worked with this one version of "foo", but I bet that effort would be more than repaid by the time saved maintaining it later.

In fact the productivity gain would be so big, it would make things like Linux distributions possible. Without it I doubt they would be workable.

Packaging Rust for Fedora

Cyberax — Wed, 09 Nov 2022 16:12:49 +0000

There are two issues in play:

1. It takes a lot of time to rebuild packages. This can't be helped, but computing is pretty cheap these days.

2. It takes a lot of bandwidth to distribute the updates. This is mostly because the current package update infrastructure is horribly wasteful. It forces users to re-download the whole package if only a handful of bytes are changed. That's why OSTree with delta-encoding would also help a lot for system packages. User-level apps should be sandboxed and can also use delta-updates.

Packaging Rust for Fedora

ras — Wed, 09 Nov 2022 09:52:24 +0000

> they can just rebuild static binaries anyway.

I'm not sure they could. Debian has some 23,000 packages depending on libc6. Rebuilding all of them and sending the result to the mirrors ain't going to happen quickly.

Packaging Rust for Fedora

mathstuf — Fri, 04 Nov 2022 13:31:36 +0000

> if your code then you are half-step from trying to use types defined in one version for the other version.

There are ways around this. For example, log 0.3.9 depends on 0.4 to serve as a bridge between the two nominally semver-incompat versions and not force an ecosystem-wide upgrade.

https://crates.io/crates/log/0.3.9/dependencies

Packaging Rust for Fedora

mb — Thu, 03 Nov 2022 21:11:48 +0000

> but I just don't think it's what Rust needs.

It's a feature that types and traits from incompatible versions of the same crate are in fact different types and traits.
That enables you to implement multiple versions of a trait on one single struct simultaneously.
That is sometimes useful, if you want to implement a library crate that must provide multiple different versions of an API (a foreign trait) to its users.

Packaging Rust for Fedora

khim — Thu, 03 Nov 2022 19:50:37 +0000

Touché. This is minor improvement, though: if you are already at the point where you have different versions of the same component as immediate dependency if your code then you are half-step from trying to use types defined in one version for the other version.

This doesn't work in Rust and I don't think it's worth introducing meta-object protocol to allow that (although some languages [actually do that](https://www.gnu.org/software/guile/manual/html_node/The-Metaobject-Protocol.html)) but I just don't think it's what Rust needs.

Packaging Rust for Fedora

mb — Thu, 03 Nov 2022 19:22:30 +0000

> Except when you need to use two versions in one component, but that's next level of crazyness.

You can also use two different versions of the same dep-crate in one crate simultaneously, as long as you map at least one of the deps to a different name (under which it is then accessible in the Rust code). That's easily possible with Cargo.toml:

[dependencies]
thedep02 = { version = "0.2", package = "thedep" }
thedep = { version = "1.0" }

shared libraries [ no longer specific to either Rust or Fedora ]

nim-nim — Thu, 03 Nov 2022 09:16:16 +0000

The full move to Harfbuzz is required to properly support modern opentype (TTF and OTF) features, be it latin legatures, non latin scripts, emojis, color, etc. Harfbuzz has two decades of enhancements, fixes and rewriting over the old freetype code it was originally forked from.

High resolution (aka banal 4K screens) is not directly involved, except inasmuch the very old pre-opentype fonts that render worse with the new system, mostly because they were designed around the old system bugs, are useless on such screens (way too small). And also, they look like crap compared to banal vector fonts rendered on such screens: there is a point at which no amount of pixel hand-tuning can compete with more pixels to render fine details. It’s no longer distorted sharp vs blurry fidelity, it’s sharp fidelity vs blocky pixelated that relies on old bugs.

Packaging Rust for Fedora

firstyear — Thu, 03 Nov 2022 02:38:41 +0000

OpenSUSE does not allow network access during builds but we do allow vendoring in rust packages. We also have supporting tooling to work with our product security to scan for security issues, update them, and respond.

We have a flourishing rust ecosystem with many new package maintainers, and I even presented at Rust Conf about the approach and how successful it has been.

The primary driver to go "vendoring only" was that there will always be "an exception" where vendoring is the only choice and you can't use distro packages. So why split the problem - just choose a single correct work flow, and make sure it's robust.

(Note - I am the maintainer of Rust in SUSE and OpenSUSE).

Packaging Rust for Fedora

bluca — Wed, 02 Nov 2022 19:34:26 +0000

Language developers: "We don't need no stinking distributions! Curation is useless! Our language-specific stores are the best!"

Language-specific stores: https://lwn.net/Articles/913555/

Packaging Rust for Fedora

Cyberax — Wed, 02 Nov 2022 18:41:13 +0000

If you're working in an environment where you're legally liable for the quality, then you'll have to store all the previous versions and lock dependencies even more. You sure as hell won't just apply random updates from distros without testing them.

Packaging Rust for Fedora

Wol — Wed, 02 Nov 2022 17:22:12 +0000

That pretty much sums up why I run gentoo ...

I was running the latest most up-to-date SUSE, I needed the latest, most up-to-date lilypond, and never the twain shall meet ...

Cheers,
Wol

shared libraries [ no longer specific to either Rust or Fedora ]

farnz — Wed, 02 Nov 2022 17:15:51 +0000

Indeed - and the ability to pin an older version than the distro supports is not a real fix for that very reason. Without someone stepping up to maintain a fork of pango-1.43 suited for use on systems that don't gain from the changes in pango-1.44, you're trading off a known bug (the rendering issues in pango-1.44 on "normal" screens) for future problems (unknown bugs in pango-1.43 fixed in a later pango-1.x release including security-relevant bugs, ABI changes in later pango-1.x that increase the blast radius of trying to pin 1.43 etc).

Ultimately, while this is nice for debugging whether or not it's a pango change that's breaking things, it's a dangerous route for fixing things - it results in an unsupportable system because you can't rely on users having the versions of libraries you expect them to have (because they're "randomly" pinning versions that happen to not have bugs that annoy them, and then complaining because there are bugs they don't like).

shared libraries [ no longer specific to either Rust or Fedora ]

nim-nim — Wed, 02 Nov 2022 16:40:22 +0000

> And in terms of distros requiring a single version - the default in all distros I've looked at is to say that you can't install both pango-1.43 and pango-1.44 at the same time

You *could* if pango-1.43 and pango-1.44 where both packaged and maintained (and for a lot of components this is technically possible, just rename the pango-1.43 to oldpango-1.43 to allow two version streams in parallel).

However *no* *one* wants to expand the effort to package multiple versions of the same thing for you. The amount of work snowballs quickly.

Packaging Rust for Fedora

seanyoung — Wed, 02 Nov 2022 16:39:15 +0000

> > Why that rule of not downloading anything during build?
>
> Distribution build systems do download things before build but from the pile of things already packaged and audited not a blind internet pull. The magmas do exactly the same their stuff is not pulling from the internet but from internal audited repositories (giant curated monorepos).

So my proposal was to use cargo --locked with a distribution-owned copy of crates.io. That means exactly the opposite of a "blind internet pull". I means neatly a curated listed of packages that distribution can audit and lock down precisely.

> The reason the stupid gratuitous inflation of artifact numbers and versions works for the magmas but not for distributions is that the magmas have more (paid) people to hurl at checking their repositories than distributions.

Not at all. Modern tooling like rust is finally getting the reusability of software components right. Many rust crates do one thing, and do one thing very well. Some crates are very sophisticated and some serve a simple purpose; either way, you don't want to open-code all these components yourself. Do you want to open-code how to get the terminal size, for all the unixes and Windows etc or use a simple crate? I do, I don't want to waste time getting some icky ioctl right on some little-used OS, and even if I did, my code will almost certainly have bugs in it because it is not as widely tested.

The tooling provided by distributions is no good for software development. I want to be able to select versions of libraries, not just the one provided by the distribution, possibly point it towards a git repo where I have some of my own patches. I want automatic (recursive) dependency downloading, based on semver. I want to be able to enable features on libraries, not just the features the distribution provides (i.e. ./configure --enable-foo etc). All of this simply by having a simple config file which lists by dependencies, which I can update at any time without changing my distribution. This is what rust/cargo provides.

In my 30 years of software development it has been a constant that the distribution does not provide the version/fix/etc I need, so I have to work around it. The distribution package tooling is utterly useless for software development so very few software developers use it.

I get that the task for distributions is so much harder because of it. How about working upstream with cargo/rust? It's a very warm and welcoming community. Surely there something we can fix together. The result would be amazing.

Packaging Rust for Fedora

nim-nim — Wed, 02 Nov 2022 16:29:05 +0000

The problem is not the mirroring, it's the checking (and maintaining).

If you freeze down a dependency for 15 years, and have any form of liability, you need to check every single fix released during those 15 years upstream to see if it applies to your configuration.

To a layman (judge) not applying a “free, public” fix will make you guilty as hell in case of a problem.

Packaging Rust for Fedora

nim-nim — Wed, 02 Nov 2022 15:40:54 +0000

Also, finally, not a problem to develop with whatever artifact version you want as long as you are able to clamp down at end-release time and rebase on the version the release team blessed (be it the one Google chose to release its latest kubernetes version with or the one Debian of Fedora chose to base their distro on).

It’s this clamping down devs object to, even though it happens the same way distro and magma-side, because it is materially impossible to check every possible artifact version under the sun, and even the magmas are not rich enough yet to attempt it.

Packaging Rust for Fedora

nim-nim — Wed, 02 Nov 2022 15:11:03 +0000

> There are no selection process which may decide whether fork of some product is original enough to be included.

There is no such selection process distro side either, the sole selection process is finding enough people motivated to build an n-th version of the same thing, and the only way to fail this process is to make building the n-th version too hard compared to the benefits it provided over the (n-1) other versions.

Which, is really easy to do when you are pulling hundreds of other modules which are all needing review individually.

Packaging Rust for Fedora

nim-nim — Wed, 02 Nov 2022 15:04:42 +0000

> Why that rule of not downloading anything during build?

Distribution build systems do download things before build but from the pile of things already packaged and audited not a blind internet pull. The magmas do exactly the same their stuff is not pulling from the internet but from internal audited repositories (giant curated monorepos).

The reason the stupid gratuitous inflation of artifact numbers and versions works for the magmas but not for distributions is that the magmas have more (paid) people to hurl at checking their repositories than distributions. And, more managers to hurl at devs that try to use non-blessed artifacts. And, more lawyers to hurl at people seeking damages if stuff fails. And, finally, a software world where they are the only ones able to provide trusted binaries suits them just fine.

The reason the stupid gratuitous inflation of artifact numbers and versions works for Joe nobody developer is that his liability if his stuff does not work is limited, so he does not feel the need to check the things his build systems pulls from everywhere, and he does not understand nor supports or helps the idiots that try to pull out this Sisyphean task distro-side.

shared libraries [ no longer specific to either Rust or Fedora ]

anselm — Wed, 02 Nov 2022 12:54:49 +0000

And in terms of distros requiring a single version - the default in all distros I've looked at is to say that you can't install both pango-1.43 and pango-1.44 at the same time.

Most distros, however, will let you install pango-1.x and pango-2.x at the same time. The general idea behind this is that pango-1.44 should support everything that pango-1.43 supported, plus perhaps some extra stuff, so that it doesn't matter if you swap pango-1.43 out for pango-1.44 because applications that have so far worked with pango-1.43 will just ignore the new extra stuff, and upgrading pango for applications that do require the new extra stuff in pango-1.44 shouldn't be a problem as far as other pango-using applications are concerned.

If stuff from pango-1.x actually goes away then the new pango should be 2.0, and applications that depend on pango-1.x should just be able to keep using the older version because the two versions can be installed side-by-side. How well the authors of individual software packages adhere to this philosophy is, of course, anyone's guess (and of course there's the edge case of an application relying on a bug in pango-1.43 and breaking when the bug is fixed in pango-1.44, which arguably is not the pango package's fault).

shared libraries [ no longer specific to either Rust or Fedora ]

Wol — Wed, 02 Nov 2022 12:36:57 +0000

> And in terms of distros requiring a single version - the default in all distros I've looked at is to say that you can't install both pango-1.43 and pango-1.44 at the same time. You have to choose one or the other; eventually, if there's a pango-1.46 that adds new ABI that an application depends upon, users using your "fix" have to upgrade and accept the problem.

Dunno whether it's the "default" or not (and I haven't had the need to use it myself), but gentoo's slot mechanism allegedly makes running multiple versions easy. Maybe other distros should have something similar?

Cheers,
Wol

shared libraries [ no longer specific to either Rust or Fedora ]

farnz — Wed, 02 Nov 2022 09:28:36 +0000

Ah. In the case you're describing, you're building an unsupported configuration - it's a very short term workaround, since sooner or later, the packages users depend upon will require the newer pango's ABI and not work with the older pango's ABI. You're trading bugs - you've got users to choose an unsupported configuration (since the distro maintainers will tell you to update to latest releases first) that works for now, in preference to a supported configuration where bugs can be fixed.

Unless the distro explicitly supports this action, you're creating more trouble down the road - the user will at some point have to choose between a pango that works but doesn't display nicely on their display, or not running a current version of their application. In a statically linked world, they'd have to choose older application version that displays nicely, or newer one that doesn't - and that's a tradeoff the users understand - complete with "I won't upgrade from Debian 13 to Debian 14 because it breaks my Scribus" attitudes. What you've got users to build is a system where things are unreliable (because sometimes they will run things that accidentally or deliberately depend on the ABI of newer pangos), and where they cannot see their way clear to a fix (because ABIs aren't something the typical user understands - they just have two bits of information, "program X crashes" and "I can't upgrade pango").

The long term solution is to work with upstream - either deliberately fork pango, and have two alternatives you can install as a deliberate act, one with the old engine, one with the new, or get the new engine to the point where it's as good as the old on low-DPI displays - because anything else gives users the illusion that they're running a supported system, while actually having a system that'll break at a moment's notice.

And in terms of distros requiring a single version - the default in all distros I've looked at is to say that you can't install both pango-1.43 and pango-1.44 at the same time. You have to choose one or the other; eventually, if there's a pango-1.46 that adds new ABI that an application depends upon, users using your "fix" have to upgrade and accept the problem. Further, most distros explicitly don't support you running a mix of older and newer packages; if you want distro support for anything, you need to run the distro update mechanism and run the latest versions of all packages. In the case of users who've pinned an old version of pango, they're out of luck if they want distro support, because the distro attitude will be "well, you're running outdated code. You need to upgrade pango to fix it".

Packaging Rust for Fedora

pabs — Wed, 02 Nov 2022 05:00:29 +0000

Crates don't necessarily contain source code, for eg the rand crate contains files generated by a Python program from another file. The Python script and the real source isn't in the crate, just in the VCS. Of course, the VCS could have the same issue, but that is a bit less likely these days.

shared libraries [ no longer specific to either Rust or Fedora ]

ssokolow — Tue, 01 Nov 2022 23:43:46 +0000

I'm not really following your comment about distros requiring a single distro-wide version dependency. Maybe some do? but I haven't encountered that. My experience has been that if you upgrade in place over a period of years, you gradually accumulate multiple versions of support libraries because local and 3rd party apps don't necessary rebuild against the latest library versions in sync with the distro release schedule. That's minorly annoying, but so what? OK, you have to be careful not be overly enthusiastic about deleting old library versions, but you have them backed up, right? And for that matter the older version is probably still downloadable from the distro if you know what you are looking for.

I think they're talking about how distro maintainers do things like maintaining their own patchsets to un-vendor libraries, even if that puts things into a configuration upstream does not approve of and refuses to support.

Packaging Rust for Fedora

ssokolow — Tue, 01 Nov 2022 23:40:05 +0000

Only crates.io admins can truly pull something from the site once it's published and they reserve that ability for things like legal compliance.

cargo yank keeps the package version available for download by any downstream package that has it in its Cargo.lock file.

Packaging Rust for Fedora

ssokolow — Tue, 01 Nov 2022 23:37:06 +0000

It also trips up people from commonwealth and former commonwealth countries when they learn that saying "I'm putting this into the public domain" isn't enough in jurisdictions like Germany.

See this legal review of the CC0 for more on that.

(TL;DR: The CC0 has a license fallback because German law doesn't have a concept of prematurely giving up your copyright, and it has a legally binding promise not to sue, because German law doesn't let you pre-emptively sign away rights and protections that will be written into the law at some future date.)

Packaging Rust for Fedora

ssokolow — Tue, 01 Nov 2022 23:24:05 +0000

They're probably lumping anything with a query builder API under "ORM", even though things like Python's SQLAlchemy make the Object-Relational Mapper layer an optional higher-level wrapper around the lower-level query builder system.

Packaging Rust for Fedora

salimma — Tue, 01 Nov 2022 18:32:00 +0000

Hopefully better automation can address some of the pain points; our company hired an intern this summer to work on a tool to address some of the issues brought up:

https://lists.fedoraproject.org/archives/list/devel@lists...

The tool:
- recursively checking out packages
- updating existing packages while carrying over manual changes
- creating compatibility packages when upgrading packages with dependents
- automate parallel COPR test builds of sets of packages in dependency order
- chain-build all packages in Koji with requested side tag
- merging and chain-building all packages across release branches
- help review a rust update in Bodhi and verify no compatibility issues are introduced

It does not automatically commit anything, of course, and right now is a
bit opinionated in favor of picking the lowest satisfactory version
possible, which is probably not what we want long term -- but it
hopefully provides a nice foundation on which to build on

shared libraries [ no longer specific to either Rust or Fedora ]

sfeam — Tue, 01 Nov 2022 17:36:38 +0000

In the case of pango, my understanding is that the breakage was deliberate, in the sense that someone made a choice between adding support for new font types on very high resolution displays was more important and keeping in place a framework that was designed for older font types and displays with a mere 95dpi (see e.g. https://github.com/harfbuzz/harfbuzz/issues/2394#issuecom...). On the one hand I have a bit of sympathy for pango having to choose between a new framework that is supportable going forward and an old framework whose support has languished. On the other hand, that means people downstream who were perfectly happy with desktop text rendering on their existing systems saw it as breakage. It's a whole lot easier to advise "replace this one library and your desktop will work again" than to say "you'll have to rebuild your whole desktop infrastructure to fix this, or stick with an old version of the whole system, or buy new monitors or change the fonts in your existing apps and documents, etc". The distros were caught in the middle, and may well not have been aware of the forced choice between backward and forward compatibility.

This particular problem is not one of "program X requires version a.1, while program Y requires version a.2"; it's more "this user needs version a.1 for both X and Y to work, while that system over there needs version a.2 for both X and Y to work".

I'm not really following your comment about distros requiring a single distro-wide version dependency. Maybe some do? but I haven't encountered that. My experience has been that if you upgrade in place over a period of years, you gradually accumulate multiple versions of support libraries because local and 3rd party apps don't necessary rebuild against the latest library versions in sync with the distro release schedule. That's minorly annoying, but so what? OK, you have to be careful not be overly enthusiastic about deleting old library versions, but you have them backed up, right? And for that matter the older version is probably still downloadable from the distro if you know what you are looking for.

Packaging Rust for Fedora

Wol — Tue, 01 Nov 2022 15:58:48 +0000

ORM?

(Oh, and 20kb queries aren't the problem. It's queries that are hand-written and huge. And by the way, the guys expected to write the queries are NOT database bods, they're end users who need the information. (And without whom, there's no point having a database!!!))

Oh - and if ORM means "Object Relational Model", then just ditch the relational and use Pick instead :-)

Cheers,
Wol

Packaging Rust for Fedora

kleptog — Tue, 01 Nov 2022 15:41:30 +0000

IMHO if you're doing anything significant in SQL, an ORM with a generative query API is a must. Then you have a single source of truth for your database structures. The environment can validate the queries at compile time. Migrations can be autogenerated. And you can write your queries in a way that makes sense. If this results in 20KB queries being sent to the SQL server, that's Not My Problem.

Packaging Rust for Fedora

kleptog — Tue, 01 Nov 2022 13:30:21 +0000

> 0. As I mentioned, the meaning of a legal document is generally determined by its contents, not by other stuff, no matter how relevant that other stuff might seem.

AIUI this is a Common Law thing. In much of the world the legal framework takes into consideration what was intended, how the parties understood it and how a reasonable person would understand it. That's why you don't need to add "don't be an asshole" to contracts, it's implied.

Yes, this trips up US/UK companies occasionally when they come up to a European court with the defence: "we adhered to the letter of the contract". Whether courts will understand the correct meaning of "free" is not something I worry about.

Packaging Rust for Fedora

ceplm — Tue, 01 Nov 2022 13:20:02 +0000

> Because it was written in python 2, and python developers decided to phase out python 2.

Sure, so it has absolutely nothing to do with the argument which was taken here against Linux distributions and packaging.

Packaging Rust for Fedora

Wol — Tue, 01 Nov 2022 13:14:09 +0000

Is this the Windows XSX thing?

As I understand it, this keeps multiple copies of dlls, so that you DON'T overwrite an old version with a new one and screw up programs that rely on the old version. So what we want is shared, versioned liberaries, that are happy to share if they're using the same version, but don't trample over someone else's *different* version of the same libary.

Bit like a curated crates setup - if they can make that do that, you'll only have one copy of each crate, and hopefully they'll have some mechanism whereby you can update the crates manifest/linkage without having to relink the program, so if it all goes pear-shaped with a new version, you can revert the link-update.

Cheers,
Wol

Packaging Rust for Fedora

pizza — Tue, 01 Nov 2022 13:10:06 +0000

Great, now what do you do when you don't have the full corresponding source code for that application and all of its statically-linked dependencies?