Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Posted Dec 1, 2019 0:53 UTC (Sun) by mcatanzaro (subscriber, #93033)In reply to: Soller: Real hardware breakthroughs, and focusing on rustc by cpitrat
Parent article: Soller: Real hardware breakthroughs, and focusing on rustc
So here we are.
Posted Dec 1, 2019 1:00 UTC (Sun)
by mcatanzaro (subscriber, #93033)
[Link] (198 responses)
Now it's not a huge deal because right now it's only librsvg and Firefox. But now imagine these same 143 dependencies statically linked to 100 different libraries, plus 100 different applications, and you can see why additional use of Rust is going to cause problems.
It's entirely solvable, but I'm not going to get my hopes up.
Posted Dec 1, 2019 2:37 UTC (Sun)
by mjg59 (subscriber, #23239)
[Link] (195 responses)
Posted Dec 1, 2019 3:13 UTC (Sun)
by Paf (subscriber, #91811)
[Link] (150 responses)
Wasted space and memory for many copies of the same library?
Inability to upgrade to newer library versions without updating everything?
Posted Dec 1, 2019 3:23 UTC (Sun)
by mjg59 (subscriber, #23239)
[Link] (135 responses)
Posted Dec 1, 2019 6:01 UTC (Sun)
by marcH (subscriber, #57642)
[Link]
BTW I heard the packaging similarities between cargo and NPM are not a complete coincidence.
Posted Dec 2, 2019 3:05 UTC (Mon)
by cyphar (subscriber, #110703)
[Link] (132 responses)
I don't think anybody is suggesting upgrading major versions of crate dependencies out from under projects which use a particular version. However, the main usecase where being able to update a shared library is vital is responding to security vulnerabilities -- if you need to patch every vendored version of a particular library in every package which uses it, it's going to be a nightmare. You could make the argument for updating minor or patch releases if the project says its fine with it in its Cargo.toml (since that's what cargo does, and any user which built the upstream source locally would pick up the newest release matching the requirements) but that should depend on what kind of testing the distribution has.
To be fair though, this has been a problem for a lot longer than Rust. We've had this problem with Go for a long time as well -- and modules don't really help us much either (except that you can do the vendoring as a pre-build stage rather than having to commit the vendored sources -- which arguably makes the security handling story worse). I dread the day that there's an RCE in golang.org/x/net -- 90% of Go packages in most distributions (except Debian maybe) would be vulnerable for a long time.
Posted Dec 2, 2019 3:07 UTC (Mon)
by cyphar (subscriber, #110703)
[Link] (130 responses)
Posted Dec 2, 2019 8:01 UTC (Mon)
by joib (subscriber, #8541)
[Link] (129 responses)
Yes, this means more work for builders and more bandwidth usage for mirrors compared to using shared libraries, but doesn't seem like showstopper problems.
For the Rust project to define and commit to a stable ABI, OTOH, is a fairly big and hairy commitment with plenty of opportunity for subtle bugs to slip through. Also, as Rust is still a relatively young language, committing to a stable ABI at this point seems premature.
A third option, if one really wants to use shared libraries, could be to create distro packages of all the Cargo deps of all Rust packages the distro wants to ship, say that the "stable ABI" is "whatever the version of Rust we're currently shipping does", and every update of rustc would then be an ABI break necessitating a rebuild of all Rust packages. This probably needs quite a lot of automation to be implemented to work well.
Posted Dec 2, 2019 9:32 UTC (Mon)
by roc (subscriber, #30627)
[Link] (1 responses)
Posted Dec 2, 2019 9:44 UTC (Mon)
by joib (subscriber, #8541)
[Link]
Posted Dec 2, 2019 9:51 UTC (Mon)
by nim-nim (subscriber, #34454)
[Link] (119 responses)
> Yes, this means more work for builders and more bandwidth usage for mirrors compared to using shared libraries, but
Practically, that does not work, because the constant churn is incompatible with checking anything.
The value of shared libraries is not that they save some disk. The value of share libraries is that they are *shared*, ie provide a common code state others can check and fix.
Ultimate gentooism works fine for individual devs that accept periodic breakage and loss of data (not that dev systems have a lot of data to lose in the first place). It does not work for normal production systems where some level of data loss protection is expected.
Reaching this level of robustness basically means taking some time to freeze code state and QA the result. Continual rebuilds make too many things move at once make QA and QA investigations possible.
Rust devs must cadence their changes like everyone else to work with everyone else.
Posted Dec 2, 2019 10:17 UTC (Mon)
by joib (subscriber, #8541)
[Link] (117 responses)
So no different really from the current way when a C shared library is updated, except that dependencies need to be rebuilt in order to include the updated code.
Posted Dec 2, 2019 15:19 UTC (Mon)
by nim-nim (subscriber, #34454)
[Link] (116 responses)
All those new systems with missing/incomplete/not working shared library support are removing the distribution shared /global layer and provide no replacement.
This, is often by design, because a lot of their proponents have zero notion of component QA and lifecycle management, and can’t imagine anything except “always rebuild everything against the latest 3s old internet version” or “always rebuild everything against moldy x years old locked-in-stone bundled code”.
The “speed” and “ABI” technical arguments are used to evade the lifecycle management question.
Speed is not the primary concern of most users. Manageability is. That’s why most distros do not build with insane optimization settings, and on the contrary, enable debugging options that trade speed for robustness.
Posted Dec 2, 2019 15:39 UTC (Mon)
by rgmoore (✭ supporter ✭, #75)
[Link] (1 responses)
And development speed at the expense of making it slow and difficult to roll out critical patches should not be the primary concern of a language that's allegedly focused on security.
Posted Dec 3, 2019 9:19 UTC (Tue)
by nim-nim (subscriber, #34454)
[Link]
That only works for systems dedicated to a single app, ie app kiosks (which arguably, is Mozilla’s target. It’s just not the target of most projects).
Posted Dec 2, 2019 15:44 UTC (Mon)
by farnz (subscriber, #17727)
[Link] (101 responses)
As an employee at a company that uses a lot of open source, I can unequivocally tell you that the distribution shared lib store is not of value to us; instead, we statically link all but a core "platform" of libraries (glibc and similar low level libraries), and then statically link the rest. The distribution's value to us is that it provides a long tail of utilities in ready-built form for developers to depend on.
It is simpler for us to statically link everything we use, and have processes for automating rebuilds when we have to do a security fix than it is to try and managed shared libraries. This allows us to have temporary version skew between libraries (e.g. "security fix for zlib breaks $IMPORTANT_TOOL, so fix everything else to minimise our vulnerability surface and work on $IMPORTANT_TOOL, rather than remain vulnerable until we fix $IMPORTANT_TOOL"). It also allows us to build the versions we want of everything we use - no need to depend on the distribution working out the compatibility issues between MIT and Heimdal Kerberos, for example, as we can use whichever has the features we need.
Posted Dec 3, 2019 9:38 UTC (Tue)
by nim-nim (subscriber, #34454)
[Link] (98 responses)
The difference between distros and vendor-specific systems is that distros managed this scaling (maybe poorly, but that's way better than what anyone else managed so far)
Posted Dec 3, 2019 9:51 UTC (Tue)
by farnz (subscriber, #17727)
[Link] (97 responses)
Actually, it scales a heck of a lot better than the distro model; we have automated rebuilds and redeploys, complete with pulling in new code from upstream, and we have many thousands of deployed products using hundreds of thousands of open source libraries as well as our in-house code.
The trick is that because we're rebuilding all the time, and keeping up to date, there's only a small trickle of manual work when a rebuild fails - and that can be targeted at the people who care about the product, with the stick of "if you don't get this done by $DATE, we will turn off the product completely". A distro can't do this - it has too many varied users to care about, many of whom care about things we don't. Plus, because we're building everything multiple times, we can do things like have one product use a "common codecs" subset of ffmpeg, while another product uses a much broader build of ffmpeg - tthe second product is more likely to have ffmpeg related issues, but them's the breaks.
In contrast, the distro model depends on co-ordinating large numbers of people, who by and large don't care about the final products we build atop their work, and getting things that we don't care about working is as important to them as getting our products onto current library versions. There is no way, for example, that Debian would accept an upgrade to OpenSSL that temporarily broken all MTAs in the distro - whereas we can upgrade OpenSSL for the things that need newer TLS support, and leave our MTAs on the older versions.
Posted Dec 3, 2019 12:25 UTC (Tue)
by pizza (subscriber, #46)
[Link] (1 responses)
Congratulations, you are the 0.01%.. For every competent shop like yours, there are ten thousand (if not more) that don't have any meaningful regression tests/etc at all. The IoT dumpster fire is only just getting started.
Posted Dec 3, 2019 13:15 UTC (Tue)
by farnz (subscriber, #17727)
[Link]
Sure, and you think that those shops are going to slow down their development process to fit the needs of a distribution? We talk about the IoT dumpster fire because those shops do nothing in the way of update automation - they throw the code over the fence and it's done. In as far as they interact with distributions at all, it's because they choose an ancient and out-of-date baseline (I've been offered Ubuntu 6.10 recently as the "supported" distribution by an IoT SoC design firm), and expect that we'll agree that because it's Linux, it's secure.
Posted Dec 4, 2019 7:41 UTC (Wed)
by nim-nim (subscriber, #34454)
[Link] (94 responses)
Congratulations, you have a scaling factor of one (company).
Designing things that can be reused by others is a lot harder than inventing one-of-a-kind company-specific systems.
Posted Dec 4, 2019 8:36 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (91 responses)
Posted Dec 4, 2019 8:46 UTC (Wed)
by marcH (subscriber, #57642)
[Link] (90 responses)
Posted Dec 4, 2019 9:18 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (89 responses)
Once the CS dept confirms that there are no new issues coming from them, start the gradual general deployment, incrementing the rollout percentage to 100% over the next week or so.
Posted Dec 4, 2019 9:35 UTC (Wed)
by rodgerd (guest, #58896)
[Link] (88 responses)
Posted Dec 4, 2019 20:19 UTC (Wed)
by marcH (subscriber, #57642)
[Link] (3 responses)
Posted Dec 4, 2019 20:22 UTC (Wed)
by farnz (subscriber, #17727)
[Link] (2 responses)
Who in their right mind does a rebuild and redeploy without automatic tests and gradual deployment?!? Surely, by now, that's implicit in "rebuild and redeploy"?
Posted Dec 4, 2019 23:43 UTC (Wed)
by marcH (subscriber, #57642)
[Link] (1 responses)
1. I wasn't answering you.
Posted Dec 5, 2019 0:23 UTC (Thu)
by farnz (subscriber, #17727)
[Link]
Perhaps you shouldn't have been implicit about who you were answering if you dislike the confusion - I replied because LWN's commenting system sent me an e-mail as your comment was a reply to a reply to one of my comments.
And yes, I agree that the implicitness is a problem. A lot of what I'm seeing in this thread is people who are making assumptions about the work practices of another group which no longer apply today - and then asserting that that group is doing something bad because you're assuming something about how they work.
Posted Dec 5, 2019 12:17 UTC (Thu)
by nim-nim (subscriber, #34454)
[Link] (83 responses)
https://lwn.net/Articles/805305/
Maybe the education needs to go the other way? Or, maybe, you’re claiming k8s in 1998’s technology (in that case, what is is your 2019 alternative, please do share)
Posted Dec 5, 2019 12:20 UTC (Thu)
by farnz (subscriber, #17727)
[Link] (80 responses)
Just because distros are an imperfect solution, and k8s is also an imperfect solution, does not imply that they have nothing to learn from each other - and it looks like k8s is attempting to learn from distros, while you're saying that distros do not need to learn from Cargo, k8s, NPM, Jenkins, GitHub Actions, Azure Pipelines, and other such modern things that have learnt from distros.
Posted Dec 5, 2019 15:15 UTC (Thu)
by nim-nim (subscriber, #34454)
[Link] (79 responses)
For example: I spent two years performing 10s of thousands of Go components builds. When Google decided to go full power on modules this year, I filled upstream all the missing bits I needed to make modules work distro-side.
I was laughed out of the conversation by people that were convinced like you that they had learnt everything they needed to learn, that they were the future, distros the past and who cared about them?
End result: the switch to modules as default in Go was cancelled this summer. Between my report and the release date, most key points I had pointed had been duplicated by other people in other build contexts (and duplicated enough the Go project could not continue dismissing them).
So much for distros not understanding new tech. Distros may not understand the coding side as well as devs. They certainly understand the building and distributing part way better than devs.
I hope we can finally make Go modules work reliably, in distros and elsewhere, this winter. But if they don’t work out for distros, you can bet they won’t work out for a lot more people and organizations. No matter how uncool and stuck in the past devs feel distros are.
Posted Dec 5, 2019 16:31 UTC (Thu)
by farnz (subscriber, #17727)
[Link] (40 responses)
I find your projection interesting - I'm actually largely on the distro side, just pointing out that there are other successful models than Linux distros. But you've made it quite clear that either I fall in line with your views based on your experience of Go (not Rust, the language under discussion!) or you will see me as the enemy, so maybe I should stop pushing for more dynamic linking…
Posted Dec 5, 2019 18:16 UTC (Thu)
by nim-nim (subscriber, #34454)
[Link] (39 responses)
I’m just so utterly fed up with all the “distros don’t understand modern tools and should learn some things” vibe.
There are people (not just me, and for every dev language, be it rust, go, python, take your pick) that spend months if not years distro-side transforming dev deliverables into something that can be used and reused in a generic way, and the first dev who harcodes a product and organisation-specific workflow in Jenkins, using distro components as basis, is convinced he knows more about code management and building than distros.
And that would not matter much (though it *is* annoying) if the same people that do not want to invest the time and energy to make things work, were not also continuously lobbying against the changes needed by the people doing the fixing, and asking those people to drop their work and do it using the pet tool and practices of the day. Even when the pet tool and practices are demonstratively unable to handle a lot more, than the perimeter the dev is interested in.
For example, for rust, the baseline question (how to build a whole system against a shared baseline) has not been answered. Distros (and others) are expected to adopt rust, and shamed for not adopting rust yet, and being backwards, etc, etc, when rust does not provide a way to define a system-wide integration baseline, and rust devs do not make this baseline possible even if the tooling was available, because they are continuously deprecating things and relying on latest features only.
Because dev speed, right. The speed that translates in years of integration delay, unless the whole rust project is treated as a private giant Mozilla SCM tree. And that’s innovating how exactly? That’s reproducing the BSD unified CVS tree. Except the BSDs managed to produce a whole system, and rust is nowhere near that so far. Even using modern tooling the BSDs could not dream of.
Posted Dec 5, 2019 19:21 UTC (Thu)
by farnz (subscriber, #17727)
[Link] (37 responses)
The deep problem to face down in Rust is the dependency on monomorphization of generics. This is a deep and difficult problem that nobody has a solution to yet, and arguing for dynamic linking as the solution just reveals ignorance of the problem.
Monomorphization is a really good tool for developers - you write the code once, with generic parameters that have to be filled to make the code useful, and then the compiler duplicates the code for each set of filled in parameters; the compiler's optimizer then kicks in and reduces the set of duplicated functions down to the minimum amount of duplicated machine code given the parameters you've supplied. This is great for correctness - I've only got one version of my code to debug - and for performance - no indirect calls via a vtable, just optimal machine code. The trouble is that when you expose a generic as API, which is a common thing to want to do (e.g. Rust's Vec type is a generic), the result of compiling just your library is not machine code that could go in a dynamic library, but some form of intermediate representation that needs compiling further before you have machine code. However, for obvious reasons, the dynamic linker does not and should not include a full LTO phase.
C++ avoids this by saying that if you have generics in your interface, the only way to use your code is as a header-only library. Rust builds to "rlibs", which are a mix of IR where the optimizer needs more information, and machine code where it's possible to generate that; these are effectively static only, because of the need to have an optimizing compiler around to turn the IR into machine code.
There are people on the Rust side trying to find a solution to this, but it's a deeply difficult problem. At the same time, people who work on Cargo (Rust's build system) are taking lessons from distro people on how best to manage packages given that you need static linking - e.g. "cargo-crev" helps you limit yourself to a trusted base of libraries, not just any random code, and makes it easier to audit what's going into your compile. Cargo makes it impossible (by design) to depend on two semver-compatible versions of the same library; you must pick one, or use semver-incompatible versions (at which point you can't interact between the two incompatible libraries - this permits implementation details of a library you use to depend on incompatible but supported versions of the same thing, for example). Cargo workspaces unify dependency resolution for a set of Rust code into one lump, so that I run my top-level "cargo build", and get warned up-front if I'm working with incompatible libraries in different parts of the same workspace, just as apt warns you if you depend on two incompatible dpkgs. Cargo permits you to yank broken version from the central index, which warns people that they can't use them any more (e.g. because this version is insecure and you must upgrade). And there's work being done to make it possible to run your own partial mirror of the package indexes, limited to the packages that you have vetted already - so you can't even build against something old, because it's not in your own mirror.
This contrasts to Go, where it's common to specify the library you require as a VCS tag from a public VCS hosting service like GitHub. Cargo makes that possible if you really, really want to do it, but it's not easy to do - the goal is very much focused on being able to build up a set of trusted "vendored" packages in your workspace, and then build your code only against the trusted vendored packages, offline, in a reproducable fashion, even if the rest of the world becomes inaccessible.
Posted Dec 5, 2019 21:02 UTC (Thu)
by nim-nim (subscriber, #34454)
[Link] (35 responses)
From a distributor point of view the only things missing to make the build part manageable is an easy way to point builds to a system-wide workspace, and a way to populate the index of this workspace in a granular way. Basically
The new Go module system is a lot less elaborate than your description. However, it will practically allow achieving 1-4 (not always by deliberate upstream design, but the end result will be the same).
I agree that the Rust community has generally nicer and saner behaviors than some I’ve seen Go side. However, natural selection will keep only fit projects alive in both communities. As long as the core tooling enables good practices, I’ll expect the kind of convergent evolution we’ve seen between rpm and apt (for example).
On the deploy front, however, without shared libs, a large number of applications, with correct system-wide build dependency management, probably implies frequent cascading rebuilds, with the associated stress mirror, network, disk and user side. Even more so if the dev community lacks discipline and continually churns code that does not work with older semvers.
We’re not seeing this effect yet because first, there are not so many first class applications written in those new languages, and second, there are no effective system-wide build dependency management systems yet (so we’re missing a large number of rebuilds that would probably be desirable from a security or bugfix perspective)
Users are deeply suspicious of mass updates. The software world as a whole failed them all too often. It’s a lot easier to make them accept a single lib update, than replacing large sets of apps that happen to have the same code built-in (and, I’m talking about technophile users here, not the kind of user targeted by RHEL).
Posted Dec 5, 2019 21:36 UTC (Thu)
by nim-nim (subscriber, #34454)
[Link]
The critical part is managing a system-wide shared layer, either at run (shared libs) or build time.
You can buy new network links and new hardware. Without a system-wide shared layer, you need a lot more human brain-cells, and that’s a lot harder to procure, especially in a community context.
Posted Dec 6, 2019 12:00 UTC (Fri)
by farnz (subscriber, #17727)
[Link] (33 responses)
All of your asks have been possible with Cargo for some years now.
Given that distros still aren't happy with the Rust story, I'm guessing that you've missed a requirement or two that matters to distros here - it would be helpful to know what's missing so that it can be added to Cargo.
Posted Dec 6, 2019 16:30 UTC (Fri)
by nim-nim (subscriber, #34454)
[Link] (32 responses)
Nevertheless, it is probably automate-able (with lots of pain). But, it is useless, if the crates themselves are constantly churning and invalidating the system workspace.
If the system workspace is too hard to set up and use, no one will try to apply a code baseline, and creating a baseline from scratch from projects that didn’t try to converge on a baseline is hard.
Posted Dec 6, 2019 16:39 UTC (Fri)
by farnz (subscriber, #17727)
[Link] (4 responses)
If all you want is what C/C++ and Golang give you, just set up a workspace, and build in there. Most of the steps I've described only exist because you wanted to not depend on what was present on disk, but wanted to identify the missing packages and build them; C/C++ don't even have dependency tracking for you to copy form.
If all you want is what Golang provides, create a file Cargo.toml in your buildroot, containing a workspace section:
Then, unpack your crate into a subdirectory of your buildroot, and cargo build --offline will do the needful using the system crates.
Posted Dec 6, 2019 16:50 UTC (Fri)
by nim-nim (subscriber, #34454)
[Link] (3 responses)
However, I suppose that what makes my distributions friends working on rust most unhappy, is the constant dev churn, because devs do not know or use the default system crate locations, and just push change over change without trying to reuse stabilized crate versions.
Posted Dec 6, 2019 17:01 UTC (Fri)
by farnz (subscriber, #17727)
[Link] (2 responses)
In practice, you'd do this setup once, and not repeat it - unlike an env var, it's then persistent forever. There's no need to version the paths of crates; crates include metadata that identifies their version, and you can rename them to include versioning without ill effect in your system crate pile.
And the dev churn really isn't that bad in Rust - not least because Cargo does semver aware versioning to begin with, so most crates ask for things by semver-aware version.
Posted Dec 6, 2019 20:05 UTC (Fri)
by nim-nim (subscriber, #34454)
[Link] (1 responses)
That’s a dev POW. A dev sets up its environment, it then never changes.
To get distribution synchronization, you need an environment shared by every dev, every packager, and the buildsystem. That means either an hardcoded default location, or a setting in a default system config file, or at least something dirt easy to set, like an environment variable.
Not relying on everyone setting manually the same default in each and every build env.
> There's no need to version the paths of crates;
Unless there will never be two rust apps that need the different versions of the same crate to build, you need to version paths because otherwise the paths will then collide at the share distribution level.
And you can say
> you can rename them to include versioning without ill effect in your system crate pile.
Yes we can do a lot of things at the system level. At some point people tire of working with stuff that need massaging before being used.
> most crates ask for things by semver-aware version
If crates always ask for the latest semver available, because devs have to pull things from the internet (there is no shared system crate store) so why not pull the latest one while you're at it, you *will* get a race to the next version and constant churn.
Posted Dec 6, 2019 20:14 UTC (Fri)
by farnz (subscriber, #17727)
[Link]
Crates do not ask for the latest semver available, because Cargo locks down crate versions on first build of a binary target. So, once I've specified (say) lazy-static = "^1", the first time I do a build with that dependency, Cargo will record the version of lazy-static it found, and use that for subsequent builds; I have to explicitly demand an update from Cargo before it will move onto a new version.
If I'm working in a workspace, then Cargo will resolve deps workspace-wide, and lock me down workspace-wide; this is the recommended way to develop Rust, as it means that you don't bump versions of crates you depend upon, unless you explicitly ask Rust to do that.
And I would note that this is no different to my experiences as a C developer working on the kernel and graphics drivers - when doing that, I used the kernel, libdrm, Mesa3D, intel-gpu-tools and the entire X.org stack from git to avoid duplicating work, and ensure that if I *did* have a fix for a problem, it wouldn't conflict with upstream work. The only difference in Rust is that, right now, there's a lot of churn from Rust being a young language.
Posted Dec 6, 2019 16:41 UTC (Fri)
by nim-nim (subscriber, #34454)
[Link] (26 responses)
and boom, the systems knows about it as soon as the fontconfig index is rebuilt (and you can force the rebuilt with fc-cache).
That’s how hard it should be to register a crate in the system workspace. Drop in standard place. Optionally, run a reindex command.
And using this store should be either the default behaviour or just a shell variable.
Anything more complex than that will require more motivation to use. Humans are not good at using inconvenient things by default.
Posted Dec 6, 2019 16:44 UTC (Fri)
by farnz (subscriber, #17727)
[Link] (25 responses)
Hang on - you asked how to translate the deps chain from Cargo to the distro packaging system, and now you're talking about how you bypass the distro packaging system.
This is moving the goalposts - an equivalent is given that I need the font Kings Caslon, how do I find and install the Debian package for it? It's a *lot* harder than just copying the file into place - I have to somehow translate the font name "Kings Caslon" to a Debian package name, download and unpack *that* package, and *then* I can drop it into place.
If all you want is to use a system package source, just put the package sources in a known location, and (as per my other comment) set up a workspace to build your new code in. Job's done - all of the rest of the work is about setting up that system package source to begin with.
Posted Dec 6, 2019 18:42 UTC (Fri)
by nim-nim (subscriber, #34454)
[Link] (24 responses)
# dnf install font(kingscaslon)
Sorry about that:) It is good that we have this conversation.
What you feel is difficult, is not difficult at all distribution-side. Distributions had to invent names for all the things they ship, to make them exist in their index. So defining naming conventions, is part of the distribution bread and butter.
In my example the distribution did not even invent a mapping, the mapping was defined upstream, it’s the output of
$ fc-query --format '%{=pkgkit}' <fontfile>
I see we come from very different perspectives, and what is evident to me is not to you (and vice versa). Therefore, I will expand quite a bit.
Distributions want to make things available to the rest of the distribution, and to their users.
For rust, those things necessarily include the crates rust software was built from, because rust does not use dynamic libs. Therefore base-lining at the source code level is imposed by the language. Base-lining is a requirement to coordinate the work of all the distribution contributors.
Things that distributions can handle best:
1. exist under a standard system-wide root (/usr/share/fonts, /usr/lib, FHS and XDG are distribution keystones)
2. have a standard filename or path under this root, that avoids collisions between different artefacts and versions of the same artefact
3. are discovered and used by whatever needs them, as soon as they are dropped under the default root, with the default filename (eventually after reindexing)
4. have standard artefact ids and versions, and standard version conventions that map to something similar to semver
And then, for convenience (but convenience matters a lot to over-committed distribution contributors):
1. you have a command that takes things to deploy anywhere on the filesystem, and deploys them to their standard root (with $PREFIX handling pretty please, many build systems use fakeroots not advanced filesystem overlays)
2. you have a command that reindexes the whole root, if the language needs it (fc-cache, ldconfig)
4. you have a command, that outputs what a new artefact needs in various contexts (for building, for running, for testing, etc). Need = artefact ID + minimal semver, not artefact ID + locked version.
All of this can eventually be recreated and redefined by distributions, if not provided upstream. However, the more work it is, the more likely distribution people will focus on mature languages, for which this work has been done a long time ago (ie C/C++).
A big difference with IDE and dev tools, is that distributions emphatically do *not* want to frob structured formats to do things. Just give use standard commands. We don’t want to do xml for language A, yaml for language B, json for language C, toml for language D, ini for F, etc. Format wars are dev fun, not integrator fun. Everything is a file under a standard place with a bunch of standard manipulation commands and eventually a set of standard shell variables is plenty enough for us.
When all of this exist, packaging a new upstream artefact is just:
1. take the upstream bunch of files
2. execute the command that tells what other artefacts and artefact versions they need for building (and optionally, testing)
3. translate the artefact ids into distro ids
4. use semver to map the minimal versions asked by the artefacts into whatever is available in the distribution baseline
5. install the result (make it exist in the standard root, reindex the result)
6. run the standard build command
7. run the standard install command (with prefix-ing). In the rust case, that would include the command that deploys the crate under the default root, for reuse
8 run the standard test command
9. run the standard "what are you" command on the resulting files, so the corresponding ids and versions can be added to the distribution index
Forcing a version is possible in 4, as a manual exception in the build process, but it’s not the default behaviour, because doing otherwise is just piling technical debt.
And that’s basically all tooling side. Once this part is done, there is no difficulty in packaging large numbers of artefacts, as long as their code and dependencies are sane, and the churn is not incompatible with the target cadencing
Posted Dec 6, 2019 19:02 UTC (Fri)
by nim-nim (subscriber, #34454)
[Link] (22 responses)
Conventions and defaults create network effects. Humans strive on habits and routines.
The difference between “plop file here and it will be used” and “just edit this file in <my pet> format, for <every> artefact, and set <whatever shared root you want to use” is technically slight but huge from a human networking point of view.
Posted Dec 6, 2019 19:14 UTC (Fri)
by farnz (subscriber, #17727)
[Link] (21 responses)
The thing is that I'm a Fedora packager - none of what you're explaining to me is news to me, and as someone who *also* develops in Rust, I don't see the difficulties in mapping Rust to Fedora packages (although I'm aware that they exist, they are largely a consequence of Fedora fighting hard to dynamically link instead of statically link, which, due to the monomorphization problem, is a hard problem for Rust).
Can you please stop assuming that I'm clueless, and start focusing on what, exactly, it is about Rust and Cargo that makes it hard for distros to package Rust code, beyond the fact that Rust crates are hard to dynamically link using a linker designed around the needs of 1980s C code?
Posted Dec 6, 2019 19:48 UTC (Fri)
by nim-nim (subscriber, #34454)
[Link] (20 responses)
Fedora is fighting hard to overcome the churn Rust side.
Some of the most awful and broken things in modularity, were constructed specifically for Rust, not to dynamically link, but to cope with the terrible amount of churn the SIG had to face.
And why do you get this churn in Rust? Because Rust has no notion of a shared code state layer at the system level. There is no baseline to build upon, every dev just bumps its code needs all the time.
The baseline could be constructed with shared system libs (that rust does not have) or with a default shared workspace at the system level (that rust does not have either). Or, maybe someone will have another idea.
Fact being, this baseline does not exist now. Without something to synchronize upon, people do not synchronize. Without synchronization, it’s a race to the latest version bump. It’s no more complex than that.
Posted Dec 6, 2019 20:03 UTC (Fri)
by farnz (subscriber, #17727)
[Link] (19 responses)
Rust does have the notion of a shared state - it's part of the workspace model. And the only reason the churn was an issue for the SIG (I followed this, BTW) is that it becomes impossible to maintain shared library packages for crates when they are being developed at high speed, and you don't have tooling to use Cargo to keep up with the churn.
In other words, this isn't a technical issue - it's that Rust is currently a young language, and developers are releasing early and releasing often. The only reason Fedora doesn't have a churn problem with C++ or C code is that the libraries in use in C and C++ have been around for a long time, and are consequentially relatively stable, as compared to Rust libraries, and thus Fedora being 12 months behind current doesn't actually make much difference.
If Fedora wanted to, it could treat the churn in Rust the way it does in C and C++ code - stick to relatively stale versions, and hope that the features users want are present. The trouble with that from Fedora's point of view is that with the current pace of Rust application development, Fedora would look bad because of all the missing features that upstream advertises, but Fedora does not support.
Posted Dec 6, 2019 20:21 UTC (Fri)
by nim-nim (subscriber, #34454)
[Link] (11 responses)
That would not work for Rust.
That works for C/C++, because the distribution versions choices are materialized in dynamic libraries. It’s easy for upstreams to check what library versions are used by major distros. It’s easy to make sure they have something that works with those versions, if they want to reach users. Also that pushes upstreams to agree on the next version they’ll consolidate upon because they know distros won’t ship every possible variation under the Sun.
Yes, I know, terrible dev oppression with stale versions.
In the meanwhile, the stuff produced by oppressed devs gets distributed, and the stuff produced by high speed rust devs, does not.
A formula one can be reinvented for every race, and stop at pit stands every lap (high speed car fixing). A normal car better run a long time without changes. Changes in normal cars better be batched R&d side for next year’s model.
Posted Dec 6, 2019 23:36 UTC (Fri)
by farnz (subscriber, #17727)
[Link] (10 responses)
I disagree fundamentally with you, then; the only reason I ever use my distros -devel packages for work C and C++ code is because it's usually a right pain in the backside to go to upstream and build what I want for myself. Cargo makes it easy for me to build my upstream Rust dependencies for myself, so why wouldn't I do that?
This leads to different incentives - for C and C++ libraries, I want to make life easy for distros, not because distros are necessarily good, but because they are considerably easier to deal with than either vendoring all my dependencies or getting my potential users to build them in a way that I can use. In contrast, over in Rust land, I can tell my users to run cargo install $cratename, and my crate is automatically built and installed for them.
So, from where I'm sitting as a dev, the distros fill a gap in the C and C++ ecosystem that's filled by Cargo in the Rust ecosystem. All the distros need to provide is C and C++ packages, plus rustup, and I can get users to install my code without further involvement from the distro. Remember the trifecta: users, operations, developers; because C and C++ have (quite frankly) awful tooling for non-developer installation of code, there's room for distros to get in quite a lot of operations stuff while acting as a link between developers and users, because if distros refuse to provide that link, the users can't get the developers' code to work.
In contrast, because Cargo provides most of what a package manager provides to a user (install, upgrade, uninstall, dependency resolution, rdep handling), if distros try too hard to get in the way, I can bypass them relatively easily. You thus have a problem - the operations leverage over developers in the C world is that without the distro compromise that operations will accept, users can't run developer code. This isn't true in the Rust world - Cargo is good enough as a package manager that I can bypass operations.
So, distros no longer have leverage; how do operations get heard in this world, given that if you push back against good development practices like "releases early, release often", users will bypass distros to get the tooling they want? Remember that Linux is already a minority OS - if you're saying that you're going to hold it back further, users will just bypass distros if it's easy to do so, as they're the people who, today, are already motivated to find solutions to problems beyond following the pack.
Posted Dec 6, 2019 23:56 UTC (Fri)
by pizza (subscriber, #46)
[Link] (5 responses)
Not according to Microsoft -- As of July 2019, Windows Server is now the minority on Azure [1], and that's the cloud provider with the lowest Linux percentage.
[1] https://www.zdnet.com/article/microsoft-developer-reveals...
Posted Dec 6, 2019 23:58 UTC (Fri)
by farnz (subscriber, #17727)
[Link] (3 responses)
That's server-side, not total - and server side is exactly the place where you have developers who can run tools like Cargo instead of being helplessly dependent on distros. In other words, that's where distros are least useful to begin with, beyond the base libraries to get your developers going.
Posted Dec 7, 2019 0:43 UTC (Sat)
by pizza (subscriber, #46)
[Link] (2 responses)
I have no idea what point you're trying to make, beyond "distros are useless, because reasons"
Posted Dec 7, 2019 11:42 UTC (Sat)
by farnz (subscriber, #17727)
[Link] (1 responses)
I'm saying that Linux distros are not a significant driver for distribution of code; server-side, you do whatever the devs want you to, client side is iOS, Android, Windows etc.
This, in turn, pulls down their influence - why should the authors of Krita, or someone writing code to run on Amazon AWS or Google Cloud Engine, care if their users have to run "cargo install", or "yarn install", instead of "apt install"? Unlike C++ land, where dependency management is otherwise manual, modern languages don't need distribution packaging to be easy to install and use code written in those languages - and that means that distros no longer hold the place of power they do in C++ land.
Posted Dec 8, 2019 21:47 UTC (Sun)
by flussence (guest, #85566)
[Link]
Posted Dec 7, 2019 0:37 UTC (Sat)
by Cyberax (✭ supporter ✭, #52523)
[Link]
Posted Dec 7, 2019 10:19 UTC (Sat)
by nim-nim (subscriber, #34454)
[Link] (3 responses)
And, for Rust, you do not want to make it easy on distributions.
You’re pushing complexity on distributions.
You’re pushing complexity on users (cargo install foo is nice for a limited set of fringe components, do you actually expect users to construct the thousands of components composing a system like that? That requires big company-level of hand-holding).
And, you’re surprised distributions feel C and C++ libraries are better suited for systems works (what distributions do)? And do not want Rust anywhere near their core in its current state?
Really, what did you expect?
A system that does not incentivize, making the life of others easier, will result in little adoption by those others.
And you can feel distributions are “not necessarily good”, but that’s irrelevant, unless you want to pick up the distribution work yourself. That won’t leave you with much time for dev, but such is real life, removing someone that provides a service to you, does not make the need for the service go away.
Posted Dec 7, 2019 11:49 UTC (Sat)
by farnz (subscriber, #17727)
[Link]
You're missing the point - code is written by developers. If developers move en-masse to a new language, distros don't get a choice about wanting C instead of Rust - they have to live with it, because the users of code want the new features of the new versions of code.
In C land, distros have power because there's no dependency management without them. In Rust land, if distros get between users and devs, they're trivial to bypass. We've already seen what happens with C code where the developers and users agree that it's beneficial to move to the new shiny - distros have to move, too, whether they like it or not, because users want the new GNOME etc, and thus distros have to accept GNOME's dependencies.
Stop thinking that distros matter to users - they don't, particularly, and there's a reason that no other OS has a distribution-like model. Users care about applications that make their computers do useful things; if distributions make it harder to get useful things than bypassing distributions, then users will bypass distributions.
If the distributions can't work out how to work with Rust developers, and system software migrates to Rust, then distributions will get reduced to sufficient tooling to run rustup and Cargo, because that's all that users will want from them; if distributions don't want to end up here, then they need to work with Rust now to get to a state where they are still relevant.
And note that Rust does make it as easy as it can on distributions - Cargo even translates its own dependency information to a simple JSON form for you, with or without resolving versions at your discretion. If distributions want Rust to do more than it does for them, then they need to engage with the Rust community to explain what they need and why.
The discussion with nim-nim elsewhere in this thread is typical, FWIW - as they asked for things from Rust, I explained how to get what they want from Cargo, and then they went off into unrelated territory. My favourite part of that was their claim that I just need to run dnf install font(kingscaslon), and dnf would package my font for me, then install it…
Posted Dec 7, 2019 13:11 UTC (Sat)
by pizza (subscriber, #46)
[Link] (1 responses)
This is the bit that I keep coming back to. _someone_ has to do this work, and if it's not the distros [1], and not the language/tool vendors, then it's going to have to be you [2]. It's legit to say that the traditional distro model has its flaws and room for improvement, but one would have to be pretty naive to claim the sorts of problems and concerns distros have traditionally solved (ie "testing, deploying and maintaining configuration managed baseline systems useful for 3rd parties to develop on/for") somehow no longer matter.
[1] Not just distros ala Ubuntu, but whatever k8s and various proprietary cloud vendors come up with to fill the same role.
[2] "you" can mean an individual developer or an tools/platform team that big orgs tend to have. With every one of them reinventing the same wheel. If only there were independent organizations that could take on this work for the benefit of all involved... I know, we'll call them "distributions"...
Posted Dec 7, 2019 14:12 UTC (Sat)
by farnz (subscriber, #17727)
[Link]
And therein lies the rub - part of the problem here is that distros are saying that someone else should do that work - and it looks like, at least for Rust, the language/tool vendors are going to do it well enough for users' and developers' needs, and simply bypass the distros altogether, with things like crater for testing the entire package ecosystem in a single build.
Note that this has already happened for JavaScript - if that's your language of choice, you get Node.js from the distro, and then switch to NPM or yarn from there on in; users are happy to run bundles created by developer tooling, and the distro becomes an irrelevance as soon as it's good enough to get Node.js to run. Maybe that's a good end point for distros - enough runutime etc to let the language tooling take over?
Posted Dec 6, 2019 20:32 UTC (Fri)
by nim-nim (subscriber, #34454)
[Link] (6 responses)
So, if the community wanted it, the technical aspects could be changed.
However, it is too easy to attribute it to the language age. Java never outgrew the churn state, and few would call the language young today. Early choices, made when a community is young, and its members few, easily set in hard habits later.
Posted Dec 7, 2019 4:18 UTC (Sat)
by mathstuf (subscriber, #69389)
[Link] (5 responses)
Then distros' jobs would be to say "hey, there's a security fix in x.y.z, please update your minimum to x.y.z+1 and make a new release" and every stable-tracking distro gets a similar fix available. Of course, if the distros provide an index that cargo can read, just not providing the minimum and giving a minimum-with-fixes available (treating the intermediate versions as "yanked") will likely have the same effect. Investigation into that as a viable alternative would be necessary.
Posted Dec 7, 2019 8:46 UTC (Sat)
by nim-nim (subscriber, #34454)
[Link] (4 responses)
That’s the thundering herd effect that makes it unpractical, to package large amounts of software, if upstream devs do not coordinate their version needs a minimum (and they can coordinate via shared libs, unified master company scm like Google, anything else but the coordination is necessary). semver alone does not work if everyone keeps requiring semver tip versions.
You can delay the effect with containers or static linking, but as soon as a critical fix is needed, it comes back with a vengeance. The dev dream of “any dev uses whatever versions it wants and the rest of the supply chain will cope” is utopic in presence of un-perfect code, that will always require an eventual fix.
> Of course, if the distros provide an index that cargo can read
To find out what is available on my current distribution:
$ dnf repoquery -q --provides $(dnf repoquery --whatprovides 'crate()') | grep 'crate('
crate(abomonation) = 0.7.3
To install one of those
$ sudo dnf install 'crate(abomonation) = 0.7.3'
That could probably be streamlined and plugged into cargo if someone wanted to (it would be much better if cargo provided a standard name and version mapping for rpm and apt at least, that would simplify reading it back cargo-side; the artefact naming is pretty flexible distribution side but the version format less so, as it is used to compute upgrade paths. The original semver format, before non Linux people added rpm and deb-incompatible things in it, was easy to map).
And then the result of the install is useless for the rust dev if there is no system workspace enabled by default. The crates will be installed but not used.
> Of course, if the distros provide an index that cargo can read, just not providing the minimum and giving a minimum-with-fixes available (treating the intermediate versions as "yanked") will likely have the same effect.
Yes, that’s the most likely scenario distro-side. There is a limited set of checked and available versions, anything else is considered yanked. Doing things any other way means drowning under version combinations.
Posted Dec 7, 2019 11:52 UTC (Sat)
by farnz (subscriber, #17727)
[Link]
Every one of those crates is using the Cargo name and versioning - there is a trivial mapping from Cargo to dnf notation, and back again. The trouble is that the distro has not written the code to do that mapping for me, but is expecting that Cargo will magically stop using its own dependency resolution and handling (which is at least as good as dnf's) and start using the distro's code, just because that's what the distro wants.
Posted Dec 7, 2019 13:33 UTC (Sat)
by mathstuf (subscriber, #69389)
[Link] (2 responses)
If OpenSSL 1.0.2j has some critical fix, why would I want to allow my software to compile against 1.0.2i anymore? Bumping the minimum helps *everyone* get better versions over time, not just those who remember to rebuild their OpenSSL.
The current flurry of changes needed for minvers support is because it is new in Cargo. If it had been there from the beginning, it likely would have been much easier to handle. But, this is the same problem with any bootstrapped initiative be it reproducible builds, new architectures, etc. The initial push is large, but the constant churn should be quite low.
Posted Dec 7, 2019 14:09 UTC (Sat)
by smurf (subscriber, #17840)
[Link] (1 responses)
All of this can happen automagically, given modern tooling (which Debian for one is slowly moving towards – much too damn slowly for my taste) … except for the "fix the inevitable regressions" part of course.
Posted Dec 8, 2019 11:33 UTC (Sun)
by mathstuf (subscriber, #69389)
[Link]
I'm arguing that Debian shouldn't make its own house fine and not let upstream know that there's an issue with one of their deps.
Posted Dec 6, 2019 19:10 UTC (Fri)
by farnz (subscriber, #17727)
[Link]
Yet again, I am completely lost in what you're suggesting - you keep jumping around between what has to be done to package something (which is the core of the discussion), and what can be done when you *have* packaged something.
I have an unpackaged TTF file for Kings Caslon (it's a proprietary font, so not in any distro). When I run dnf install font(kingscaslon) on Fedora 31, it does not find my TTF, package it, and install it in the right place - how do I make that command work for me, since you claim it solves the problem of packaging a font?
Rust has already solved the naming problem to Rust's satisfaction - in practice, you use the name any given crate has on the primary crate index for the community, and you're done. For Rust's needs, because Cargo does semver-aware version resolution, and tries to minimise the number of crates it pulls in subject to dependency resolution, this also keeps the dependency tree under control.
It sounds, from what you saying, like distros can't handle C libraries - they don't meet 2 or 4. Rust meets all four of your requirements, and the only bit missing is the command to translate from Rust's crates.io names to distro names, and use that for dep resolution instead of asking Cargo to download from crates.io.
Posted Dec 9, 2019 1:22 UTC (Mon)
by kvaml (guest, #61841)
[Link]
Posted Dec 10, 2019 12:41 UTC (Tue)
by jezuch (subscriber, #52988)
[Link]
Just my 2¢
Posted Dec 5, 2019 18:22 UTC (Thu)
by marcH (subscriber, #57642)
[Link] (37 responses)
I think we're getting dangerously close to "he said that she said that they said..." :-)
> So much for distros not understanding new tech. Distros may not understand the coding side as well as devs. They certainly understand the building and distributing part way better than devs.
Well said.
> I was laughed out of the conversation...
By the way: while sometimes tense, I found this particular discussion here probably the most interesting on the topic I've seen yet. I didn't see anyone laughing anyone else out of the conversation yet.
While this topic is typically territorial/tribal (we're only humans), I think the real issue is not so much about the ideal design and place to implement release management but more about who has the time, skills, resources and motivation to_ actually do it_. In other words:
- if "upstream" developers suck at software distribution and integration (often true), then better let distributions do the work. They've done it for ages.
I don't care that much whether the command I need is "dnf update python3-foo*" versus "pip3 update foo*". What I really care about is the release and QA efforts behind it - the results of which should be easy to browse. "Open software", not just "open source".
A third possibility is for developers to work in Linux distributions directly. Unfortunately there are too many of them, so this typically translate into just one Ubuntu repository.
Posted Dec 5, 2019 19:26 UTC (Thu)
by nim-nim (subscriber, #34454)
[Link] (36 responses)
Distributions care about integrating.
Therefore, first, distros won’t have a hard time migrating to demonstratively better packaging technology, if it ever becomes available (the apt/rpm duopoly exists because apt and rpm are so close capability wise, so there’s no good reason to drop existing investment in one for the other). flatpack as a technical format, is a poor limited copy of either of those.
And second, it won’t matter one bit, because the basic integration requirements are independent of the technical format (most of them, are even independent from the operating system; copying a lot of features of Linux packaging systems in Windows Update served Microsoft well). Therefore, changing the format, won’t remove the friction with devs, that do not want integration constrains to exist in the first place.
Lastly, the more a dev-oriented packaging format matures, the easier it is to map into deb or rpm. Those are vastly more capable than all the dev-packaging managers I've inspected. What’s blocking most mapping attempts is not missing capabilities deb or rpm side. It’s missing capabilities dev package side. Those missing capabilities require a lot of distribution work to fill, package by package.
Posted Dec 5, 2019 19:38 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link] (34 responses)
Posted Dec 5, 2019 19:56 UTC (Thu)
by farnz (subscriber, #17727)
[Link] (25 responses)
Part of the problem there is that, assuming we distribute machine-ready binaries, you're stuck choosing between combinatorial explosions in the number of binaries you test (as you need to test all reasonable combinations, even if they are ABI-compatible), or you end up losing many of the benefits the distros provide as you provide packages that simply don't work together, even though they can be installed together.
The other component is that a decent amount of the hard work distributions do (that makes the ecosystem a better place) is a consequence of that restriction - if there can only be one version of (say) zlib on your system, the distro has incentives to make sure that everything works with the current, secure, version of that package. If I can have multiple versions of it installed, why can't I stick to the known bad version that works, and just complain about attempts to move me onto the newer version?
Posted Dec 5, 2019 20:56 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link] (24 responses)
> Part of the problem there is that, assuming we distribute machine-ready binaries, you're stuck choosing between combinatorial explosions in the number of binaries you test (as you need to test all reasonable combinations, even if they are ABI-compatible)
This model does lead to a proliferation of versions, just one tardy package can force the distro to keep around a huge graph of old dependencies. But this is a balancing act and can be managed.
I worked in a company that did dependency management like this. It worked mostly fine, the major problem was "migration campaigns" when a central dependency had to be upgraded and individual teams had no bandwidth to do it.
Posted Dec 5, 2019 21:21 UTC (Thu)
by nim-nim (subscriber, #34454)
[Link] (19 responses)
There is no such insistence of a single version distribution side. Haven't you noticed the dozens of libs available in multiple versions on all major distributions? The way they've all managed the python 2 and 3 overlap?
So, technically, multiple version works in all distributions. Practically:
> the major problem was "migration campaigns" when a central dependency had to be upgraded and individual teams had no bandwidth to do it.
devs severely underestimate the connectivity of the current software world, and the team bandwidth that would be implied by the scope of versions they demand angrily.
It would be a lot easier for distributions to provide recent versions of all components, if dev projects adhered strictly to semver.
> this is a balancing act and can be managed.
And distributions are managing it. More contributors, more packages, in more versions. Less contributors, less packages, more version consolidation. Nothing more and nothing less.
Posted Dec 5, 2019 21:35 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link] (18 responses)
> Haven't you noticed the dozens of libs available in multiple versions on all major distributions? The way they've all managed the python 2 and 3 overlap?
> So, technically, multiple version works in all distributions.
To give you an example, right now I have a problem with python3-theano that has broken one our application in the 1.0.4 version. I can install 1.0.3 from the previous version of the distro by pinning the version, but this breaks _another_ application. There's no way to install 1.0.3 and 1.0.4 in parallel and say that "AppA wants 1.0.3, AppB wants 1.0.4".
I'm going to fix it by just packaging everything in a venv and abandoning system packages altogether.
> And distributions are managing it. More contributors, more packages, in more versions. Less contributors, less packages, more version consolidation. Nothing more and nothing less.
Posted Dec 5, 2019 21:57 UTC (Thu)
by nim-nim (subscriber, #34454)
[Link] (1 responses)
Now, that supposes (on the *language* side, not the *distribution* side) that
If the language provides none of those, you are stuck installing a single version on system, or playing containers, venvs, and all those kind of things, whose sole purpose is to isolate upstream language tooling, from versions it can not cope with.
And, maybe distributions should have, changed all language stacks to work in parallel version mode. Maybe they tried, and failed. Maybe they didn’t even try
However, it’s a bit rich to blame distributions, and ask them to adopt language tooling, when the problems pointed out, are inherited from language tooling in the first place.
Posted Dec 5, 2019 23:23 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link]
This means that the distro becomes nearly useless for me, except as for very basic system utilities.
> * first, the language deployment format uses different file paths of all the version sublevels you want to exist
Posted Dec 5, 2019 22:02 UTC (Thu)
by pizza (subscriber, #46)
[Link] (11 responses)
...So why isn't fixing the application(s) an option here? (Root cause analysis and all that..)
> There's no way to install 1.0.3 and 1.0.4 in parallel and say that "AppA wants 1.0.3, AppB wants 1.0.4". I'm going to fix it by just packaging everything in a venv and abandoning system packages altogether. [...] Distros have dropped the ball here. Completely. They are stuck in the "single version or GTFO" model that is not scaling past the basic system.
But.. what you just described isn't actually a "distro" problem at all, as the inability to install multiple versions of a given module system-wide is a limitation of python's own packaging system. Intentionally so, as managing multiple versions introduces a great deal of complexity. Instead of dealing with that complexity head-on they decided to take the approach of self-contained private installations/systems (aka venv).
But while that keeps applications from stepping on each other's toes, it doesn't help you if your application's dependencies end up with conflicting sub-dependencies (eg submoduleX only works properly with theano <= 1.0.3 but submoduleY only works properly with >= 1.0.4...)
(This latter scenario bit my team about a month ago, BTW)
Posted Dec 5, 2019 23:17 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link] (10 responses)
In this case this is easy, but I had much more complicated cases with binary dependencies that Just Didn't Work.
> But.. what you just described isn't actually a "distro" problem at all, as the inability to install multiple versions of a given module system-wide is a limitation of python's own packaging system.
For example, back at $MYPREV_COMPANY we had a packaging system that basically provided a wrapper launcher taking care of that. So instead of #!/usr/bin/env python3" we used "#/xxxxx/bin/env python3" which created the correct environment based on the application manifest.
> But while that keeps applications from stepping on each other's toes, it doesn't help you if your application's dependencies end up with conflicting sub-dependencies (eg submoduleX only works properly with theano <= 1.0.3 but submoduleY only works properly with >= 1.0.4...)
Posted Dec 6, 2019 8:39 UTC (Fri)
by nim-nim (subscriber, #34454)
[Link] (9 responses)
> system-wide is a limitation of python's own packaging system.
None of this works at scale. It’s all local dev-specific workarounds:
A language that actually supports multi versioning at the language level:
1. defines a *single* system-wide directory tree where all the desired versions can coexist without stomping on the neighbor’s path
Because, as you wrote yourself:
> the major problem was "migration campaigns" when a central dependency had to be upgraded and individual teams had no bandwidth to do it.
Therefore, anything manageable at scale must keep semver exceptions as exceptions, not the general case.
Most languages do not support multi-version (that’s a *language* not a distribution limitation). Some languages do:
You can trick a mono-version language locally by installing only a specific set of components for a specific app. That does not scale at the system level, because of the version combination explosion at initial initial install time, and the version combination explosion at update decision time.
The other way that devs, that like to pretend the problem is distribution-side, workaround mono-version language limitations, is to create app-specific and app-version-specific language environments. That’s what vendoring, bundling, Java PATHs, and modularity tried to do.
All of those failed the scale test. They crumble under the weight of their own version contradictions, under the weight of the accumulated technical debt. They are snowflakes that melt under maintenability constrains. They only work at the local scale (app or company-specific local scale, limited time scale of a dev environment).
You want multi-version to work for foo language, ask foo language maintainers for 1–2–3. You won’t get distributions to fix language level failures by blaming them while praising at the same time the workarounds language devs invented to avoid fixing their runtime.
¹ Both distro *and* enterprise side, I have a *very* clear view @work of what it costs enterprises; and it’s not because of distros since everything Java related is deployed in non-distro mode
Posted Dec 6, 2019 8:50 UTC (Fri)
by Cyberax (✭ supporter ✭, #52523)
[Link] (8 responses)
Pretty much all of its software is built this way, with dependency closures sometimes having many hundreds of packages. The build/packaging system supports Python, Ruby, Go, Rust, Java and other languages with only minor changes to the way binaries are launched.
So I can say for sure that this approach scales.
> A language that actually supports multi versioning at the language level
> Therefore, anything manageable at scale must keep semver exceptions as exceptions, not the general case.
> All of those failed the scale test. They crumble under the weight of their own version contradictions, under the weight of the accumulated technical debt. They are snowflakes that melt under maintenability constrains.
In reality, commercial ecosystems and even Open Source language-specific ecosystems are already solving the problem of multiversioning.
Posted Dec 6, 2019 11:20 UTC (Fri)
by nim-nim (subscriber, #34454)
[Link] (7 responses)
When you attain this size you can afford the combination explosion. Most entities (starting with community distros) can not.
So yes it does not scale. With astronomic resources you can brute-force even an inefficient system.
Go modules respect my requirements. I should know, I spend enough months dissecting the module system. They will work as multi-version. Python does not. It's not a multi version language.
Java has not solved anything. Which is why its adoption outside businesses, is dismal. Businesses can afford to pay the not-scaling tax (in app servers, in ops, in lots of things induced by Java software engineering practices). Community distros can not.
>> Therefore, anything manageable at scale must keep semver exceptions as exceptions, not the general case.
Because each exception is an additional thing that needs specific handling with the associated costs. That’s engineering 101 (in software or elsewhere).
Rules get defined to bring complexity and costs down. Exceptions exist to accommodate an imperfect reality. A working efficient system allows exceptions without making them the rule.
Posted Dec 6, 2019 18:56 UTC (Fri)
by Cyberax (✭ supporter ✭, #52523)
[Link] (6 responses)
> So yes it does not scale. With astronomic resources you can brute-force even an inefficient system.
I.e. if you own a package AppA that uses LibB then you don't care about LibB's rdepends. You just use whatever version of LibB that you need. If you need a specific older or newer version of LibB then you can just maintain it for your own project, without affecting tons of other projects.
This approach scales wonderfully, compared to legacy distro packages. Heck, even Debian right now has just around 60000 source packages and is showing scaling problems. The company I worked at had many times more than that, adding new ones all the time.
> Because each exception is an additional thing that needs specific handling with the associated costs. That’s engineering 101 (in software or elsewhere).
> Exceptions exist to accommodate an imperfect reality. A working efficient system allows exceptions without making them the rule.
Posted Dec 6, 2019 19:22 UTC (Fri)
by nim-nim (subscriber, #34454)
[Link] (4 responses)
It "scales" because you do not maintain anything, you just push code blindly.
Distributions do not only push code, they fix things. When you fix things, keeping the amount of things to be fixed manageable matters.
If you don’t feel upstream code needs any fixing, then I believe you don’t need distributions at all. Just run your own Linux from scratch and be done with it.
Please report to us how much you time you saved with this approach.
Interested? I thought not. It‘s easy to claim distributions are inefficient, when adding things at the fringe. Just replace the core not the fringe if you feel that’s so easy. You won’t be the first one to try.
Posted Dec 6, 2019 19:56 UTC (Fri)
by Cyberax (✭ supporter ✭, #52523)
[Link] (3 responses)
> It "scales" because you do not maintain anything, you just push code blindly.
There was also a mechanism to deprecate versions to nudge other teams away from aging code, recommendation mechanism, etc.
> Distributions do not only push code, they fix things. When you fix things, keeping the amount of things to be fixed manageable matters.
> If you don’t feel upstream code needs any fixing, then I believe you don’t need distributions at all. Just run your own Linux from scratch and be done with it.
Posted Dec 7, 2019 8:58 UTC (Sat)
by nim-nim (subscriber, #34454)
[Link] (2 responses)
And that’s exactly what major distributions do, when the language tooling makes it possible without inventing custom layouts like Nix does.
Upstreams do not like distribution custom layouts. The backlash over Debian or Fedora relayouting Python unnilateraly, would be way worse, than the lack of parallel instability in the upstream Python default layout.
> Incorrect. In that companies libraries are maintained by their owner teams.
It’s awfully nice when you can order devs to use the specific versions maintained by a specific owner team.
Of course, most of your complaint is that you *do* not* *want* to use the specific versions maintained by the distro teams.
So, it’s not a technical problem. It’s a social problem.
Posted Dec 7, 2019 9:05 UTC (Sat)
by Cyberax (✭ supporter ✭, #52523)
[Link] (1 responses)
> when the language tooling makes it possible without inventing custom layouts like Nix does.
> Of course, most of your complaint is that you *do* not* *want* to use the specific versions maintained by the distro teams.
It's just that right now these kinds of repos are useless, because they move really slowly for a variety of reasons. The main one is the necessity to upgrade all versions in the distribution in lockstep.
Posted Dec 7, 2019 9:47 UTC (Sat)
by nim-nim (subscriber, #34454)
[Link]
> Then this tooling needs to be implemented for other languages. As I said, I've seen it done at the largest possible scale. It doesn't even require a lot of changes, really.
Then don’t complain at distros, write a PEP, make upstream python adopt a parallel-version layout.
Major community distributions will apply the decisions of major language upstreams, it’s that simple. Major community distributions collaborate with major language upstreams. Collaborating implies respecting upstream layout choices.
In a company, you can sit on upstream decisions and do whatever you want (as long as someone finds enough money to fund your fork). That’s ultimately completely inefficient and counter productive, but humans do not like to bow to the decisions of others, so, that’s been done countless times and will continue to be done countless times.
Posted Dec 7, 2019 10:00 UTC (Sat)
by nim-nim (subscriber, #34454)
[Link]
> I maintained several projects there and spent way less time on that than maintaining a package for Ubuntu and coping with it being broken by Python updates.
That "works" because you do not care about the result being useful to others. But wasn’t your original complaint, that the python3-theano maintainers didn’t care that their package was not useful to your commercial app?
So you want a system that relies, on not caring for others, to scale, to be adopted by distributions, because the distributions should care about you?
Don’t you see the logical fallacy in the argument?
Al the things that you found too much work in the Ubuntu package, exist so the result can be used by others.
Posted Dec 6, 2019 9:10 UTC (Fri)
by smurf (subscriber, #17840)
[Link] (3 responses)
Well, we all love modules with 550+ open issues and 100+ open pull requests …
So find the offending commit that broke your app and implement a workaround that satisfies both, or file an issue (and help the Theano people deal with their bug backlog, you're using their code for free, give something back!), or dropkick the writer(s) of the code that depends on 1.0.3 to get their act together. Can't be *that* difficult.
> Distros have dropped the ball here. Completely.
Pray tell us what the distros should be doing instead?
Insisting on one coherent whole, meaning one single version of everything, is the only way to manage the complexity of a distribution without going insane. Technical debt, i.e. the inability to work with 2.1 instead of 1.19 (let alone 1.0.4 instead of 1.0.3), must be paid by the party responsible. Not by everybody else.
Posted Dec 6, 2019 9:16 UTC (Fri)
by Cyberax (✭ supporter ✭, #52523)
[Link] (2 responses)
It's been done multiple times. In proprietary systems (like in the company I worked) and in Open Source (Nix OS).
Posted Dec 6, 2019 10:17 UTC (Fri)
by smurf (subscriber, #17840)
[Link] (1 responses)
The distro's job is to assemble a coherent whole, qhich occasionally requires poking the people responsible for A to support X.2. There's no incentive whatsoever for the distro to support co-installation of X.1. Yes it's been done, but that by itself is not a good argument for doing it again.
Posted Dec 6, 2019 10:26 UTC (Fri)
by farnz (subscriber, #17727)
[Link]
That works just fine in Rust - the library version is part of the symbol mangling, so as long as you don't try to use X v1 APIs on X v2 objects (or vice-versa), you're golden. Naming symbols from X v1 and X v2 in the same code is challenging and thus uncomfortable (as it should be!), but it's perfectly doable.
What doesn't work is using an object obtained from X v1 with X v2 code - the symbols are wrong, and this only works where X v2 is designed to let it work.
Posted Dec 6, 2019 13:44 UTC (Fri)
by farnz (subscriber, #17727)
[Link] (3 responses)
The issue is when AppA depends on LibB and LibC, and LibB also depends on LibC, but LibB and AppA want different versions of LibC. Someone has to do the work to ensure that LibB works with both its preferred version of LibC and with AppA's preferred version of LibC. Multiply up according to the number of versions of LibC that you end up with in a single app.
At my employer, we handle this by hunting down those cases, and getting people to fix LibB or AppA to use the same version. Rinse and repeat - someone has to do the long tail of grunt work to stop things getting out of hand.
Posted Dec 6, 2019 18:39 UTC (Fri)
by Cyberax (✭ supporter ✭, #52523)
[Link] (2 responses)
But.
This happens during development when you try to create the dependency closure. So it's the app's developer (or maintainer) who is going to be resolving the issues. And they have choices like not using a conflicting library, forking it, just overriding the LibC version for LibA or LibB, etc. Typically just forcing the version works fine.
The same thing can happen in a full distro. But then you have to actually go and fix all LibA (or LibB) rdepends before you can fix your project.
Posted Dec 6, 2019 19:39 UTC (Fri)
by farnz (subscriber, #17727)
[Link] (1 responses)
That assumes that the versioning is such that the dependency closure can't be created, and that the test cases will catch the problem in time. I'm going to use plain C libraries as an example here, as symbol versioning is weakest in C.
If, for example, AppA depends on LibC 1.3.2 or above, and LibB depends on LibC 1.3.4 or above, but LibC 1.3.4 has broken a feature of AppA in a rare corner case, you're stuck - the dependency closure succeeds during development (it chooses LibC 1.3.4), and everything appears to work. Except it doesn't, because AppA is now broken. Parallel installability doesn't help - 1.3.4 and 1.3.2 share a SONAME, and you somehow have to, at run time, link both of them into AppA and use the "right" one.
Now, if AppA has depended on both LibB and LibC for a while, you'll notice this. Where it breaks, and where the distro model helps, is when AppA has been happily chuntering along with LibC 1.3.2; LibB is imported into the distro for something else, and bumps LibC to 1.3.4, breaking AppA. The distro notices this via user reports, and helps debug and fix this. In the parallel install world, when LibB is imported into the distro, AppA continues happily working with LibC 1.3.2; when AppA is updated to use LibB, then you get the user reports about how in some timezones, AppA stops frobbing the widgets for the first half of every hour, and you have more to track down, because you have a bigger set of changes between AppA working, and AppA no longer working (including new feature work in AppA).
Posted Dec 6, 2019 20:12 UTC (Fri)
by Cyberax (✭ supporter ✭, #52523)
[Link]
But this applies equally to ANY build infrastructure. I had Debian breaking my code because OpenJDK had a bug in zlib compression that manifested only in rare cases, I had once spent several sleepless days when Ubuntu had broken SSL-related API in Python in a minor upgrade. Bugs happen.
But even in these cases having a dependency closure helps a lot. It's trivial to bisect it, comparing exactly what's different between two states, since the closure includes everything. This is not really possible with legacy package managers.
Posted Dec 5, 2019 20:00 UTC (Thu)
by pizza (subscriber, #46)
[Link] (7 responses)
Uh? On my current Fedora 31 laptop, I have both openssl 1.1.d and 1.0.2o installed, and I know both Fedora/RH and Debian have supported (via policies and tooling) this sort of thing for at least a decade or two.
Posted Dec 5, 2019 20:03 UTC (Thu)
by farnz (subscriber, #17727)
[Link] (6 responses)
It's not well-supported, though. I can't (for example) easily use distribution packages to get me both perl 5.28 and 5.30 on the same machine, or python 3.7 and 3.8. There are packages that allow it, but they're carefully curated, and not the norm.
Posted Dec 5, 2019 20:48 UTC (Thu)
by pizza (subscriber, #46)
[Link] (1 responses)
Posted Dec 5, 2019 21:03 UTC (Thu)
by Cyberax (✭ supporter ✭, #52523)
[Link]
But it's fair to say that it's not a common way to install packages in today's distros.
Posted Dec 6, 2019 8:16 UTC (Fri)
by smurf (subscriber, #17840)
[Link]
Posted Dec 8, 2019 23:29 UTC (Sun)
by flussence (guest, #85566)
[Link] (2 responses)
Posted Dec 9, 2019 8:39 UTC (Mon)
by farnz (subscriber, #17727)
[Link]
I have a system with a C extension to perl that disagrees strongly with you - perl 5.30 is different enough to perl 5.22 (that the code was written for) that the resulting application does not function correctly with perl 5.30.
Now, it's almost certainly the C extension that's buggy, but perl 5.30 has changed enough about the way it works that it has broken code written for and tested against 5.22. In my case, because perl is only used for this one application that we're rewriting anyway, it's easy enough to stick to 5.22; however, if we still developed against perl, we'd have to debug this to keep uup to date.
Posted Dec 9, 2019 8:48 UTC (Mon)
by smurf (subscriber, #17840)
[Link]
But Perl, too, has non-negotiable and backwards-incompatible deprecations.
Posted Dec 5, 2019 19:46 UTC (Thu)
by marcH (subscriber, #57642)
[Link]
Posted Dec 5, 2019 13:59 UTC (Thu)
by smurf (subscriber, #17840)
[Link] (1 responses)
Sensible dependency management is a nontrivial task. Done well, it requires that everybody responsible for any piece of code tries hard not to break its dependents *and* quickly responds if one of its dependants introduces a regression.
Distributions [need to] do this on behalf of the libraries and programs they package. Repositories like CPAN or PyPI? not so much. Collections of Go or Rust libraries? even less.
Posted Dec 5, 2019 15:17 UTC (Thu)
by nim-nim (subscriber, #34454)
[Link]
Posted Dec 4, 2019 8:45 UTC (Wed)
by marcH (subscriber, #57642)
[Link] (1 responses)
> Congratulations, you have a scaling factor of one (company).
No, because all that automation is of course open sourced and freely shared with the rest of the world.
Oh, wait...
Posted Dec 4, 2019 9:35 UTC (Wed)
by farnz (subscriber, #17727)
[Link]
Funnily enough, the majority of it is open source - Jenkins is the big chunk of code, and does the builds and tests for us. I believe you can do the same with GitHub Actions, and I've seen projects use Azure Pipelines for the same job.
Fundamentally, just building code and generating issues if builds and/or tests fail isn't hard - and we do contribute our test cases to the libraries we use.
Posted Dec 3, 2019 10:03 UTC (Tue)
by roblucid (guest, #48964)
[Link] (1 responses)
Posted Dec 3, 2019 10:09 UTC (Tue)
by farnz (subscriber, #17727)
[Link]
And I have hard experience that says that while we roll a new binary every week (target cadence is daily, but in practice things break often enough that weekly is more accurate), Debian will delay updates that are critical to us because they break something we don't care about at all - like a weird architecture not building.
And yes, we bodge workarounds and report to upstream when a tool is broken - but we also update that library for all the users that aren't broken immediately, so that most of our fleet is protected, and the broken tool doesn't stop the business continuing on. If the tool owner doesn't get a fix in place, then the tool is dropped on a schedule - maintain it or lose it. This is again in contrast to the distro model, where binaries are shipped long after the maintainer has stopped caring because no-one has yet discovered that it eitther (a) fails to build from source, or (b) has features that interact badly with security updates, but not everyone uses.
Posted Dec 2, 2019 19:37 UTC (Mon)
by roc (subscriber, #30627)
[Link] (11 responses)
> This, is often by design, because a lot of their proponents have zero notion of component QA and lifecycle management, and can’t imagine anything except “always rebuild everything against the latest 3s old internet version” or “always rebuild everything against moldy x years old locked-in-stone bundled code”.
This is a big straw-man argument. With Cargo workspaces you can have lots of Rust libraries and executables all sharing the same dependency versions (i.e. a single Cargo.lock file). And Cargo doesn't lock you into either of those extreme policies; you can easily update on whatever schedule or policy you like. Tools like cargo-audit help you update in response to security issues.
One big issue missed by proponents of depending on Linux distro packaged libraries is that as soon as a project wants to support non-Linux-distro platforms (Windows, Mac, Android, iOS) they must solve all these problems themselves by vendoring and statically linking their dependencies on those platforms (and for C/C++, they must do this with practically no tool support for vendoring). Once they're doing that, using the packaged dependencies on Linux distros is extra work that provides no benefit to the upstream project --- in fact it hurts, because inevitably you end up using different versions of dependencies on Linux compared to everywhere else, which introduces Linux-only bugs.
Posted Dec 2, 2019 20:05 UTC (Mon)
by pizza (subscriber, #46)
[Link] (9 responses)
In other words, "let's kneecap the more capable platforms until they devolve to the same level of suckage as the proprietary platforms."
Posted Dec 2, 2019 21:02 UTC (Mon)
by roc (subscriber, #30627)
[Link] (2 responses)
Posted Dec 2, 2019 21:44 UTC (Mon)
by pizza (subscriber, #46)
[Link] (1 responses)
Even putting aside library bundling, you're going to have OS/platform-specific code paths and integration hooks, plus completely independent (and usually highly brittle) build/packaging/deployment mechanisms. Solving these problems requires a non-trivial amount of ongoing effort, and there's only so much of that to go around.
At the end of the day, it's a basic cost-benefit decision that every project has to make.
Posted Dec 2, 2019 22:11 UTC (Mon)
by roc (subscriber, #30627)
[Link]
Posted Dec 2, 2019 22:36 UTC (Mon)
by rodgerd (guest, #58896)
[Link] (5 responses)
Posted Dec 2, 2019 22:50 UTC (Mon)
by pizza (subscriber, #46)
[Link] (4 responses)
Posted Dec 3, 2019 18:16 UTC (Tue)
by hkario (subscriber, #94864)
[Link] (3 responses)
Posted Dec 5, 2019 17:13 UTC (Thu)
by pizza (subscriber, #46)
[Link] (2 responses)
To this very day, Microsoft and Apple require users to be running their latest operating system in order to use their latest in-house browsers.
And while the user can always download/use a browser provided by a third party (eg Chrome or Firefox) on those non-Linux distro platforms (Windows/Mac/Android/iOS), the same goes for traditional Linux distros.
Posted Dec 12, 2019 4:26 UTC (Thu)
by Kamilion (guest, #42576)
[Link] (1 responses)
Posted Dec 12, 2019 12:08 UTC (Thu)
by pizza (subscriber, #46)
[Link]
But that's tangential to my point, which is that it was the proprietary (as opposed to F/OSS) platforms that forced OS upgrades to obtain their latest browsers -- contrary to the assertion of the person I replied to.
Posted Dec 4, 2019 7:50 UTC (Wed)
by nim-nim (subscriber, #34454)
[Link]
opt-in is the typical choice that works at the project of company level but does not scale further.
Posted Dec 3, 2019 9:47 UTC (Tue)
by roblucid (guest, #48964)
[Link]
Posted Dec 4, 2019 12:08 UTC (Wed)
by jezuch (subscriber, #52988)
[Link]
FWIW Debian does rebuild all reverse dependencies of libraries pretty much routinely (they then get uploaded by the build bot as a binnmu - binary non-maintaner upload - version). Because despite all the talk about ABI stability, C libraries break it all the time. So we're basically halfway in this brave new world already.
Posted Dec 9, 2019 11:57 UTC (Mon)
by xophos (subscriber, #75267)
[Link] (5 responses)
Posted Dec 9, 2019 14:02 UTC (Mon)
by pizza (subscriber, #46)
[Link] (4 responses)
Meanwhile, in the past 20 years on Linux (with GCC), there have been only two ABI changes:
The first was a true ABI change relating to symbol mangling, as they implemented the standard various toolchain vendors finally hashed out together.
The second was a due to incompatible API changes in the standard library (C++11 required changes to std::string and std::list). GCC allows either API(and ABI) to be targeted at compile time, and mixing the two will result in (intentional) linking errors.
Posted Dec 9, 2019 14:07 UTC (Mon)
by mathstuf (subscriber, #69389)
[Link] (3 responses)
Uh? No. Every MSVC release had incompatible standard library ABIs until 2017 and 2019 agreed on one and are able to share artifacts. I think there's one pending ABI break to fix a bunch of mistakes (including mangling `class` and `struct` decls differently), but I expect it to *then* be stable.
I think you're referring to the Win32 *API*'s ABI (which is C), not MSVC's C++ output ABI.
Posted Dec 9, 2019 14:46 UTC (Mon)
by pizza (subscriber, #46)
[Link] (1 responses)
Fair enough, but there are a lot of various ancient Windows DLLs out there that work fine with more modern tooling. I guess that means they don't use anything out of Visual C++'s standard libraries.
(And I suppose this serves as an example of why C, with all its faults, will outlast everything else..)
Posted Dec 9, 2019 15:32 UTC (Mon)
by mathstuf (subscriber, #69389)
[Link]
Well, publicly at least. There's some interesting library isolation available on Windows that ELF doesn't allow because it throws all symbols into a single pool (RTLD_LOCAL has serious issues in practice). macOS also has it through the two-level namespace lookup.
> And I suppose this serves as an example of why C, with all its faults, will outlast everything else
I've said elsewhere at least (maybe here too) before that the C ABI is certainly useful as a lowest common denominator. It's "easy" to implement and grok (at least rationales if not the implementation), is easy to make bindings to because there's no mangling, and supports most things that you need to talk to hardware and other languages.
However, that doesn't mean there isn't some room to raise that floor. Some things that are definitely missing from it:
- bitfield specification (probably "what GCC does" is the answer today, but is not actually specified anywhere in the C specification)
The first has to do with struct layout and has a de facto answer today. The next two don't affect mangling (since parameters and return types don't do so today either), but having some way to mark up a C function declaration with that information would allow Rust to have a much stronger ABI it could export. The last is implemented by C++ and just needs formalization of the shadow pointer that the function gets for the return value memory.
There's discussion of supporting value exceptions to C, but this is probably just some formalization of a tagged union. It'd be nice to have, but can be emulated with a tagged union today and I wouldn't put it on the list without also tackling the above issues first. This and the "new instance" dtor specification on a struct have C language implications as well.
Posted Dec 9, 2019 20:06 UTC (Mon)
by madscientist (subscriber, #16861)
[Link]
One hopes they will finally fix the wait_for() methods on C++ cond variables, etc. to actually work properly as required by the standard, among these mistakes :-/.
Posted Dec 2, 2019 4:42 UTC (Mon)
by mjg59 (subscriber, #23239)
[Link]
Posted Dec 4, 2019 19:36 UTC (Wed)
by flussence (guest, #85566)
[Link]
Why do we want individual system libraries mutating into node_modules-style oil spills, exactly? “Because Google gets away with it” is not a valid bar to aim for.
Posted Dec 1, 2019 3:56 UTC (Sun)
by Elv13 (subscriber, #106198)
[Link] (13 responses)
That's not really true. LTO-ed static binaries are significantly smaller than the sum of the .so. Most library users only use a small part of their codebase. With static binaries, you don't have to pay for what you don't use. This is often why embedded system go to extremes to use .a even when the libs were never tested that way. For flatpak/snaps (yes, i know you can share libs there too, but very often you wont), you actually save an order of magnitude of space.
As for the security and version pinning issues, those are, however, very concerning downsides. In both case, this is more of a package manager issue than something related to the shared vs. static library. A Gentoo Portage style (and yes, cargo) package manager could relink security updates and enforce *API* semver of static libraries. The use of shared libraries is half an DPKG/RPMimplementation details and half a political issue.
Posted Dec 1, 2019 14:23 UTC (Sun)
by HelloWorld (guest, #56129)
[Link] (6 responses)
I don't believe that's true in general. You need to specifically instruct the compiler and linker to do that. -ffunction-sections and --gc-sections and all that.
Posted Dec 1, 2019 23:06 UTC (Sun)
by Elv13 (subscriber, #106198)
[Link] (2 responses)
In theory, it isn't even "function level", an `if` in the code that is known to be dead code at compilation can just be removed. This enable more code to be inlined, unrolled and vectorized (if you choose -O3 instead of -Os, of course).
Posted Dec 4, 2019 11:39 UTC (Wed)
by jezuch (subscriber, #52988)
[Link] (1 responses)
Posted Dec 4, 2019 17:49 UTC (Wed)
by Elv13 (subscriber, #106198)
[Link]
Posted Dec 2, 2019 1:46 UTC (Mon)
by Wol (subscriber, #4433)
[Link] (2 responses)
And why doesn't it do that by default? Maybe because dynamic libraries have brainwashed it into thinking that's the right thing to do?
Change your mindset, stop believing "The Linux Way" is perfect, and maybe, just maybe, you'll realise that there is another path - that there are MANY paths - and believing in "the one true way" is not, actually, a very sensible thing to do.
Cheers,
Posted Dec 2, 2019 20:28 UTC (Mon)
by nix (subscriber, #2304)
[Link] (1 responses)
Not everything is a conspiracy. Most things like this in the toolchain are just not fully baked yet. :)
Posted Dec 2, 2019 23:08 UTC (Mon)
by Wol (subscriber, #4433)
[Link]
Cheers,
Posted Dec 1, 2019 14:24 UTC (Sun)
by Wol (subscriber, #4433)
[Link] (5 responses)
> That's not really true. LTO-ed static binaries are significantly smaller than the sum of the .so. Most library users only use a small part of their codebase. With static binaries, you don't have to pay for what you don't use. This is often why embedded system go to extremes to use .a even when the libs were never tested that way. For flatpak/snaps (yes, i know you can share libs there too, but very often you wont), you actually save an order of magnitude of space.
And saving space by not pulling in all of a megabyte library because they just want to use one routine? My experience of static libraries was that each routine was an independent blob that got copied into the executable IF REQUIRED. That gives you instant ASLR, meant that if you wanted just one routine out of a large library you got just that one routine, etc etc. Okay, if library routines called each other you had to load the library a couple of times to resolve all the references, but hey ho, that's no hardship.
That's a security benefit of itself - if one routine in the library is compromised, you can only abuse it if the program actually uses it - if it's not used it's not there!
Cheers,
Posted Dec 3, 2019 12:00 UTC (Tue)
by anton (subscriber, #25547)
[Link] (4 responses)
Your comment about "load the library a couple of times to resolve all the references" is so wrong that I wonder if I misunderstand what you mean.
Finally, if one routine in a library has a vulnerability, that vulnerability can only ever be used by an attacker if the program actually calls it, no matter how the routine was linked. And if the program calls it, it will certainly be there even with static linking.
Posted Dec 3, 2019 14:46 UTC (Tue)
by farnz (subscriber, #17727)
[Link]
There are, however, two real space savings:
Posted Dec 3, 2019 16:41 UTC (Tue)
by excors (subscriber, #95769)
[Link]
That's true, though I think unused code (whether vulnerable or not) can still be useful in exploits when there is a vulnerability in the actively-used parts of the program. A buffer overflow bug might allow a ROP attack, but the attacker needs to find suitable ROP gadgets somewhere in the process's address space. And some Spectre variants trick the CPU into speculatively executing code at attacker-controlled addresses, but that code must contain a gadget to exfiltrate data from the speculative world. If the application is statically linked and the dead code is removed, it becomes harder for an attacker to find suitable gadgets. (But in practice I guess this is only significant for small applications that use large libraries; if it's a large application then it'll probably have enough gadgets already.)
Posted Dec 3, 2019 17:14 UTC (Tue)
by Wol (subscriber, #4433)
[Link] (1 responses)
If I'm distributing a program, I have to include shared libraries. They may already be on the system, but ...
And while only the required routines may be pulled into ram, they are memory-mapped, and depending on your page size rather more may be pulled in than required.
> Static linking does not give you ASLR at all.
Do you mean the locations are randomised every time the application is run? In which case you're right, although there is nothing stopping a static loader from doing the same.
> Your comment about "load the library a couple of times to resolve all the references" is so wrong that I wonder if I misunderstand what you mean.
You may well do. Remember I'M DISCUSSING ALTERNATIVES TO LINUX. When the linker loads the master object, it creates a load of dangling references. When it loads a library to resolve those references, it *only* resolves those references, so if one library routine calls another it may create a new dangling reference - which means the library needs to be loaded again to resolve it! Don't forget - the library is a collection of independent binary blobs and only those blobs that are needed are copied into the executable.
> Finally, if one routine in a library has a vulnerability, that vulnerability can only ever be used by an attacker if the program actually calls it, no matter how the routine was linked.
So you're saying that, even though a dynamic library is memory-mapped, an attacker can't jump to an arbitrary position in that library?
Cheers,
Posted Dec 8, 2019 15:37 UTC (Sun)
by nix (subscriber, #2304)
[Link]
The problem here is that your terminology is so wrong that people who know the terminology can barely understand what you're saying. You're talking about gcc -lfoo -lbar -lfoo, right? That's not loading. That's linking. Loading is what ld.so does at runtime, and it never loads libraries more than once in the common case (modulo very obscure cases like dlmopen() and audit libraries).
Posted Dec 1, 2019 15:44 UTC (Sun)
by mcatanzaro (subscriber, #93033)
[Link] (41 responses)
If you don't personally feel that bundling libraries on this magnitude is a technical problem, that's fine. But as you are no doubt well aware, the status quo in Linux distribution communities is to minimize use of bundled dependencies as far as possible. If we are to start rewriting existing C and C++ code using Rust, this means most applications will eventually be linking to a couple dozen shared libraries written in Rust, each such library statically linked to hundreds of Rust dependencies, most of which will be duplicated among the other shared libraries used by the application. Perhaps you're fine with this -- perhaps it is the future of Linux distributions -- but for the purposes of my argument it's sufficient to recognize that a lot of important developers don't want to see this happen, and those developers are not going to be using Rust.
I think Rust needs those developers. Rust currently proposes to upend two well-established norms: (a) the use of memory unsafe languages in systems programming, and (b) current aversion to use of bundled dependencies, especially for system libraries. Challenging either norm, on its own, would be tremendously difficult. Challenging both at the same time seems like a strategic mistake. The Rust community loses essential allies, and we wind up with more and more code written in unsafe languages.
Currently, Rust seems like the only memory-safe systems programming language that's generated enough interest among Linux developers to have serious long-term potential to upend C and C++. I see potential to have a future, perhaps 30 years from now, where distributing libraries written in C or C++ is considered malpractice, and where modern code is all written in Rust (or future memory-safe systems languages). But I don't see that happening without more acceptance and buy-in from the existing Linux development community, and Rust doesn't seem to be on track to achieve it. The Rust community and the Linux distribution community are separate silos with separate goals, and it's hard to see outside one's own silo. If the Rust community were to better understand and address the concerns of the Linux distribution community, the potential exists to significantly increase adoption and acceptance of Rust.
Posted Dec 1, 2019 20:38 UTC (Sun)
by roc (subscriber, #30627)
[Link]
If you specifically mean "Rust needs those developers if it's to completely usurp C/C++", then sure. Maybe enough Rust people are inspired by that to make some concessions.
Posted Dec 1, 2019 20:57 UTC (Sun)
by mjg59 (subscriber, #23239)
[Link] (21 responses)
Posted Dec 1, 2019 22:40 UTC (Sun)
by rodgerd (guest, #58896)
[Link]
Posted Dec 1, 2019 23:57 UTC (Sun)
by mcatanzaro (subscriber, #93033)
[Link] (19 responses)
Let's say Rust is currently poised to be successful in the user application space. It's not poised to be successful at replacing the C and C++ used to power our operating systems. And that's a shame, because we have a lot of unsafe C and C++ code that needs to be replaced over the coming decades.
Posted Dec 2, 2019 0:06 UTC (Mon)
by mjg59 (subscriber, #23239)
[Link] (18 responses)
Posted Dec 2, 2019 0:15 UTC (Mon)
by mcatanzaro (subscriber, #93033)
[Link] (17 responses)
Unless your application code never provides any untrusted input to the runtime -- which seems very unlikely -- the runtime is security-sensitive too, and it is all C and C++. 40 years from now, this will hopefully no longer be the case. We'll still have distributions, but they will use a safe systems programming language instead. It could be Rust, but it seems Rust does not want to target this space, so I suppose more likely it will be something else instead.
Posted Dec 2, 2019 0:21 UTC (Mon)
by mcatanzaro (subscriber, #93033)
[Link]
Posted Dec 2, 2019 0:30 UTC (Mon)
by mjg59 (subscriber, #23239)
[Link] (15 responses)
Basically: If nobody cares about the shared libraries a distribution ships, why should a distribution care?
Posted Dec 2, 2019 9:57 UTC (Mon)
by nim-nim (subscriber, #34454)
[Link] (14 responses)
The user-side part of the equation has been learning what containered apps mean in the last years. It's not impressed. They’re a lot worse than downloading random exe files from random web sites on the internet (worse because containers permit extending the amount of crap third party code in the end result).
Posted Dec 2, 2019 12:00 UTC (Mon)
by pizza (subscriber, #46)
[Link] (12 responses)
This is a very naive belief.
Look no further than the rise of "wget http://some/random/script.sh | sudo bash" build scripts. Also, proprietary software is far, far worse, and manages to do just fine in the market.
Posted Dec 2, 2019 14:16 UTC (Mon)
by NAR (subscriber, #1313)
[Link] (10 responses)
And this is the problem with the distribution landscape. Most developers won't bother learning the intricacies of creating packages for N distributions (especially if they are working on e.g. Mac). There's also a quite big chance that at least one of their dependencies are not packaged, so either they package those too (even more work) or bundle the dependencies (which is a big no-no in the eyes of distribution people). But I'm starting to see "Installation Instructions" like "docker pull ...", so even the "wget ... | bash" kind of instructions can be obsoleted. If distributions want to stay relevant, they might need to invent some automatic mechanism that takes a package from npmjs.com, pypi.org, cpan.org, hex.pm, etc. and creates parallel-installable distribution packages.
Posted Dec 2, 2019 17:03 UTC (Mon)
by mcatanzaro (subscriber, #93033)
[Link] (9 responses)
Expecting third-party software developers to package software for Linux distributions doesn't make a lot of sense either. They can if they want to, targeting the biggest and most important distros, but surely most will probably prefer to distribute containerized software that works everywhere using docker or flatpak or similar. Nothing wrong with that. It doesn't mean distros are no longer relevant, it just means there are nowadays better ways to distribute applications to users. Your application is still 100% useless if the user doesn't have a distro to run it on.
I see no problem with "the distribution landscape." It works well for building the core OS and for distributing popular open source applications. It was never a good way for distributing third-party software, and that's fine.
Posted Dec 2, 2019 18:06 UTC (Mon)
by rahulsundaram (subscriber, #21946)
[Link]
If distributions are reduced to only the core bits and everything else is managed by per language package management systems, distributions have far less relevance than they used to have and therefore their packaging policies don't have as much influence as it used to as well. This may very well be the better role for distributions but historically distributions have attempted to package up the whole world of open source software and pretty much the only way regular users would get the software installed on their systems. This isn't the case any longer
Posted Dec 2, 2019 18:28 UTC (Mon)
by mjg59 (subscriber, #23239)
[Link] (7 responses)
Posted Dec 2, 2019 18:41 UTC (Mon)
by pizza (subscriber, #46)
[Link] (4 responses)
(Or will they just download some customizable container template containing some precompiled libraries from a third party?)
Posted Dec 2, 2019 18:48 UTC (Mon)
by mjg59 (subscriber, #23239)
[Link] (1 responses)
Posted Dec 4, 2019 8:08 UTC (Wed)
by nim-nim (subscriber, #34454)
[Link]
In the meanwhile, real-world containers in Docker… etc public stores have been publicly audited by security researchers and those researches found out first, that those containers did rely on distributions for their content and second, that the more they tried to replace the distribution layer with custom dev-friendly ways to do things, the less up to date and secure the result ended up.
Things that may work technically are large out-of band container content inspection by Microsoft (GitHub) Google (Go), where Microsoft or Google or another internet giant orders devs to fix their open source code if they want to continue being listed in the audited container store.
I doubt devs will love this ordering a lot more than being told politely by distributions to fix things because they are becoming un-packageable.
And, that’s nothing more than a capture of free software commons by commercial entities, taking advantage of the lack of nurturing of those commons by the people who benefit from them today.
Posted Dec 5, 2019 6:37 UTC (Thu)
by sionescu (subscriber, #59410)
[Link] (1 responses)
Posted Dec 5, 2019 21:31 UTC (Thu)
by nim-nim (subscriber, #34454)
[Link]
So, good model for a cloud giant.
Atrocious model for everyone *not* a cloud giant.
Posted Dec 2, 2019 21:47 UTC (Mon)
by mcatanzaro (subscriber, #93033)
[Link]
Of course, the shared libraries that distributions need to ship is 90% of what I care about. So you lose me, and other developers working in this space. We wind up with "Rust is cool for application development, but don't use it for systems programming." If that's the goal, then OK, but I suspect that's not really the goal of Rust.
Posted Dec 3, 2019 20:10 UTC (Tue)
by Wol (subscriber, #4433)
[Link]
And that imho is exactly the attitude that led to the failure of the LSB :-(
It was focussed too much on allowing the distros to tell applications what they provided - which is sort-of okay for Open Source applications, but I tried to push it (unsuccessfully) towards a way where apps (quite possibly closed-source) could tell the distro what they needed. I really ought to try to get WordPerfect 6 running under Wine again, but it would have been nice for WordPerfecrt for Linux 8 to have an lsb "requires" file I could pass to emerge, or rpm, or whatever, and it would provide the pre-requisites for me.
Oh well ...
Cheers,
Posted Dec 2, 2019 15:27 UTC (Mon)
by nim-nim (subscriber, #34454)
[Link]
Proprietary software is not intrinsically worse. The level of atrocity is limited by the commercial requirement to support the result.
Container-ish distribution and vendoring are proprietary software habits at their worst, without any support requirement.
Posted Dec 4, 2019 7:13 UTC (Wed)
by patrakov (subscriber, #97174)
[Link]
Posted Dec 2, 2019 0:05 UTC (Mon)
by rahulsundaram (subscriber, #21946)
[Link] (12 responses)
Rust has no real incentive to play by distribution rules. Firefox's usage of Rust and subsequent increasingly widespread adoption will not be hampered by distributions
Distributions don't have the strong influence they used to as guiding upstreams towards certain behaviour. Upstream projects like Rust will simply bypass distributions aided by widespread use of containers and tooling like Cargo. Distributions are now in the defensive, figuring out how to stay relevant. That's the new reality.
Posted Dec 2, 2019 3:29 UTC (Mon)
by pizza (subscriber, #46)
[Link] (11 responses)
And what happens when this brave new reality encounters its inevitable "zlib" moment?
The Rust ecosystem has some very serious tunnel vision -- Well-maintained open-source project (ala Firefox) are the exception, not the rule.
It's all fine and dandy to say that distributions are irrelevant or obsolete, but that doesn't make their experience and advice on sustainability/maintainability wrong.
Posted Dec 2, 2019 3:44 UTC (Mon)
by rahulsundaram (subscriber, #21946)
[Link] (7 responses)
Maybe but it is clear that power centre has shifted away from distributions (and I say that as someone who has involved in distribution maintenance for atleast a decade). It is up to distributions to convince the language ecosystems why the experience is valid or useful
Posted Dec 2, 2019 10:11 UTC (Mon)
by nim-nim (subscriber, #34454)
[Link] (6 responses)
So either those devs work with free software distributions to provide this stability, or the market for their app segment will be captured by non-free-software third parties, that perform this distribution role.
This is happening right now with Cloudera, OpenShift, etc.
The end result of not working with distributions, and producing things end users can not use with any level of security, is not users adopting the dev model, it’s users giving up on distributions *and* upstream binaries, and switching to proprietarized intermediated product sources.
And, practically, that also means devs having to follow the whims of the proprietary intermediaries if they want any influence on how their code is actually used. Do you really think they will love better? Even if the proprietary intermediaries provide them with free-as-beer SDKs?
Posted Dec 2, 2019 11:05 UTC (Mon)
by NAR (subscriber, #1313)
[Link] (2 responses)
Actually providing stability is a reason to bypass distributions. It's more than annoying when installing/upgrading an unrelated application upgrades a common dependency too with an incompatible new version...
Posted Dec 3, 2019 10:56 UTC (Tue)
by roblucid (guest, #48964)
[Link]
Library updates should support the API, incompatible API changes require a new package, which may provide a legacy API or support co-existence with an installation of the old library by changing filenames or directories. Shared libraries actually do permit applications choosing different implementations if required.
Rather than 'just in case' silos, fix the bugs and write competent software. Bad security fixes breaking stuff are bugs, regressions which ought be fixed and the sysadmin is the only one who can decide the right mitigation in the deployment ..
The ability to secretly rely on vulnerabilities IS NOT a benefit to the end user
Posted Dec 3, 2019 13:52 UTC (Tue)
by farnz (subscriber, #17727)
[Link]
To expand on that, there are three groups involved here, not two:
Distributions that survive end up being a compromise between the three. They update fast enough to keep developers happy; they're good enough at stopping things that you don't want to have happen to keep operations happy; they make the computer do enough useful things that users are happy. But note that distributions are not essential in this set - they just happen to be one form of compromise between the three groups that has historically worked out well enough to survive. Containers are another - especially combined with CI - where you build a complete FS image of the "application" and run it; you regularly update that image, and all is good.
Basically, things go wrong when the desires of the three groups are unbalanced - if a distribution becomes "this is what operations want", then users and developers leave it to ossify; if a distribution becomes "this is what users want", it loses developers and operations; if it becomes "this is what developers want", it loses users and operations. And as every real individual is a combination of users, developers and operations to various degrees, the result of such ossification is a gradual loss of people to work on the distribution.
As soon as distributions see themselves as "in opposition" to developers (or operations, or users), they're at risk - this is about finding a balance between developers' love of using the best technologies they know about, and operations' love of not bringing in tech for the sake of it that results in users getting value from the distribution.
Posted Dec 2, 2019 14:48 UTC (Mon)
by walters (subscriber, #7396)
[Link] (2 responses)
I know the meaning of the individual words in your message, but the way you've combined them isn't making much sense to me... ("proprietarized intermediated product sources"? Really?)
Posted Dec 4, 2019 8:57 UTC (Wed)
by nim-nim (subscriber, #34454)
[Link] (1 responses)
The difference between effective open sourcing and over the wall code dumping others can not use is the existence of things like Oracle Linux.
I haven’t seen the equivalent OpenShift-side but I may not have looked hard enough.
Posted Dec 4, 2019 9:47 UTC (Wed)
by zdzichu (subscriber, #17118)
[Link]
Posted Dec 2, 2019 22:42 UTC (Mon)
by rodgerd (guest, #58896)
[Link] (2 responses)
Posted Dec 2, 2019 22:52 UTC (Mon)
by pizza (subscriber, #46)
[Link]
Posted Dec 3, 2019 6:36 UTC (Tue)
by tzafrir (subscriber, #11501)
[Link]
So how will such a case work with bundled libraries? Nobody fixes problematic libraries down the stack? Developers do occasionally keep patched versions of such libraries? But then it is each developer on their own, right?
The same problem does not go away with bundled libraries. It may even get worst, because each developer / project is on its own tackling those issues.
Posted Dec 2, 2019 9:40 UTC (Mon)
by joib (subscriber, #8541)
[Link] (4 responses)
The current distro model is *a* solution (certainly not the only possible solution, and not necessarily the "best" one either, depending on how you define "best") to the problem of how to handle distribution of mostly C code, and largely set in stone 25 years ago.
If one looks at the where the world is going, it seems that more and more we're seeing applications bundling their own dependencies, be it in the form of container images, language specific build/dependency management systems with static linking like Rust or Go, or per-application virtual environments for dynamic languages like Python/Ruby/Javascript/Julia/etc. And things like app stores where applications can rely on a fairly limited system interface, bundling everything else.
I think the onus is on distros to adapt to the world as it is, not wishing for a bygone age. Otherwise they'll just be relegated to the position of an increasingly idiosyncratic way of providing the basic low-level OS while people get the actual applications they want to run from somewhere else. I also think that distros can have a very useful role to play here, e.g. keeping track of security updates for all these apps with bundled dependencies etc.
Posted Dec 2, 2019 15:33 UTC (Mon)
by nim-nim (subscriber, #34454)
[Link] (3 responses)
Posted Dec 2, 2019 17:18 UTC (Mon)
by joib (subscriber, #8541)
[Link] (2 responses)
Providing a single coherent interface to manage packages, QA, integration testing, vetting of packages, following the distro policy, security updates (as previously mentioned) are certainly valuable things that distros do. I'd like to see distros continuing this valuable work, rather than being delegated to an increasingly irrelevant provider of the low level OS. And to do this, I think distros need to better accommodate how software is developed and deployed today, rather than wishing for a bygone era where gcc, make and shell was all that was needed.
Posted Dec 5, 2019 14:46 UTC (Thu)
by nim-nim (subscriber, #34454)
[Link] (1 responses)
Distributions, however, have minimal standards to retain the trust of their userbase.
Anything that needs too much work, to attain those standards, will get kicked outside distros, just because no one wants to work on it.
That’s why alternatives to gcc, make and shell get not traction. dev-only tech, that only helps devs, and makes everyone except devs miserable, won’t be adopted by anyone except devs. If those devs want their stuff to get into distributions, they can do the work themselves, or make it easier for others to do this work.
If those devs don’t want to do the work, and don't want to help others do it, they can stop moaning distributions are unfriendly, and prepare for world where the thing they deploy on is controlled by Apple, Google or Microsoft, and they need to construct all the parts over this proprietary baseline in Linux From Scratch mode, depending on proprietary cloud services.
Devs can certainly kill distros. So far, they haven’t demonstrated any working plan once this is done.
Posted Dec 5, 2019 15:11 UTC (Thu)
by pizza (subscriber, #46)
[Link]
That's a key point -- Traditional distributions do a lot of work integrating a large pile of unrelated code into a mostly-cohesive (and usable, and maintainable) whole. Application (and container) developers rely heavily on that work, allowing themselves to focus on the stuff they care about (ie their application)
If those distributions go away, that low-level systems development and integration work still needs to get done by _someone_, and to be blunt, application developers have shown themselves to be, even when willing, to be generally incapable of doing this work.
Oddly enough, the developers that are both willing and capable seem to recognize the value (and time/effort savings) that distributions bring to the table -- because without the distros, the developers would have a lot more work on their hands.
Posted Dec 1, 2019 16:38 UTC (Sun)
by nim-nim (subscriber, #34454)
[Link] (1 responses)
The top-level code (own project code) is usually clean. But, it’s standing on the shoulders of obsolete bug-ridded garbage no one looks at.
Devs love this system because it does not force them to look at the updates of the stuff they depend on (and, they hate changes done by others). The end result is entirely predictable from a human POW. It’s classical no-one-looks-at-it technical debt accumulation common in proprietary projects.
Vendoring, is the solution invented by people who did not want to do free software and open source in the first place, to pretend doing it, now the market forces them to, while avoiding all the community interactions built-in traditional free software workflows.
Posted Dec 3, 2019 10:34 UTC (Tue)
by roblucid (guest, #48964)
[Link]
Posted Dec 1, 2019 5:24 UTC (Sun)
by roc (subscriber, #30627)
[Link] (1 responses)
Unfortunately it does seem pretty clear that a halfway approach --- where each application depends on lots of shared libraries, each of which statically links a lot of dependencies, so there's lots of code duplication even within a single process --- would be really bad.
Posted Dec 1, 2019 20:32 UTC (Sun)
by roc (subscriber, #30627)
[Link]
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
> been updated due to a security vulnerability (and the update is possible per the Cargo specification) is somewhat
> straightforward and amenable to automation by distros (and the equivalent for Go and other languages that build
> statically).
> doesn't seem like showstopper problems.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Speed is not the primary concern of most users.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
- Ship it!
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
2. Not very convenient for the main topic to be implicit. Might explain some of the length and confusion.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
1. the build system asks cargo the lists of crates+minimum semver needed for a build,
2 the answer is translated into distribution package dependencies,
3 the build system installs those dependencies,
4. each dependency installs the corresponding crate code + the system workspace index part that tells cargo to use this crate and nothing else
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
[workspace]
members = [
"/path/to/system/crates/*",
"*"
]
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
/usr/share/fonts
~/.local/share/fonts/
or XDG_DATA_HOME/fonts if you like non-standard layouts
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
3. you have a command, that can output the artefact ids and versions, corresponding to a filename (fc-query)
(mostly, adding a namepace like font(), if upstream naming forgets it will be injected in a dependency graph, that includes many different kinds or artefacts)
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
crate(abomonation/default) = 0.7.3
crate(actix) = 0.8.3
crate(actix-codec) = 0.1.2
crate(actix-codec/default) = 0.1.2
crate(actix-connect) = 0.2.3
crate(actix-connect/default) = 0.2.3
crate(actix-connect/http) = 0.2.3
crate(actix-connect/openssl) = 0.2.3
crate(actix-connect/rust-tls) = 0.2.3
…
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
- elsif PyPI/NPM/cargo/k8s/other [finally] gets a software distribution clue (https://lwn.net/Articles/806230/ k8s), then distros should probably stop duplicating the effort and getting in the way. As you wrote, devs are more specialised and know their particular technology better. They just need to realise at last that writing product code is a tiny part of software.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Not quite following. If you update a dependency then you start walking its rdepends graph and updating it. There's no expectation that AppA would work with a random version of LibB.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Yes, there is.
No. You're cherry-picking.
No. No. No. You are about as wrong as you can get.
No. Distros have dropped the ball here. Completely. They are stuck in the "single version or GTFO" model that is not scaling past the basic system.
Soller: Real hardware breakthroughs, and focusing on rustc
* first, the language deployment format uses different file paths of all the version sublevels you want to exist
* second, there was a way at the language level, to point to a specific version, if several are found on disk.
Soller: Real hardware breakthroughs, and focusing on rustc
I can not use distor infrastructure for it. I have to build my own packages and manage all of them.
> * second, there was a way at the language level, to point to a specific version, if several are found on disk.
If we're stuck on Python then we can as well continue. Python supports all of these, yet no public distro utilizes it. Some unpopular distros like Nix do.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
One application is a commercial simulator that we can't fix. We'll probably just update our application to work with 1.0.3 by adding a workaround or just put it in a container. This workload should have been containerized anyway...
No it's not. Python supports venvs, custom PYTHONPATH and custom loaders.
Correct. This is a problem, but it happens during development, not package installation so the developer can work around it (by fixing deps, forking them, pinning previous versions, etc.)
Soller: Real hardware breakthroughs, and focusing on rustc
> No it's not. Python supports venvs, custom PYTHONPATH and custom loaders.
– venv is poor man's containerization (you can achieve the same with full distro containers, if venv “works” then distros work too)
– PYTHONPATH is the same technical debt engine which has made Java software unmaintainable¹
– custom loaders are well, custom. they're not a generic language solution
2. defines a *single* default version selection mecanism that handles upgrading and selects the best and most recent version by default (ie semver)
3. defines a way to declare, an application-level exception for a specific set of components
– the “backwards” C/C++, because of shared lib versionning
– Go, because Google wanted to create an internet-wide Go module repository, so they had to tackle the version collision problem
Probably others.
Soller: Real hardware breakthroughs, and focusing on rustc
I worked in a company that is in the top 10 of the world's companies and has the name starting with an "A" and the second letter not being "p".
Nothing of this is needed. Nothing at all. Please, do look at how Java works with Maven, Ruby with Gems, Go with modules, or Rust with Cargo. They solved the problem of multiversioning in the language tooling without your requirements.
I don't follow. Why?
So far this is basically your fantasy. You are thinking that only seasoned distros that can wrangle the dependency graph into one straight line can save the world.
Soller: Real hardware breakthroughs, and focusing on rustc
> I worked in a company that is in the top 10 of the world's companies and has the name starting with an "A" and the second letter not being "p".
> I don't follow. Why?
Soller: Real hardware breakthroughs, and focusing on rustc
I maintained several projects there and spent way less time on that than maintaining a package for Ubuntu and coping with it being broken by Python updates.
There's nothing inefficient there. It allows to efficiently place the onus of evolving the dependencies on the correct people - package owners.
What is "semver exceptions" then?
What are "exceptions" in the system I'm describing?
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Correct.
Incorrect. In that companies libraries are maintained by their owner teams. The difference is that they typically maintain a handful of versions at the same time, so that all their dependants can build it.
In my experience, they mostly break things by updating stuff willy-nilly without considering downstream developers.
That's pretty much what we'll be doing eventually. The plan is to move everything to containers running on CoreOS (not quite LFS, but close enough).
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Except that it's impossible to use it, because it's not possible to parallel install libraries.
Then this tooling needs to be implemented for other languages. As I said, I've seen it done at the largest possible scale. It doesn't even require a lot of changes, really.
Incorrect again. I would love to see a distro-maintained repository with vetted package versions, with changes that are code-reviewed by distro maintainers.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Parallelly installable versions of libraries. With a mechanism allowing to create a dependency closure for each application with specific library versions.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
When a security fix breaks a tool, that's when you bodge a workround and report the regression to upstream.
As is once your company's attention is on another priority, the extra work created for fear of rare 'perhaps' events won't be done.
Binary forking almost every library is brain dead
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
At times too, useful new implementation features have been supportable without altering applications.
Bundling and customising creates inertia and long term stagnation.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
- RAII support (some way to tag a decl as saying "returns a new instance" where the struct decl says what to do to release it)
- marking parameters as "moved" (to elide unnecessary RAII destruction and the C++ zombie-moved state)
- RVO (caller-allocated memory for the return value) would be nice to have as well
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Wol
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Wol
Soller: Real hardware breakthroughs, and focusing on rustc
Wol
Soller: Real hardware breakthroughs, and focusing on rustc
And saving space by not pulling in all of a megabyte library because they just want to use one routine?
Do you mean pulling into the executable file, or pulling into RAM? In either case, what you imply does not happen:
The rest of your claim are off-mark, too. Static linking does not give you ASLR at all. If you disable ASLR in the kernel, a statically-linked binary always gets loaded to the same position and is therefore no better than with dynamic linking; conversely, if you do not disable ASLR, all the code (including the shared libraries) is ASLRed.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Wol
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Look no further than the rise of "wget http://some/random/script.sh | sudo bash" build scripts.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Yes, eventually everyone should compile their own stuff. Most task containers on Borg contain exactly one statically-linked binary in addition to a very thin runtime mounted read-only from the host.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Wol
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Distributions that provide piles of half-baked, half-checked, half-fixed third party code in the form of never-updates shared libraries and other packages will be hard pressed to find a market, among anyone who values not losing his data.
</troll>
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
I think devs are so drunk on their new tooling ability to bypass free software distributions, they forget users do require a level of stability.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
In practice, CentOS built images once and did not provide any updates for them.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc
They might also like the way, fossilising the library by bundling prevents the software working when implementations are upgraded yet provide a compat API for unaltered applications which don't depend on enhanced features.
A cynic might deem it planned obsolescence.
Soller: Real hardware breakthroughs, and focusing on rustc
Soller: Real hardware breakthroughs, and focusing on rustc