Soller: Real hardware breakthroughs, and focusing on rustc

Posted Dec 2, 2019 14:16 UTC (Mon) by NAR (subscriber, #1313)
In reply to: Soller: Real hardware breakthroughs, and focusing on rustc by pizza
Parent article: Soller: Real hardware breakthroughs, and focusing on rustc

Look no further than the rise of "wget http://some/random/script.sh | sudo bash" build scripts.

And this is the problem with the distribution landscape. Most developers won't bother learning the intricacies of creating packages for N distributions (especially if they are working on e.g. Mac). There's also a quite big chance that at least one of their dependencies are not packaged, so either they package those too (even more work) or bundle the dependencies (which is a big no-no in the eyes of distribution people). But I'm starting to see "Installation Instructions" like "docker pull ...", so even the "wget ... | bash" kind of instructions can be obsoleted. If distributions want to stay relevant, they might need to invent some automatic mechanism that takes a package from npmjs.com, pypi.org, cpan.org, hex.pm, etc. and creates parallel-installable distribution packages.

Soller: Real hardware breakthroughs, and focusing on rustc

Posted Dec 2, 2019 17:03 UTC (Mon) by mcatanzaro (subscriber, #93033) [Link] (9 responses)

"if distributions want to stay relevant" just doesn't make any sense, and never did. How are you going to use npmjs.com, pypi.org, cpan.org, or hex.pm without a computer operating system to access it from? If not a Linux distribution, what else?

Expecting third-party software developers to package software for Linux distributions doesn't make a lot of sense either. They can if they want to, targeting the biggest and most important distros, but surely most will probably prefer to distribute containerized software that works everywhere using docker or flatpak or similar. Nothing wrong with that. It doesn't mean distros are no longer relevant, it just means there are nowadays better ways to distribute applications to users. Your application is still 100% useless if the user doesn't have a distro to run it on.

I see no problem with "the distribution landscape." It works well for building the core OS and for distributing popular open source applications. It was never a good way for distributing third-party software, and that's fine.

Soller: Real hardware breakthroughs, and focusing on rustc

Posted Dec 2, 2019 18:06 UTC (Mon) by rahulsundaram (subscriber, #21946) [Link]

>"if distributions want to stay relevant" just doesn't make any sense, and never did. How are you going to use npmjs.com, pypi.org, cpan.org, or hex.pm without a computer operating system to access it from? If not a Linux distribution, what else?

If distributions are reduced to only the core bits and everything else is managed by per language package management systems, distributions have far less relevance than they used to have and therefore their packaging policies don't have as much influence as it used to as well. This may very well be the better role for distributions but historically distributions have attempted to package up the whole world of open source software and pretty much the only way regular users would get the software installed on their systems. This isn't the case any longer

Soller: Real hardware breakthroughs, and focusing on rustc

Posted Dec 2, 2019 18:28 UTC (Mon) by mjg59 (subscriber, #23239) [Link] (7 responses)

If all a distribution is providing is the infrastructure to run containers, then the number of shared libraries that the distribution needs to ship is pretty small. Why prioritise that case over the far more common case?

Soller: Real hardware breakthroughs, and focusing on rustc

Posted Dec 2, 2019 18:41 UTC (Mon) by pizza (subscriber, #46) [Link] (4 responses)

...and what exactly is supposed to go into those containers? Will everyone be compiling every dependency themselves from scratch?

(Or will they just download some customizable container template containing some precompiled libraries from a third party?)

Soller: Real hardware breakthroughs, and focusing on rustc

Posted Dec 2, 2019 18:48 UTC (Mon) by mjg59 (subscriber, #23239) [Link] (1 responses)

Someone still has to provide shared objects that existing code depends on somehow, but that's pretty orthogonal to why distributions have traditionally preferred shared objects - the benefits of them are already gone if everybody's including copies of them in containers.

Soller: Real hardware breakthroughs, and focusing on rustc

Posted Dec 4, 2019 8:08 UTC (Wed) by nim-nim (subscriber, #34454) [Link]

That’s a pie in the sky argument. You don’t want to depend on distributions so you assume someone else will pick up the flag (and, moreover, you assume that that someone else, when confronted with the same issues distributions face today, will make different choices, because we all know distribution choices are bad, say the devs that have never tried integrate at scale)

In the meanwhile, real-world containers in Docker… etc public stores have been publicly audited by security researchers and those researches found out first, that those containers did rely on distributions for their content and second, that the more they tried to replace the distribution layer with custom dev-friendly ways to do things, the less up to date and secure the result ended up.

Things that may work technically are large out-of band container content inspection by Microsoft (GitHub) Google (Go), where Microsoft or Google or another internet giant orders devs to fix their open source code if they want to continue being listed in the audited container store.

I doubt devs will love this ordering a lot more than being told politely by distributions to fix things because they are becoming un-packageable.

And, that’s nothing more than a capture of free software commons by commercial entities, taking advantage of the lack of nurturing of those commons by the people who benefit from them today.

Soller: Real hardware breakthroughs, and focusing on rustc

Posted Dec 5, 2019 6:37 UTC (Thu) by sionescu (subscriber, #59410) [Link] (1 responses)

Yes, eventually everyone should compile their own stuff. Most task containers on Borg contain exactly one statically-linked binary in addition to a very thin runtime mounted read-only from the host.

Soller: Real hardware breakthroughs, and focusing on rustc

Posted Dec 5, 2019 21:31 UTC (Thu) by nim-nim (subscriber, #34454) [Link]

And Google manages that because they have a *centralized* organization with a *single* *unified* SCM tree.

So, good model for a cloud giant.

Atrocious model for everyone *not* a cloud giant.

Soller: Real hardware breakthroughs, and focusing on rustc

Posted Dec 2, 2019 21:47 UTC (Mon) by mcatanzaro (subscriber, #93033) [Link]

I suppose that's a fine point. Every distribution still needs a fairly large number of shared libraries, many of which are security-critical, but if you don't care about them then it's reasonable to not prioritize them.

Of course, the shared libraries that distributions need to ship is 90% of what I care about. So you lose me, and other developers working in this space. We wind up with "Rust is cool for application development, but don't use it for systems programming." If that's the goal, then OK, but I suspect that's not really the goal of Rust.

Soller: Real hardware breakthroughs, and focusing on rustc

Posted Dec 3, 2019 20:10 UTC (Tue) by Wol (subscriber, #4433) [Link]

> If all a distribution is providing is the infrastructure to run containers, then the number of shared libraries that the distribution needs to ship is pretty small. Why prioritise that case over the far more common case?

And that imho is exactly the attitude that led to the failure of the LSB :-(

It was focussed too much on allowing the distros to tell applications what they provided - which is sort-of okay for Open Source applications, but I tried to push it (unsuccessfully) towards a way where apps (quite possibly closed-source) could tell the distro what they needed. I really ought to try to get WordPerfect 6 running under Wine again, but it would have been nice for WordPerfecrt for Linux 8 to have an lsb "requires" file I could pass to emerge, or rpm, or whatever, and it would provide the pre-requisites for me.

Oh well ...

Cheers,
Wol