LWN: Comments on "Soller: Real hardware breakthroughs, and focusing on rustc"

Soller: Real hardware breakthroughs, and focusing on rustc

pizza — Thu, 12 Dec 2019 12:08:04 +0000

Yes, as of mid-2019 and four years of latest-MS-platform exclusivity, Edge is now just a wrapper around Chromium.

But that's tangential to my point, which is that it was the proprietary (as opposed to F/OSS) platforms that forced OS upgrades to obtain their latest browsers -- contrary to the assertion of the person I replied to.

Soller: Real hardware breakthroughs, and focusing on rustc

Kamilion — Thu, 12 Dec 2019 04:26:07 +0000

I seem to be running Microsoft Edge 79.0.309.47 (Official build) (64-bit) on Windows 7, contrary to your assertion.

Soller: Real hardware breakthroughs, and focusing on rustc

jezuch — Tue, 10 Dec 2019 15:14:55 +0000

Yes, out-of-bounds access to a span is always a bug, but if you care about memory safety, you can't just ignore it completely. For example, Java's approach is to mandate the bounds check at the semantic level, and elide the checks at run time if the JVM can prove that the out-of-bounds access is impossible. I think Rust's approach is to panic (but only in debug builds?) but prefer APIs that are safe even without range checks. C++'s approach is not to care about memory safety :) Well, OK, C++'s approach is to declare it UB and tell everyone to deal with it (and cause many headaches).

Soller: Real hardware breakthroughs, and focusing on rustc

ghane — Tue, 10 Dec 2019 14:12:49 +0000

> Code points 0x11 through 0x14
> DC1 and DC3 as XON and XOFF

... which map onto ^Q and ^S respectively.

Soller: Real hardware breakthroughs, and focusing on rustc

jezuch — Tue, 10 Dec 2019 12:41:42 +0000

I'm a little late to the party and it's already late for what I'm proposing, but the Rust project has just finished collecting proposals for things to focus on in 2020. This year, for example, the focus was things like ergonomics (IIRC). Making Rust work well with distributions would be a valid goal to focus on for the project, IMVHO. So the best place to rant about this would be #rust2020 :)

Just my 2¢

Soller: Real hardware breakthroughs, and focusing on rustc

madscientist — Mon, 09 Dec 2019 20:06:50 +0000

> I think there's one pending ABI break to fix a bunch of mistakes

One hopes they will finally fix the wait_for() methods on C++ cond variables, etc. to actually work properly as required by the standard, among these mistakes :-/.

Soller: Real hardware breakthroughs, and focusing on rustc

mathstuf — Mon, 09 Dec 2019 15:32:13 +0000

> I guess that means they don't use anything out of Visual C++'s standard libraries.

Well, publicly at least. There's some interesting library isolation available on Windows that ELF doesn't allow because it throws all symbols into a single pool (RTLD_LOCAL has serious issues in practice). macOS also has it through the two-level namespace lookup.

> And I suppose this serves as an example of why C, with all its faults, will outlast everything else

I've said elsewhere at least (maybe here too) before that the C ABI is certainly useful as a lowest common denominator. It's "easy" to implement and grok (at least rationales if not the implementation), is easy to make bindings to because there's no mangling, and supports most things that you need to talk to hardware and other languages.

However, that doesn't mean there isn't some room to raise that floor. Some things that are definitely missing from it:

- bitfield specification (probably "what GCC does" is the answer today, but is not actually specified anywhere in the C specification)
- RAII support (some way to tag a decl as saying "returns a new instance" where the struct decl says what to do to release it)
- marking parameters as "moved" (to elide unnecessary RAII destruction and the C++ zombie-moved state)
- RVO (caller-allocated memory for the return value) would be nice to have as well

The first has to do with struct layout and has a de facto answer today. The next two don't affect mangling (since parameters and return types don't do so today either), but having some way to mark up a C function declaration with that information would allow Rust to have a much stronger ABI it could export. The last is implemented by C++ and just needs formalization of the shadow pointer that the function gets for the return value memory.

There's discussion of supporting value exceptions to C, but this is probably just some formalization of a tagged union. It'd be nice to have, but can be emulated with a tagged union today and I wouldn't put it on the list without also tackling the above issues first. This and the "new instance" dtor specification on a struct have C language implications as well.

Soller: Real hardware breakthroughs, and focusing on rustc

pizza — Mon, 09 Dec 2019 14:46:05 +0000

> I think you're referring to the Win32 *API*'s ABI (which is C), not MSVC's C++ output ABI.

Fair enough, but there are a lot of various ancient Windows DLLs out there that work fine with more modern tooling. I guess that means they don't use anything out of Visual C++'s standard libraries.

(And I suppose this serves as an example of why C, with all its faults, will outlast everything else..)

Soller: Real hardware breakthroughs, and focusing on rustc

mathstuf — Mon, 09 Dec 2019 14:07:57 +0000

> Windows' ABI has been stable since the beginning of time (barring the Win16->Win32->Win64 transitions) and until recently was exclusively implemented with C++.

Uh? No. Every MSVC release had incompatible standard library ABIs until 2017 and 2019 agreed on one and are able to share artifacts. I think there's one pending ABI break to fix a bunch of mistakes (including mangling `class` and `struct` decls differently), but I expect it to *then* be stable.

I think you're referring to the Win32 *API*'s ABI (which is C), not MSVC's C++ output ABI.

Soller: Real hardware breakthroughs, and focusing on rustc

pizza — Mon, 09 Dec 2019 14:02:19 +0000

Oh? Windows' ABI has been stable since the beginning of time (barring the Win16->Win32->Win64 transitions) and until recently was exclusively implemented with C++.

Meanwhile, in the past 20 years on Linux (with GCC), there have been only two ABI changes:

The first was a true ABI change relating to symbol mangling, as they implemented the standard various toolchain vendors finally hashed out together.

The second was a due to incompatible API changes in the standard library (C++11 required changes to std::string and std::list). GCC allows either API(and ABI) to be targeted at compile time, and mixing the two will result in (intentional) linking errors.

Soller: Real hardware breakthroughs, and focusing on rustc

xophos — Mon, 09 Dec 2019 11:57:04 +0000

C++ has no stable abi either.

Soller: Real hardware breakthroughs, and focusing on rustc

smurf — Mon, 09 Dec 2019 08:48:44 +0000

*Some* code writtn for a different century. Possibly even *most* code.

But Perl, too, has non-negotiable and backwards-incompatible deprecations.

Soller: Real hardware breakthroughs, and focusing on rustc

farnz — Mon, 09 Dec 2019 08:39:20 +0000

I have a system with a C extension to perl that disagrees strongly with you - perl 5.30 is different enough to perl 5.22 (that the code was written for) that the resulting application does not function correctly with perl 5.30.

Now, it's almost certainly the C extension that's buggy, but perl 5.30 has changed enough about the way it works that it has broken code written for and tested against 5.22. In my case, because perl is only used for this one application that we're rewriting anyway, it's easy enough to stick to 5.22; however, if we still developed against perl, we'd have to debug this to keep uup to date.

Soller: Real hardware breakthroughs, and focusing on rustc

kvaml — Mon, 09 Dec 2019 01:22:04 +0000

Thanks for the education to this non-Rust user. That was very clear and lucid.

Soller: Real hardware breakthroughs, and focusing on rustc

flussence — Sun, 08 Dec 2019 23:29:59 +0000

That's a non-issue for perl because perl has QA, it doesn't Move Fast And Break Downstream on a brittle six-week cycle. 5.30 will run code that was written for a different century in the same program as code written today.

Soller: Real hardware breakthroughs, and focusing on rustc

flussence — Sun, 08 Dec 2019 21:47:13 +0000

Distribution of code and walled-garden proprietary OSes are two different universes.

Soller: Real hardware breakthroughs, and focusing on rustc

nix — Sun, 08 Dec 2019 15:37:59 +0000

> You may well do. Remember I'M DISCUSSING ALTERNATIVES TO LINUX. When the linker loads the master object, it creates a load of dangling references. When it loads a library to resolve those references, it *only* resolves those references, so if one library routine calls another it may create a new dangling reference - which means the library needs to be loaded again to resolve it! Don't forget - the library is a collection of independent binary blobs and only those blobs that are needed are copied into the executable.

The problem here is that your terminology is so wrong that people who know the terminology can barely understand what you're saying. You're talking about gcc -lfoo -lbar -lfoo, right? That's not loading. That's linking. Loading is what ld.so does at runtime, and it never loads libraries more than once in the common case (modulo very obscure cases like dlmopen() and audit libraries).

Soller: Real hardware breakthroughs, and focusing on rustc

mathstuf — Sun, 08 Dec 2019 11:33:32 +0000

I think you're missing the minimum deps thing. With that resolution (which avoid ratcheting to newer-and-newer deps for everyone), you get the *oldest* that is compatible. Distros can just not provide the old version and the lowest compatible version would be sufficient. But for the sake of everyone else, sending a patch upstream that says "use a newer version of your dep which solves $some_problem" would be great.

I'm arguing that Debian shouldn't make its own house fine and not let upstream know that there's an issue with one of their deps.

Soller: Real hardware breakthroughs, and focusing on rustc

RedDwarf — Sun, 08 Dec 2019 09:03:05 +0000

> there is a huge quantity of existing C++ code in the world. That code can't be replaced overnight. Being able to take a new compiler and identify pre-existing bugs in that code

P1179 says "This approach aims for low false positives in “good” code that follows modern C++ conventions". I'm no expert, but AFAIK that analysis is never going to work in C++98 code. That existing C++ code, full of pointer arithmetic, is never going to be able to make use of this.

That old code is possibly without any automated test, and using raw threads in ways making you cry. If you want to rewrite it in modern C++ ("modern", and still without a standard work-stealing/fork-join framework) you may as well rewrite it in Rust.

Soller: Real hardware breakthroughs, and focusing on rustc

roc — Sun, 08 Dec 2019 04:58:06 +0000

I agree with almost everything you said until you got to
> If that C++ code can eventually become memory-safe though iterative improvements to tooling

Sure, but this isn't going to happen unless C++ *and that legacy C++ code* change dramatically.

It's understandable that C++ developers look for a silver bullet that can catch all or a vast majority of memory safety bugs without requiring them to do very much work. But there is no reason to believe that's possible, and very good reasons to believe it's impossible ... namely that people have been trying to build such tools for a long time and gotten nowhere close. It's understandable, but belief in the silver bullet is ultimately counterproductive here. Responsible C++ developers need to accept that the silver bullet (probably) does not exist and plan accordingly.

Soller: Real hardware breakthroughs, and focusing on rustc

FraserJGordon — Sun, 08 Dec 2019 00:34:02 +0000

Yes, that distinction is important (my initial reading of the paper suggested it tends towards reporting too many errors rather than too few so I got the impression it was fairly comprehensive). But I wanted to respond to the statement that C++ is "is nowhere close to being safe to use, and it never will be" as it seemed more negative than warranted. If people are working on making C++ safe(r) then I think that's worth highlighting. Even if it is only the most common memory safety bugs in the language, it'll still result in better software than we have currently.

Unarguably, Rust has already solved those safety bugs (and more). It is seriously worth considering for new software projects.

On the other hand, there is a huge quantity of existing C++ code in the world. That code can't be replaced overnight. Being able to take a new compiler and identify pre-existing bugs in that code is an easier sell than rewriting from scratch. If that C++ code can eventually become memory-safe though iterative improvements to tooling then that ought to be seen as a victory for Rust (as a pioneer of borrow checking and other safety features).

The situation looks more bleak for C; its type system is likely too limited to fully express what would be needed for memory safety. Unfortunately, it is also the common denominator for cross-language interfaces...

Soller: Real hardware breakthroughs, and focusing on rustc

roc — Sat, 07 Dec 2019 21:21:07 +0000

You're blurring the distinction between "find some common errors" and "prevent memory safety bugs in general". Sutter's lifetime checker is the former. Rust achieves the latter. Making C++ "safe to use" requires the latter; Sutter's lifetime checker won't do it.

Soller: Real hardware breakthroughs, and focusing on rustc

RedDwarf — Sat, 07 Dec 2019 14:35:25 +0000

There is no need to guess here. As I mention in https://lwn.net/Articles/806249/ the original paper from the designer is available for anybody to see, and he explicitly mentions bounds checking.

Yes, by making it UB it's "___allowing___ the implementer to use the fastest possible approach". I'm not going to say it's common at all, but somebody could well be in a situation where he can't afford the bounds checking. If you simply can't afford it your only option is to test the hell of your code, and then send it to the real war zone without any protections. Not my first option, but if it's the only available option...

In any case this is what the standard (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n...) says about:
- undefined behavior
"Permissible undefined behavior ranges from ignoring the situation completely with unpredictable results, to behaving during translation or program execution in a documented manner characteristic of the environment (with or without the issuance of a diagnostic message), to terminating a translation or execution (with the issuance of a diagnostic message)"

- implementation-defined behavior
"behavior, for a well-formed program construct and correct data, that depends on the implementation andt hat each implementation documents"

In the case of an out of bounds access to a span:
a) It's not "correct data", using an out of bounds index is always a bug, not something that "can happen, let's handle it", so it can't be implementation-defined behavior. There is never implementation-defined behavior for buggy code.
b) Having an option to enable/disable the out of bounds checks at runtime falls into the "behaving during ... program execution in a documented manner" category of UB.

Soller: Real hardware breakthroughs, and focusing on rustc

farnz — Sat, 07 Dec 2019 14:12:04 +0000

And therein lies the rub - part of the problem here is that distros are saying that someone else should do that work - and it looks like, at least for Rust, the language/tool vendors are going to do it well enough for users' and developers' needs, and simply bypass the distros altogether, with things like crater for testing the entire package ecosystem in a single build.

Note that this has already happened for JavaScript - if that's your language of choice, you get Node.js from the distro, and then switch to NPM or yarn from there on in; users are happy to run bundles created by developer tooling, and the distro becomes an irrelevance as soon as it's good enough to get Node.js to run. Maybe that's a good end point for distros - enough runutime etc to let the language tooling take over?

Soller: Real hardware breakthroughs, and focusing on rustc

smurf — Sat, 07 Dec 2019 14:09:04 +0000

No remembering should be necessary. Just take each package depending on libopenssl-dev, rebuild it, test it (the missing piece of the puzzle in all-too-many packages …), upload the result if it changed, rinse&repeat.

All of this can happen automagically, given modern tooling (which Debian for one is slowly moving towards – much too damn slowly for my taste) … except for the "fix the inevitable regressions" part of course.

Soller: Real hardware breakthroughs, and focusing on rustc

mathstuf — Sat, 07 Dec 2019 13:33:08 +0000

> That’s the thundering herd effect that makes it unpractical, to package large amounts of software, if upstream devs do not coordinate their version needs a minimum (and they can coordinate via shared libs, unified master company scm like Google, anything else but the coordination is necessary). semver alone does not work if everyone keeps requiring semver tip versions.

If OpenSSL 1.0.2j has some critical fix, why would I want to allow my software to compile against 1.0.2i anymore? Bumping the minimum helps *everyone* get better versions over time, not just those who remember to rebuild their OpenSSL.

The current flurry of changes needed for minvers support is because it is new in Cargo. If it had been there from the beginning, it likely would have been much easier to handle. But, this is the same problem with any bootstrapped initiative be it reproducible builds, new architectures, etc. The initial push is large, but the constant churn should be quite low.

Soller: Real hardware breakthroughs, and focusing on rustc

pizza — Sat, 07 Dec 2019 13:11:31 +0000

> And you can feel distributions are “not necessarily good”, but that’s irrelevant, unless you want to pick up the distribution work yourself. That won’t leave you with much time for dev, but such is real life, removing someone that provides a service to you, does not make the need for the service go away.

This is the bit that I keep coming back to. _someone_ has to do this work, and if it's not the distros [1], and not the language/tool vendors, then it's going to have to be you [2]. It's legit to say that the traditional distro model has its flaws and room for improvement, but one would have to be pretty naive to claim the sorts of problems and concerns distros have traditionally solved (ie "testing, deploying and maintaining configuration managed baseline systems useful for 3rd parties to develop on/for") somehow no longer matter.

[1] Not just distros ala Ubuntu, but whatever k8s and various proprietary cloud vendors come up with to fill the same role.

[2] "you" can mean an individual developer or an tools/platform team that big orgs tend to have. With every one of them reinventing the same wheel. If only there were independent organizations that could take on this work for the benefit of all involved... I know, we'll call them "distributions"...

Soller: Real hardware breakthroughs, and focusing on rustc

farnz — Sat, 07 Dec 2019 11:52:29 +0000

Every one of those crates is using the Cargo name and versioning - there is a trivial mapping from Cargo to dnf notation, and back again. The trouble is that the distro has not written the code to do that mapping for me, but is expecting that Cargo will magically stop using its own dependency resolution and handling (which is at least as good as dnf's) and start using the distro's code, just because that's what the distro wants.

Soller: Real hardware breakthroughs, and focusing on rustc

farnz — Sat, 07 Dec 2019 11:49:40 +0000

You're missing the point - code is written by developers. If developers move en-masse to a new language, distros don't get a choice about wanting C instead of Rust - they have to live with it, because the users of code want the new features of the new versions of code.

In C land, distros have power because there's no dependency management without them. In Rust land, if distros get between users and devs, they're trivial to bypass. We've already seen what happens with C code where the developers and users agree that it's beneficial to move to the new shiny - distros have to move, too, whether they like it or not, because users want the new GNOME etc, and thus distros have to accept GNOME's dependencies.

Stop thinking that distros matter to users - they don't, particularly, and there's a reason that no other OS has a distribution-like model. Users care about applications that make their computers do useful things; if distributions make it harder to get useful things than bypassing distributions, then users will bypass distributions.

If the distributions can't work out how to work with Rust developers, and system software migrates to Rust, then distributions will get reduced to sufficient tooling to run rustup and Cargo, because that's all that users will want from them; if distributions don't want to end up here, then they need to work with Rust now to get to a state where they are still relevant.

And note that Rust does make it as easy as it can on distributions - Cargo even translates its own dependency information to a simple JSON form for you, with or without resolving versions at your discretion. If distributions want Rust to do more than it does for them, then they need to engage with the Rust community to explain what they need and why.

The discussion with nim-nim elsewhere in this thread is typical, FWIW - as they asked for things from Rust, I explained how to get what they want from Cargo, and then they went off into unrelated territory. My favourite part of that was their claim that I just need to run dnf install font(kingscaslon), and dnf would package my font for me, then install it…

Soller: Real hardware breakthroughs, and focusing on rustc

farnz — Sat, 07 Dec 2019 11:42:00 +0000

I'm saying that Linux distros are not a significant driver for distribution of code; server-side, you do whatever the devs want you to, client side is iOS, Android, Windows etc.

This, in turn, pulls down their influence - why should the authors of Krita, or someone writing code to run on Amazon AWS or Google Cloud Engine, care if their users have to run "cargo install", or "yarn install", instead of "apt install"? Unlike C++ land, where dependency management is otherwise manual, modern languages don't need distribution packaging to be easy to install and use code written in those languages - and that means that distros no longer hold the place of power they do in C++ land.

Soller: Real hardware breakthroughs, and focusing on rustc

nim-nim — Sat, 07 Dec 2019 10:19:29 +0000

> This leads to different incentives - for C and C++ libraries, I want to make life easy for distros, not because distros are necessarily good, but because they are considerably easier to deal with than either vendoring all my dependencies or getting my potential users to build them in a way that I can use.

And, for Rust, you do not want to make it easy on distributions.

You’re pushing complexity on distributions.

You’re pushing complexity on users (cargo install foo is nice for a limited set of fringe components, do you actually expect users to construct the thousands of components composing a system like that? That requires big company-level of hand-holding).

And, you’re surprised distributions feel C and C++ libraries are better suited for systems works (what distributions do)? And do not want Rust anywhere near their core in its current state?

Really, what did you expect?

A system that does not incentivize, making the life of others easier, will result in little adoption by those others.

And you can feel distributions are “not necessarily good”, but that’s irrelevant, unless you want to pick up the distribution work yourself. That won’t leave you with much time for dev, but such is real life, removing someone that provides a service to you, does not make the need for the service go away.

Soller: Real hardware breakthroughs, and focusing on rustc

nim-nim — Sat, 07 Dec 2019 10:00:55 +0000

>> When you attain this size you can afford the combination explosion. Most entities (starting with community distros) can not.

> I maintained several projects there and spent way less time on that than maintaining a package for Ubuntu and coping with it being broken by Python updates.

That "works" because you do not care about the result being useful to others. But wasn’t your original complaint, that the python3-theano maintainers didn’t care that their package was not useful to your commercial app?

So you want a system that relies, on not caring for others, to scale, to be adopted by distributions, because the distributions should care about you?

Don’t you see the logical fallacy in the argument?

Al the things that you found too much work in the Ubuntu package, exist so the result can be used by others.

Soller: Real hardware breakthroughs, and focusing on rustc

nim-nim — Sat, 07 Dec 2019 09:47:59 +0000

>> when the language tooling makes it possible without inventing custom layouts like Nix does.

> Then this tooling needs to be implemented for other languages. As I said, I've seen it done at the largest possible scale. It doesn't even require a lot of changes, really.

Then don’t complain at distros, write a PEP, make upstream python adopt a parallel-version layout.

Major community distributions will apply the decisions of major language upstreams, it’s that simple. Major community distributions collaborate with major language upstreams. Collaborating implies respecting upstream layout choices.

In a company, you can sit on upstream decisions and do whatever you want (as long as someone finds enough money to fund your fork). That’s ultimately completely inefficient and counter productive, but humans do not like to bow to the decisions of others, so, that’s been done countless times and will continue to be done countless times.

Soller: Real hardware breakthroughs, and focusing on rustc

Cyberax — Sat, 07 Dec 2019 09:05:23 +0000

> And that’s exactly what major distributions do
Except that it's impossible to use it, because it's not possible to parallel install libraries.

> when the language tooling makes it possible without inventing custom layouts like Nix does.
Then this tooling needs to be implemented for other languages. As I said, I've seen it done at the largest possible scale. It doesn't even require a lot of changes, really.

> Of course, most of your complaint is that you *do* not* *want* to use the specific versions maintained by the distro teams.
Incorrect again. I would love to see a distro-maintained repository with vetted package versions, with changes that are code-reviewed by distro maintainers.

It's just that right now these kinds of repos are useless, because they move really slowly for a variety of reasons. The main one is the necessity to upgrade all versions in the distribution in lockstep.

Soller: Real hardware breakthroughs, and focusing on rustc

nim-nim — Sat, 07 Dec 2019 08:58:39 +0000

> Incorrect. In that companies libraries are maintained by their owner teams. The difference is that they typically maintain a handful of versions at the same time, so that all their dependants can build it

And that’s exactly what major distributions do, when the language tooling makes it possible without inventing custom layouts like Nix does.

Upstreams do not like distribution custom layouts. The backlash over Debian or Fedora relayouting Python unnilateraly, would be way worse, than the lack of parallel instability in the upstream Python default layout.

> Incorrect. In that companies libraries are maintained by their owner teams.

It’s awfully nice when you can order devs to use the specific versions maintained by a specific owner team.

Of course, most of your complaint is that you *do* not* *want* to use the specific versions maintained by the distro teams.

So, it’s not a technical problem. It’s a social problem.

Soller: Real hardware breakthroughs, and focusing on rustc

nim-nim — Sat, 07 Dec 2019 08:46:02 +0000

> any recursive fix needs a new release which bumps the minimum for the depending crate…

That’s the thundering herd effect that makes it unpractical, to package large amounts of software, if upstream devs do not coordinate their version needs a minimum (and they can coordinate via shared libs, unified master company scm like Google, anything else but the coordination is necessary). semver alone does not work if everyone keeps requiring semver tip versions.

You can delay the effect with containers or static linking, but as soon as a critical fix is needed, it comes back with a vengeance. The dev dream of “any dev uses whatever versions it wants and the rest of the supply chain will cope” is utopic in presence of un-perfect code, that will always require an eventual fix.

> Of course, if the distros provide an index that cargo can read

To find out what is available on my current distribution:

$ dnf repoquery -q --provides $(dnf repoquery --whatprovides 'crate()') | grep 'crate('

crate(abomonation) = 0.7.3
crate(abomonation/default) = 0.7.3
crate(actix) = 0.8.3
crate(actix-codec) = 0.1.2
crate(actix-codec/default) = 0.1.2
crate(actix-connect) = 0.2.3
crate(actix-connect/default) = 0.2.3
crate(actix-connect/http) = 0.2.3
crate(actix-connect/openssl) = 0.2.3
crate(actix-connect/rust-tls) = 0.2.3
…

To install one of those

$ sudo dnf install 'crate(abomonation) = 0.7.3'

That could probably be streamlined and plugged into cargo if someone wanted to (it would be much better if cargo provided a standard name and version mapping for rpm and apt at least, that would simplify reading it back cargo-side; the artefact naming is pretty flexible distribution side but the version format less so, as it is used to compute upgrade paths. The original semver format, before non Linux people added rpm and deb-incompatible things in it, was easy to map).

And then the result of the install is useless for the rust dev if there is no system workspace enabled by default. The crates will be installed but not used.

> Of course, if the distros provide an index that cargo can read, just not providing the minimum and giving a minimum-with-fixes available (treating the intermediate versions as "yanked") will likely have the same effect.

Yes, that’s the most likely scenario distro-side. There is a limited set of checked and available versions, anything else is considered yanked. Doing things any other way means drowning under version combinations.

Soller: Real hardware breakthroughs, and focusing on rustc

mathstuf — Sat, 07 Dec 2019 04:18:07 +0000

There is work on picking minimum versions rather than maximizing them when selecting dependency versions. There are a number of packages which don't specify accurate minimum versions in their crates, but I've been trying to sweep my dependencies for them at least. But, any recursive fix needs a new release which bumps the minimum for the depending crate…so, no exactly the easiest thing. But, if/when that lands in stable, I'd expect to see CI for such builds to become more popular.

Then distros' jobs would be to say "hey, there's a security fix in x.y.z, please update your minimum to x.y.z+1 and make a new release" and every stable-tracking distro gets a similar fix available. Of course, if the distros provide an index that cargo can read, just not providing the minimum and giving a minimum-with-fixes available (treating the intermediate versions as "yanked") will likely have the same effect. Investigation into that as a viable alternative would be necessary.

Soller: Real hardware breakthroughs, and focusing on rustc

pizza — Sat, 07 Dec 2019 00:43:27 +0000

Okay, you've managed to completely lose me. You've said that Linux is both minority and it isn't, that this is relevant and it isn't, and distros don't matter -- except when they do.

I have no idea what point you're trying to make, beyond "distros are useless, because reasons"

Soller: Real hardware breakthroughs, and focusing on rustc

Cyberax — Sat, 07 Dec 2019 00:37:39 +0000

And likely that most of these servers use language-specific package managers, only utilizing the distros for the base system.

Soller: Real hardware breakthroughs, and focusing on rustc

farnz — Fri, 06 Dec 2019 23:58:15 +0000

That's server-side, not total - and server side is exactly the place where you have developers who can run tools like Cargo instead of being helplessly dependent on distros. In other words, that's where distros are least useful to begin with, beyond the base libraries to get your developers going.