Szorc: Mercurial's Journey to and Reflections on Python 3
Szorc: Mercurial's Journey to and Reflections on Python 3
Posted Jan 14, 2020 21:32 UTC (Tue) by roc (subscriber, #30627)In reply to: Szorc: Mercurial's Journey to and Reflections on Python 3 by dvdeug
Parent article: Szorc: Mercurial's Journey to and Reflections on Python 3
The problem is that there aren't enough people who want to be able to run the latest and greatest software on museum architectures to actually support those architectures through just their own efforts, e.g. by maintaining an out-of-tree LLVM backend. Thus they try to influence other developers to do work to help them out. Sometimes they get their preferences enshrined as distro policy to compel other developers to work for them. Sometimes it's borderline dishonest, raising deliberately vague concerns like "LLVM's limited platform support" to discourage technology choices that would require them to do work.
This is usually just annoying, but when it means steering developers from more secure technologies to less-secure ones, treating museum-architecture support as more important than cleaning up the toxic ecosystem of insecure, unreliable code that everyone actually depends on, I think it's egregious.
Posted Jan 15, 2020 3:25 UTC (Wed)
by dvdeug (guest, #10998)
[Link] (13 responses)
> Eventually one of them will try to use your program on a FAT-formatted USB stick with Shift-JIS filenames or whatever, .... As a responsible programmer you want to make your program work for that user...
>That experience will encourage you to start your next project in a different language (one whose designers have already considered this problem and solved it properly) so you won't have the same pains if your project becomes popular and needs to handle obscure edge cases.
You know that getting these programs to work on these systems is a priority for some people. You can do the hard work like people did for C (and a number of other languages), and write a direct compiler. You can piggyback on GCC, like a half-dozen languages have. You can compile to one of the languages in the first two groups, like any number of languages have, or write an interpreter in a language in one of the three groups, like vastly more languages have. The Rust developers instead chose to handle this in a way that wouldn't support some of their userbase. That has nothing to do with being a "more secure technology"; that's "choosing to drop customer requirements that would take work to support".
I see where you're coming from, but on the other hand, if your competition supplies a feature that people want, perhaps it's on you to implement that feature, and perhaps developers will consider excors' advice above about using a language that won't have this problem.
(As a sidenote, when you say "maintaining an out-of-tree LLVM backend", do you mean that LLVM wouldn't accept a backend for m68k, etc.? Because I don't blame anyone for not wanting to maintain an unmergable fork of a program, and that simply makes the argument against using LLVM so much stronger.)
Posted Jan 15, 2020 10:23 UTC (Wed)
by roc (subscriber, #30627)
[Link] (12 responses)
I think that's a fine way to look at it, as long we are clear about which "customers" are actually being dropped. "The Rust developers chose to handle this in a way that wouldn't support some of their userbase" sounds rather ominous, but when we clarify that "some of their userbase" means "a few obsolete and a few minor embedded-only architectures", it sounds more reasonable.
> (As a sidenote, when you say "maintaining an out-of-tree LLVM backend", do you mean that LLVM wouldn't accept a backend for m68k, etc.? Because I don't blame anyone for not wanting to maintain an unmergable fork of a program, and that simply makes the argument against using LLVM so much stronger.)
Oddly enough, this is not a theoretical question: https://lists.llvm.org/pipermail/llvm-dev/2018-August/125...
I just don't see an argument that anyone other than the m68k community should bear the cost of supporting m68k. Everyone using m68k to run modern software for any real task could accomplish the same thing faster with lower power on modern hardware, therefore they are doing it strictly for fun. No-one *needs* to run a particular piece of modern software on m68k. I don't know why gcc play along; I suspect it's inertia and a misguided sense of duty. Same goes for the other obsolete architectures.
I have more sympathy for potentially relevant new embedded architectures like C-Sky. But I suspect that sooner or later an LLVM port, upstream or not, is going to be just part of the cost of promulgating a viable architecture, especially for desktops. There are already a lot of LLVM-based languages, including Rust, Swift and Julia; clang-only applications like Firefox and Chromium (Firefox requires Rust too of course); and random other stuff like Gallium llvm-pipe.
I suspect that once you can build Linux with clang, CPU vendors will start choosing to just implement an LLVM backend and not bother with gcc, and this particular issue will become moot.
Posted Jan 15, 2020 11:46 UTC (Wed)
by dvdeug (guest, #10998)
[Link] (10 responses)
But somehow it doesn't sound reasonable that distributions that support those architectures don't support using Rust for core software? You're trying to have your cake and eat it too.
> I just don't see an argument that anyone other than the m68k community should bear the cost of supporting m68k.
You don't see an argument for cooperating with your fellow open source developers on their projects, but you do see an argument for supporting a billion dollar company that produces proprietary software and hardware with their proprietary Qualcomm Hexagon ISA.
We could start with community. If that doesn't move you, go with simple politics; free software has its own politics, and that guy who wrote the code you need to change to compile the kernel with LLVM turns out to be one of the guys who did the original Alpha port (which did all the work needed to make Linux portable beyond the 80386) and runs an Alpha in his basement, and funny, he's in a mood to be critical of your patches instead of helpful.
> with lower power on modern hardware,
How much power does it take make a modern computer? The m68k (e.g.) is a little extreme, but it's certainly a signpost that Linux, as an operating system, is not going to drop support for hardware that's not the latest and greatest. I've got a laptop that has 20 times the processor and eight times the memory of what I went to college with, that has Windows 10 on it, and response times are vastly worse than that laptop I used in college. I don't want to see Linux go that way. There's an environmental cost in forcing perfectly good hardware to be replaced, as well as a financial one.
> I suspect that once you can build Linux with clang, CPU vendors will start choosing to just implement an LLVM backend and not bother with gcc,
Cool. What you're saying is that if you have your way, the programming language that has my heart strings, Ada, will get much harder to use on modern systems, and should I return to active work on Debian I have an interest in discouraging Rust and LLVM and firmly grounding the importance of GCC in the system. Systemd tries to support old software and systems; did you look at the arguments and decide that actively breaking old software and systems would have been better?
Posted Jan 15, 2020 14:54 UTC (Wed)
by Cyberax (✭ supporter ✭, #52523)
[Link] (3 responses)
Posted Jan 21, 2020 7:02 UTC (Tue)
by dvdeug (guest, #10998)
[Link] (2 responses)
Posted Jan 21, 2020 7:47 UTC (Tue)
by Cyberax (✭ supporter ✭, #52523)
[Link] (1 responses)
LLVM can replace GCC completely.
Posted Jan 21, 2020 12:03 UTC (Tue)
by dvdeug (guest, #10998)
[Link]
LLVM could in theory replace GCC completely. And why would a company that's been working with GCC for 25 years find it worth giving up all that expertise to do so? A research project is useful, and it's possible a useful tool will come of this, but there seems to be no evidence GCC is in such a dire state for AdaCore to change the underlying platform on their core product.
Posted Jan 15, 2020 22:35 UTC (Wed)
by roc (subscriber, #30627)
[Link] (5 responses)
That sounds reasonable in isolation, but when it's part of a causal chain that results in a few hobbyists holding back important improvements for the other 99.999% of users, it becomes unreasonable.
> produces proprietary software and hardware with their proprietary Qualcomm Hexagon ISA.
While you're wielding "proprietary" as a slur, keep in mind that almost every architecture that you think it's important to support is also proprietary.
> We could start with community.
Can you elucidate the actual argument here?
> If that doesn't move you, go with simple politics; free software has its own politics, and that guy who wrote the code you need to change to compile the kernel with LLVM turns out to be one of the guys who did the original Alpha port (which did all the work needed to make Linux portable beyond the 80386) and runs an Alpha in his basement, and funny, he's in a mood to be critical of your patches instead of helpful.
The Linux community expects maintainers to evaluate patches on their merits, not taking into account that sort of quid pro quo.
Even if you think that behavior should be tolerated, I doubt it would actually happen often enough in practice that it would be worth anyone taking it into account ahead of time.
> I don't want to see Linux go that way. There's an environmental cost in forcing perfectly good hardware to be replaced, as well as a financial one.
I'm not sure what arguments you're making here. I can imagine two:
> What you're saying is that if you have your way,
FWIW "CPU vendors will start choosing to just implement an LLVM" is my prediction, not my goal. I actually slightly prefer a world where CPU vendors implement both an LLVM backend and a gcc backend, because I personally like gcc licensing more than LLVM's. I just don't think that's the future.
> Systemd tries to support old software and systems; did you look at the arguments and decide that actively breaking old software and systems would have been better?
Not sure what you're trying to say here. If the cost of supporting old systems is very low, I certainly wouldn't gratuitously break them. For example, in rr we support quite old Intel CPUs because it's no trouble. OTOH once in a while we increase the minimum kernel requirements for rr because new kernel features can make rr better for everyone running non-ancient kernels and maintaining compatibility code paths for older kernels is costly in some cases.
If systemd developers are spending a lot of energy supporting old systems used by a tiny fraction of users, when they could be spending that energy making significant improvements for the other 99.999%, then yeah I'd question that decision.
Posted Jan 17, 2020 9:10 UTC (Fri)
by dvdeug (guest, #10998)
[Link] (4 responses)
That's dramatic and silly. Red Hat doesn't care about the m68k, and presumably it's not going to hold back important improvements from them. Nor Suse. If Debian cares about the m68k, more than 0.001% of their users care about it in theory, even if they don't use it, and Ubuntu and other Debian derivatives can work around that if they care about it.
It's a free/Free market. Program developers can write what they want, and distributions can use what they want. If a distribution's priorities aren't in line with yours, you can go somewhere else. If they don't want to include your program without some features, you can include those features or not, and if not, they can patch in those features or go without. Don't gripe about those distributions; just add the features or not.
> While you're wielding "proprietary" as a slur, / Can you elucidate the actual argument here?
Ah, see, I was a developer for Debian GNU/Linux. So the idea that we should be working on Free Software as a team is important to me.
> keep in mind that almost every architecture that you think it's important to support is also proprietary.
At different levels, maybe. But a patent only runs 20 years, so any old enough CPU can be reimplemented without license. And the uses aren't proprietary; it's a bunch of hobbyists who benefit, not one big company. There's a difference between an x86-64 chip that's mass-marketed and used in a vast array of devices, and a chip that's only used on Qualcomm's SoCs, and primarily running Qualcomm's code. If LLVM is worried about the cost of bringing an architecture in house, then why let Qualcomm take your developer's time?
> The Linux community expects maintainers to evaluate patches on their merits, not taking into account that sort of quid pro quo.
So you get to judge whether a feature is important or not, but a Linux maintainer can't? You can choose what features you work on, but a Linux maintainer can't? A Linux maintainer can certainly say "your patch causes the kernel to crash; here's the traceback", and leave it at that, even if it will take fifteen minutes of their time or a dozen hours of yours to find the bug. I don't know if they can say that LLVM support isn't worth it--it's probably down to Linus himself--but they can at the very least quit if they feel they have to deal with pointless LLVM patches instead of important patches.
> I'm not sure what arguments you're making here.
I said the m68k is an extreme case. But it is a bellwether; a system that is quick to drop old hardware is much more likely to drop my old hardware, and a system that support the m68k is much less likely to go through and dump support for old hardware. It is something of a matter of pride that Linux works on old systems. Even passing the tests on many of these old systems puts a limit on how slow the software can be.
> once in a while we increase the minimum kernel requirements for rr
Which isn't much a problem because the kernel cares about backward compatibility and doesn't go around knocking off old hardware. It would be a lot more frustrating if every time you had to upgrade rr, you had to upgrade the kernel and seriously worry about the system not coming back up or important parts not working; perhaps many people would stop doing both.
Basically it comes down this paragraph again: It's a free/Free market. Program developers can write what they want, and distributions can use what they want. If a distribution's priorities aren't in line with yours, you can go somewhere else. If they don't want to include your program without some features, you can include those features or not, and if not, they can patch in those features or go without. Don't gripe about those distributions; just add the features or not.
Posted Jan 17, 2020 11:10 UTC (Fri)
by peter-b (guest, #66996)
[Link] (1 responses)
Yes it does. https://lwn.net/Articles/769468/
Posted Jan 17, 2020 13:16 UTC (Fri)
by smurf (subscriber, #17840)
[Link]
It's not the kernel [developers] that knock off hardware – it's the users who retired said hardware. If anybody had spoken up in favor of keeping (and maintaining …) the dropped architectures or drivers in question, they'd still be supported.
Posted Jan 17, 2020 14:59 UTC (Fri)
by mjg59 (subscriber, #23239)
[Link]
The kernel dropped support for 386 years ago, despite it being the first CPU it ran on.
Posted Jan 17, 2020 22:36 UTC (Fri)
by roc (subscriber, #30627)
[Link]
Of course. But we should still discuss the impact of those choices, which is not always obvious.
This subthread was triggered by just such a discussion:
which led us into a discussion of what exactly "LLVM's limited platform support" means and how important that is relative to other considerations. I learned something and I guess other readers did too.
Posted Jan 16, 2020 17:34 UTC (Thu)
by ndesaulniers (subscriber, #110768)
[Link]
Szorc: Mercurial's Journey to and Reflections on Python 3
Szorc: Mercurial's Journey to and Reflections on Python 3
Looks like it was not rejected outright, though I'm dubious it would ever be accepted. Maintaining an out-of-tree fork is more work than having it upstream --- while it is out-of-tree, all the maintenance has to be done by the m68k community, but upstreaming the backend would shift a lot of the maintenance costs from the community fans to the rest of the LLVM developers. That is exactly why they would not accept it!
Szorc: Mercurial's Journey to and Reflections on Python 3
Szorc: Mercurial's Journey to and Reflections on Python 3
GNAT developers saw the writing on the wall and are working on adding Ada support to LLVM: https://blog.adacore.com/combining-gnat-with-llvm
Szorc: Mercurial's Journey to and Reflections on Python 3
Szorc: Mercurial's Journey to and Reflections on Python 3
Szorc: Mercurial's Journey to and Reflections on Python 3
Szorc: Mercurial's Journey to and Reflections on Python 3
1) Replacing an m68k machine with something more modern hurts the environment because of the manufacturing impact of the more modern machine.
In practice m68k is so obsolete you could replace it something much faster and more efficient that someone else was going to throw away.
2) Keeping modern software running on m68k helps keep that software efficient for all users.
In practice I have not seen, and have a hard time imagining, developers saying "we really need to optimize this or it'll be slow on m68k, even though it's fast enough for the rest of our users". To the extent they care about obscure architectures, if it works at all, that's good enough.
Szorc: Mercurial's Journey to and Reflections on Python 3
Szorc: Mercurial's Journey to and Reflections on Python 3
Szorc: Mercurial's Journey to and Reflections on Python 3
Szorc: Mercurial's Journey to and Reflections on Python 3
Szorc: Mercurial's Journey to and Reflections on Python 3
> Porting Python apps to Go is currently quite a lot more viable than Rust, which is both handicapped by LLVM's limited platform support and doesn't have any batteries included.
Szorc: Mercurial's Journey to and Reflections on Python 3