Rethinking Fedora's compiler policy [LWN.net]

Rethinking Fedora's compiler policy

Posted Apr 28, 2021 17:08 UTC (Wed) by Bigos (subscriber, #96807)
Parent article: Rethinking Fedora's compiler policy

In my opinion, the GCC compiler has shown a lot of progress in user-friendliness (diagnostics, compilation speed, integration with tools, etc.) and overall pace of development in response to Clang. I don't think it is going away anytime soon. Thus justifying the enforcement of use of GCC in Fedora as supporting compiler diversity sounds pretty strange. I would say it is backwards - allowing packages to use any compiler would make it more likely the code bases would be portable across compilers (like the Linux kernel that has gained mainstream support for Clang compilation relatively recently).

If there are issues with ABI incompatibility they should be found, fixed, documented and then continuously tested. This would help not only GCC <-> Clang interoperability but also compiler interoperability between different versions. This probably involves some custom compiler flags (like the -m* flag family) with different settings and defaults across compilers, so I understand it is not an easy job (and probably would involve the distribution packagers).

TL;DR; If GCC wants to keep itself relevant it should keep improving the compiler (and I believe the community is doing a fantastic job at that!) rather than try to enforce its use in a Linux distribution.

to post comments

Rethinking Fedora's compiler policy

Posted Apr 28, 2021 17:28 UTC (Wed) by zyga (subscriber, #81533) [Link] (11 responses)

I agree with everything except for the extra cost. The motives are clear and noble but is everyone on board? This _will_ put more burden on some people. I think that apart from any technical arguments one needs to consider the cost and decide if we have the capacity and volunteer time to make that happen.

Rethinking Fedora's compiler policy

Posted Apr 29, 2021 1:58 UTC (Thu) by josh (subscriber, #17465) [Link] (10 responses)

It sounds like a tradeoff of burdens. People trying to build software that's designed to build with clang/LLVM will have a lower burden, people who benefit from compiler uniformity will have a higher one.

Rethinking Fedora's compiler policy

Posted Apr 29, 2021 13:51 UTC (Thu) by nim-nim (subscriber, #34454) [Link] (8 responses)

It’s the same trade-off as with Java packages.

Upstreams do not want to handle multiple build scenarii. So they push for “simplification” by arguing everyone should make the same build choices as them (same compiler, same dep versions, ignoring the large overlaps between projects with different preferences, or the need of security patches).

Ultimately, since upstreams do no agree on what the best build choices are, adopting upstream decisions pushes the complexity down to the person that needs to build the parts used by projects with different preferences. And those people burn out, with cascading effects, till there is little left distro side.

But, do not fear, say upstreams. We can not cope with different compilers and deps to build our own project. But we are good for building and maintaining a whole system container to host our project independently!

Yes, right. When you are Google or Amazon, perhaps. Others need to care about the commons.

Rethinking Fedora's compiler policy

Posted Apr 29, 2021 21:27 UTC (Thu) by josh (subscriber, #17465) [Link]

I've seen plenty of projects that only support building with GCC, because they use GCC extensions. And ultimately, that doesn't tend to get fixed by patching the project, it tends to get fixed by adding the requisite feature to clang.

Until recently, the Linux kernel was one of those projects. People have put extensive work into making the Linux kernel work with clang. That work includes people adding features to clang, and people committing to ongoing maintenance of clang support in the Linux kernel. And it does add complexity to the Linux kernel; that complexity is just deemed worthwhile because clang adds useful features as well.

So, sure, if you have the time and bandwidth to work with upstream and commit to helping upstream maintain support for another compiler, and upstream is willing to maintain the complexity added by supporting another compiler, then it might be reasonable downstream to use that compiler. If you don't have that time or bandwidth, you aren't prepared to commit to maintaining it upstream, or it adds enough complexity that upstream isn't willing to maintain it, then it might not be reasonable to expect.

The same arguments apply to supporting less common architectures, or operating systems. Someone has to not only do the work, but maintain the work, and upstream has to be willing to deal with any added complexity.

Rethinking Fedora's compiler policy

Posted May 1, 2021 20:03 UTC (Sat) by NYKevin (subscriber, #129325) [Link] (6 responses)

Realistically, upstream is going to do what is best for upstream, and downstream will then do what is best for downstream. If upstream only supports compiler (or build toolchain, container system, etc.) X, and downstream would prefer to use compiler Y, then downstream has a limited set of options:

1. Suck it up and use X anyway.
2. Convince upstream to support Y. In practice, this probably involves directly helping upstream to maintain support for Y, rather than merely filing bugs whenever Y breaks.
3. Fork or patch the package to support Y.
4. Drop the package.

Although #2 is a possibility, upstream has an absolute right to refuse it, and it is not very productive for downstream to complain about this choice once it has been made. Therefore, it's probably a Good Idea for downstream to have a standardized process for choosing between options 1, 3 and 4. That does not mean that every package should get the same choice, of course, just that there should be a process for making this decision (other than "everyone gets on the mailing list and argues for three days").

Rethinking Fedora's compiler policy

Posted May 1, 2021 22:28 UTC (Sat) by rodgerd (guest, #58896) [Link] (5 responses)

Three imposes huge and unreasonable work on maintainers, and 4 is holding the distro's users hostage.

Really, if people choose not to use a given (compiler, init system, whatever), trying to use distro terrorism to force them to is admission of abject failure.

Rethinking Fedora's compiler policy

Posted May 1, 2021 23:08 UTC (Sat) by pebolle (guest, #35204) [Link]

> distro terrorism

How to top this? Non of distro nazism, distro communism, distro neo-liberalism, distro colonialism really work for me. So it seems you coined a winner here.

Thanks for making free software look even more like a lunatic fringe!

Rethinking Fedora's compiler policy

Posted May 2, 2021 1:27 UTC (Sun) by NYKevin (subscriber, #129325) [Link] (3 responses)

> Three imposes huge and unreasonable work on maintainers, and 4 is holding the distro's users hostage.

So what?

Upstream produces tarballs. Downstream can take them or leave them. If upstream is so inclined, it may voluntarily decide to make changes to accommodate downstream's wishes. But there is no general obligation for upstream to do this, and it would be quite absurd if there was (consider the number of different distros out there). If downstream responds to this reality with hostility and accusations of "holding users hostage," then it is entirely reasonable for upstream to killfile downstream's emails.

Rethinking Fedora's compiler policy

Posted May 2, 2021 10:55 UTC (Sun) by Wol (subscriber, #4433) [Link] (1 responses)

But if, say, downstream relies on llvm (which is not compatible with GCC?), then upstream is basically saying "we don't want you on our distro". Which in the long run is the distro cutting off its nose to spite its face.

If people want to use a different compiler because it supports features GCC can't - or in certain circumstances WON'T - support then you have all the makings of a nasty stand-off down to the distro's intransigence.

I think this is what's getting peoples noses out of joint - the distro forcing technically bad choices onto downstream.

Cheers,
Wol

Rethinking Fedora's compiler policy

Posted May 2, 2021 15:27 UTC (Sun) by pizza (subscriber, #46) [Link]

> I think this is what's getting peoples noses out of joint - the distro forcing technically bad choices onto downstream.

And vice-versa.

Rethinking Fedora's compiler policy

Posted May 2, 2021 14:56 UTC (Sun) by jccleaver (guest, #127418) [Link]

> Upstream produces tarballs.

If only... :/

Rethinking Fedora's compiler policy

Posted May 1, 2021 23:16 UTC (Sat) by amacater (subscriber, #790) [Link]

Seeing the work on reproducible builds in Debian, SuSE and elsewhere - having multiple compilers and running as much code as you can through at lest LLVM and GCC seems eminently sensible. This just one example: building on multiple machine architectures almost requires more than one compiler to be performant.

Rethinking Fedora's compiler policy

Posted Apr 28, 2021 17:32 UTC (Wed) by jwakely (subscriber, #60262) [Link]

> If GCC wants to keep itself relevant it should keep improving the compiler (and I believe the community is doing a fantastic job at that!) rather than try to enforce its use in a Linux distribution.

GCC isn't trying to enforce anything in Fedora. The fact that some GCC developers (who are also Fedora developers) have a particular point of view doesn't mean that "GCC" (as a community or a project or anything else) is trying to do anything here.

Rethinking Fedora's compiler policy

Posted Apr 29, 2021 14:59 UTC (Thu) by Paf (subscriber, #91811) [Link]

“ I would say it is backwards - allowing packages to use any compiler would make it more likely the code bases would be portable across compilers (like the Linux kernel that has gained mainstream support for Clang compilation relatively recently).”

This is addressed in the article. In practice, projects below a certain size will often gravitate towards a single compiler. Upstream will only work with that one, and if downstreams don’t do anything, they’ll move towards that. It’s just easier.

The kernel is more than large enough to generate its own gravity and suck people towards it - Clang is changing so it can build the kernel. And, yes, the kernel is being adjusted to clang support, but that’s because communities of clang users want to build the kernel with it - the kernel is pulling *them* in.

This doesn’t happen with smaller projects. If the project isn’t big enough to drive interest from communities that want specific compilers, it will take the path of least resistance. So if it gets hard to build it with a second compiler, they’ll probably just shrug.