LWN: Comments on "Compiling Rust with GCC: an update"

Compiling Rust with GCC: an update

Wol — Sat, 01 Oct 2022 14:05:07 +0000

I think anything like that is promptly going to fall foul of different people running different rustc.

If there are two major compilers, stuff will be NACK'd if it doesn't run on both. And even if it's bikeshedded over, the "editions" stuff will make sure it's okay.

(And from the efforts the rust people are going to for linux, and also given Linus' very pragmatic not purist attitude, stuff that linux wants is not likely to suffer too much bikeshedding. Even if it does (quite likely) end up as "this only exists in unsafe code".)

Cheers,
Wol

Compiling Rust with GCC: an update

sammythesnake — Sat, 01 Oct 2022 12:44:55 +0000

That sounds like a very risky prospect to me.

If those bikeshedded features do eventually escape the shed and appear in rust (as defined by rustic or by some future official spec) they may not do so unchanged from whatever variant was implemented in whatever codebase jumped the gun. Then what we have is two implementations that are incompatible and codebases designed for each one.

At the very least, it would be necessary to have some way to indicate (analogously with editions?) which version of which features are implemented (e.g. rust 1.55 + shiny-new-feature-1-from-draft-spec-NNNN-x.yy.zzz + shiny-new-feature-2-from-draft-spec-MMMM-a.bb.ccc + ...) that code could indicate it's coded for.

To be able to compile such code, rustc (or any third implementation) would also have to provide compatible support for these incompatible variants. Variants that they explicitly declined to implement pending conclusion of bikeshedding.

I suppose they could be handled similarly to unstable features, though.

If the additions were limited to adding selected features from a future edition (e.g. rust 1.55 + feature-AAAA-from-rust-1.56) then you could reasonably hope that the more involved edition-plus-modifiers thing might at least be *forward* compatible, but still horribly convoluted and developers would likely much prefer to keep the number of modifiers specified to a minimum by simply upping the edition they specify at the first opportunity.

In a worse scenario, though, the features caught in bikeshed limbo might be decisively NACKed leaving two permanently diverged forks of the language. That can't possibly be a good thing.

Perhaps lessons can be learned from "browser feature detection" in various JavaScript libraries which has been a thing for quite some time. Is it reliably non-nightmarish...?

Compiling Rust with GCC: an update

njs — Fri, 23 Sep 2022 14:42:08 +0000

Given that gccrs is funded by the grsecurity folks, and their big product is "the kernel, but compiled with proprietary gcc plugins", I always assumed the motivation for gccrs is that they want to get rust into the gcc pipeline early enough that it's before their plugins.

Compiling Rust with GCC: an update

mathstuf — Fri, 23 Sep 2022 11:35:33 +0000

Ah…indeed. Oh well :( .

Compiling Rust with GCC: an update

Tobu — Fri, 23 Sep 2022 11:10:03 +0000

> with recent updates!

The date in that page seems to be generated on every request. Following the links, updates are from 2012 or so.

Compiling Rust with GCC: an update

basix — Wed, 21 Sep 2022 02:30:34 +0000

Small nitpick: It's RUSTC_BOOTSTRAP, not RUST_BOOTSTRAP.

Compiling Rust with GCC: an update

flussence — Sun, 18 Sep 2022 15:45:39 +0000

Game console homebrew is a weird place. I've noticed substantial *N64* support patches going into mainline during 5.x. Not sure what they were going for, but that's pretty cool.

Compiling Rust with GCC: an update

firstyear — Thu, 15 Sep 2022 03:18:07 +0000

> Further, rust is EXTREMELY vulnerable to a trusting-trust attack right now. If, at some point, someone backdoored a rust compiler to add their malicious code to any rust compiler it compiles, then it's very possible that said backdoor has propagated across a chunk of the ecosystem.

These attacks just don't happen in reality though. It's "simple to grasp" but "almost impossible to fix" which makes it extremely attractive to a broad audience to spend huge amounts of time writing think pieces about it. When in reality attacks are "complex and difficult to grasp" and "require a lot of smaller broad, annoying fixes".

No one is pulling off these backdoor compiler attacks today. And why would they? Attackers don't attack "ideologically shiny targets" they attack the lowest hanging fruit. Things like lack of mfa, typo-squatting popular libraries, uploading malicious source directly into a library, and more. But I don't see people being willing to acknowledge the broad complex social and technical systems that would actually need to be improved to resolve this.

Compiling Rust with GCC: an update

rahulsundaram — Tue, 13 Sep 2022 23:09:56 +0000

> I did indeed not know that, thanks for pointing that out! I'm not sure if it really makes things better though

It may not be better but it is more accurate. Even if one were to stick to the same conclusion they had originally, it's helpful to validate the data points. The common narrative appears to be that GCC support for various languages including Rust are being added based on unreasonable demands for some unknown but definitely odd reasons and they will inevitably fail or splinter the language and the spec based approach is an old relic of the past that must be inherently doomed one way or the other. It doesn't leave room much room for acknowledging that multiple languages do have several successfully used implementations or even a mild curiosity of why things like gccrs is even funded by commercial organizations in the first place.

> I don't hate the idea of multiple implementations, but it really gives reason to temper your expectations of what gccrs will deliver.

This is something readily acknowledged in https://github.com/Rust-GCC/gccrs/wiki/Frequently-Asked-Q... but this is some extend mitigated by funding and possibility of code sharing via things like polonius.

Compiling Rust with GCC: an update

atnot — Tue, 13 Sep 2022 19:16:36 +0000

I did indeed not know that, thanks for pointing that out! I'm not sure if it really makes things better though: A frontend for a relatively simple language with the full support of a team that earnestly believed in the specification with multiple implementations approach is today mostly irrelevant, has a single active contributor and appears to be at least half a year behind an already frozen spec. I don't hate the idea of multiple implementations, but it really gives reason to temper your expectations of what gccrs will deliver.

Compiling Rust with GCC: an update

rahulsundaram — Tue, 13 Sep 2022 17:53:32 +0000

> People demanded gcc frontends for D and Go, which are mostly abandoned and nobody uses, because why would you

I have seen this claim made in several places but this isn't what happened atleast for Go. It wasn't based on any demand from anyone. Go team themselves decided to do it.

https://go.dev/blog/gccgo-in-gcc-471

"The Go language has always been defined by a spec, not an implementation. The Go team has written two different compilers that implement that spec: gc and gccgo. Having two different implementations helps ensure that the spec is complete and correct: when the compilers disagree, we fix the spec, and change one or both compilers accordingly. Gc is the original compiler, and the go tool uses it by default. Gccgo is a different implementation with a different focus, and in this post we’ll take a closer look at it."

Compiling Rust with GCC: an update

atnot — Tue, 13 Sep 2022 17:39:46 +0000

My question is: if all this comes to pass as written, then what exactly was the purpose of gccrs over rust-codegen-gcc and mrustc?

It won't be a fully independent implementation that can be used to find ambiguities, as nebulous as that idea was. Folks who insist that Rust must have multiple compilers because that's what C has have seemingly moved the goalposts elsewhere already anyway. It doesn't help bootstrapping because it won't be pure C++ like mrustc. The people who have a weirdly selective worry about dependence on permissively licensed software won't be happy either. Who exactly is left then? GCC developers? People who's mouth froths at the sight of a Code of Conduct? People who can install a modern gcc version and all of the other rust development tools, but not rustc? Are there really enough of those?

We've seen this play out a few times before, even: People demanded gcc frontends for D and Go, which are mostly abandoned and nobody uses, because why would you. I don't see how this vision of gccrs would be any different.

Compiling Rust with GCC: an update

hunger — Tue, 13 Sep 2022 14:04:17 +0000

> Part of the idea is to stabilize rust: right now, there are no bugs in rustc, because the language is defined as what rustc says is valid. There's no standard for how rust should behave; the documentation is informative, not normative.

While totally true: How will adding a gcc-based rust compiler change this?

The current process to improve Rust is to write an RFC and provide an implementation for rustc. This is then extensively tested, feedback on the implementation is collected, all available Rust projects will be built to see what breaks, etc. With this process gcc-based Rust will always have to catch up to the "real" rust and will never be a serious alternative! The only way to avoid this is to change the process and make that less code- and more paper-based. The process would need to produce a specifications for the compiler teams to implement later. This sounds like a huge step backwards to me! Just look at C++ to see how poorly that works: Compilers are always behind the specification and it is a huge pain for projects to agree on the exact features they can use in their code (without loosing too many users that need to stick with older compilers).

But in practice this will probably not be necessary: The gcc-based rust compiler plans to reuse parts of the original rust compiler as those are factored out and become available. E.g. Polonius, the library that should eventually contain the entire borrow checker. This will significantly reduce the costs to maintain the gcc-rust project and is thus obviously a good thing. So in the long-run all possible programs that work with rust code will converge towards a shared frontend, incl. rustc, rust-analyzer, a stand-alone gcc-based rust compiler and more. Many of the Rust features will be in this shared front-end code eventually, at which point we are back at having an implementation defined language. This also re-introduces the trusting-trust issues you bring up: If the shared front-end code produces malicious high-level code, then all non-malicious backend-implementations will faithfully produce malicious binaries from their inputs.

Compiling Rust with GCC: an update

milesrout — Tue, 13 Sep 2022 11:19:15 +0000

What's the point of multiple implementations if it's insisted that they all operate completely in lockstep, a few versions behind rustc? What's the point of multiple implementations if the "rustc" version is considered the same thing as the Rust version?

The best thing about having a separate Rust implementation would be that long-needed features could be implemented and used that have been languishing in RFCs for years being bikeshedded to death. How long did it take for const generics to be added? And they're still highly limited. Where are generic associated types? Tracking issue open for FIVE YEARS. Where are DST coercions? The DST coercion tracking issue has been open for SEVEN YEARS, almost as long as Rust has existed, and they're going nowhere.

Compiling Rust with GCC: an update

paulj — Tue, 13 Sep 2022 09:54:34 +0000

Building your own trusted tool chain is following Thompson's advice on what you need to do to build trust. So... not DDC.

Compiling Rust with GCC: an update

farnz — Tue, 13 Sep 2022 08:30:33 +0000

Assuming mrustc is bug-free against its own spec, the rule is that it'll compile all code that rustc does with the same behaviour in the final compiled binary, but it will also compile programs that rustc rejects as invalid. For this comparison purpose, that's good enough - you can't trust an mrustc-built compiler if you don't also have a rustc-built compiler from the same source.

Compiling Rust with GCC: an update

kleptog — Tue, 13 Sep 2022 07:23:20 +0000

> We shouldn't be discounting attacks like that on the basis of what is popular.

Sure, but on the other hand it's sufficient if only a small group are working on solving the problem. Once it's worked out, we can automate it and roll it out everywhere. It's also relevant that there hasn't been an example found in the wild, which means it's judged very low risk.

Compiling Rust with GCC: an update

pabs — Tue, 13 Sep 2022 04:01:48 +0000

The Bootstrappable Builds project is an example of doing just that, although they aim to bootstrap all architectures from that initial machine code step, without going through the cross-compiler stage:

https://bootstrappable.org/

Compiling Rust with GCC: an update

mjg59 — Tue, 13 Sep 2022 02:39:50 +0000

I'd expect rustc built with mrustc to behave differently in various ways (such as plausibly not being correctly memory safe in the face of malformed input), but I'd expect the actual compilation process to be the same? It doesn't really matter, though, since you can just do another round of building rustc and you should have the same assertions around it being trustworthy.

Compiling Rust with GCC: an update

mjg59 — Tue, 13 Sep 2022 02:37:37 +0000

No, you can generate a trusted compiler via a directly introspectable process. Pick an architecture. Write a trivial assembler directly in machine code. Use that to bootstrap a more competent assembler. Write a trivial C compiler. Use that to build an extremely old version of gcc. Use that to build a modern version of gcc. Use that to build a cross-compiler for whatever architecture you actually care about. You now have a trusted compiler, and the rest of Diverse Double Compilation falls out of that.

Compiling Rust with GCC: an update

rahulsundaram — Mon, 12 Sep 2022 22:51:28 +0000

> While trusting trust attacks are hypothetically possible, they're of little to no concern to the vast majority of people.

We shouldn't be discounting attacks like that on the basis of what is popular. Otherwise, we run the risk of repeating the giant mess from the slew of side channel attacks that went from "hypothetically possible" to demonstrable but hard to repeat to causing industry wide changes within a few years. Good news for Rust is that there are multiple implementations already and only likely going to mature with time.

Compiling Rust with GCC: an update

moltonel — Mon, 12 Sep 2022 22:41:11 +0000

Fair enough, you're right, some amount of messiness is unavoidable. But keeping the feature diff small is a good ideal to aim for.

Compiling Rust with GCC: an update

paulj — Mon, 12 Sep 2022 22:29:22 +0000

You're making an assumption that a subversion of a compiler binary can only be carried out by a binary of the same compiler.

There is no good reason to make this assumption. Particularly given the existence of file formats that aggregate different executable blobs together, along with hooks to allow execution to occur on loading. Even without those formats, there is simply no good reason to think the attacker who can (originally) cause a distributed binary of compiler A to be subverted must be limited to targeting the further subversion of /only/ compiler A source and binary.

Hell, even Thompson's original PoC targeted /two/ sets of sources for subversion of output.

The chances that a binary subversion targets your mrustc compiler AND your C++ compiler to compile mrustc may be lower than a subversion targeting just one, but that's kind of assuming you and your work-flow are not specially interesting to a skilled and sufficiently capable attacker. And such assumptions are not a basis to state "it is eliminated".

There are other assumptions in the DDC paper, e.g., that we could dig up some old compiler that existed before our potential-target. But then... we're still trusting a number of things, including the MAC algorithm. And MACs have a finite shelf-life - they weaken over time. Maybe the probability is low, but that depends on the juiciness of the target and the threat-model - and "lower probability" is different to "eliminated", unless you're into hand-waving.

Compiling Rust with GCC: an update

himi — Mon, 12 Sep 2022 22:24:24 +0000

>That was my understanding so far. But the implementation of const generics before they reached the 1.49 milestone casts a doubt. How will gccrs describe its current feature set ? "1.49 plus const generics as of 1.60 plus features X and Y" ? "1.60 except feature Z" ? Users will simplify that in their head to a single version number, and possibly write an optimistic MSRV to Cargo.toml.

Don't they just mean that during development they're targeting 1.49 for the moment, and in order to be able to support testing with the kernel (and satisfying the preference from a lot of kernel devs for having multiple compilers) they're also trying to get a move on with some more advanced features the kernel project needs? As in, they haven't set their supported version in stone, they're just using some of them as goal posts, with the kernel stuff as an additional goal post.

Presumably before they announce they're production ready for kernel builds they'll reach something a lot closer to feature parity. And presumably they'll only announce "supports version 1.foo" when they have full feature parity with version 1.foo, for exactly the reasons you've noted.

They're still in fairly early development - maybe wait until they're closer to actual release candidates before dinging them for how they're talking about feature/version support . . .

Compiling Rust with GCC: an update

himi — Mon, 12 Sep 2022 22:12:43 +0000

I'd have to actually check the bootstrap process for rust (it's probably been five years since I last tried), but I have a feeling you have a second step beyond just building rustc with mrustc, though that may be more of a verification step. mrustc doesn't have a lot of the memory safety features of rustc, and only implements the minimal features needed to compile rustc, so I don't think a compiler built by it will behave the same as the compiler built by rustc, even if it's the same version (i.e. mrustc building rustc 1.54 compared with rustc 1.54 building rustc 1.54).

It does still break (or dilute greatly) the chain of trust, but you'd need a more careful verification to make sure nothing hinky was going on.

Compiling Rust with GCC: an update

mjg59 — Mon, 12 Sep 2022 22:08:31 +0000

In what way is it not eliminated?

Compiling Rust with GCC: an update

mathstuf — Mon, 12 Sep 2022 18:38:52 +0000

> How will gccrs describe its current feature set ?

If I were the gccrs developers, I'd probably describe it as "1.49 with additional implemented features". Or, more succinctly, "see our issue tracking for specifics because we are still under heavy development".

> Users will simplify that in their head to a single version number, and possibly write an optimistic MSRV to Cargo.toml.

Users will do awful things. What do you suppose anyone do about it?

> it's normal for gccrs to want to release features without exactly following rustc's release history,

Yeah, railroading any implementation to limitations such as "you cannot make stdlib function X const before you implement $complicated_compiler_feature that took rustc developer 5 years to do" is…really strict and basically saying "none may pass"-style goalposts along this road.

> but as a user I yearn for a simple "gccrs version X matches the features of rustc version Y".

I sympathize, but I've also been in this field long enough to know that such things are usually hiding Lovecraftian contortions to make such a facade.

Compiling Rust with GCC: an update

moltonel — Mon, 12 Sep 2022 17:10:24 +0000

> You do have CI set up for your MSRV, right?

*I* do, but I'm pretty sure that's not universally the case. The point is not that the issue can be tested for, but that there is a new issue to be aware of. Also, I'm willing to bet that some developers will only setup their CI with gccrs, for philosophical reasons or to reduce resource usage.

Gccrs could probably prevent the issue by adopting rustc's #[feature(...)] system, but feature names are a bit implementation-specific, it's not clear how to make that work well with multiple implementations.

> unlikely to declare "supports Rust 1.X" without supporting every (compiler feature) that X implies

That was my understanding so far. But the implementation of const generics before they reached the 1.49 milestone casts a doubt. How will gccrs describe its current feature set ? "1.49 plus const generics as of 1.60 plus features X and Y" ? "1.60 except feature Z" ? Users will simplify that in their head to a single version number, and possibly write an optimistic MSRV to Cargo.toml.

To be fair, these feature differences are surmountable (maybe sites like caniuse.rs could start tracking gccrs and cg_gcc), and it's normal for gccrs to want to release features without exactly following rustc's release history, but as a user I yearn for a simple "gccrs version X matches the features of rustc version Y".

Compiling Rust with GCC: an update

mathstuf — Mon, 12 Sep 2022 14:37:29 +0000

> I don't want a scenario where I declare an MSRV of 1.49 but unknowingly use a 1.50 feature.

You do have CI set up for your MSRV, right? If you have an MSRV but only test/develop with stable, this is already a problem.

> Or a gccrs that announces 1.60 compatibility but ignores some "minor" 1.59 feature.

I think the gcc-rs developers are aware of such things and are unlikely to declare "supports Rust 1.X" without supporting every (compiler feature) that X implies. Or at least as far as the rustc test suite of 1.X is able to diagnose.

Compiling Rust with GCC: an update

moltonel — Mon, 12 Sep 2022 12:29:37 +0000

It's great to see the progress in both projects, kudos to all involved.

I'm a bit worried reading that gccrs is cherry-picking features (const generics and various kernel-needed features) far beyond their announced target (1.49). I don't want a scenario where I declare an MSRV of 1.49 but unknowingly use a 1.50 feature. Or a gccrs that announces 1.60 compatibility but ignores some "minor" 1.59 feature. Gccrs got a lot of backlash from users worried about language/ecosystem split, it should be very careful with its features roadmap.

Compiling Rust with GCC: an update

moltonel — Mon, 12 Sep 2022 12:04:43 +0000

You might also find surprising that gcc still doesn't support compiling for Apple M1 on MacOS. Gcc does support M1 on Linux, so this missing target triplet doesn't concern the kernel, but it's still an interesting counter point to the "Gcc supports more targets than LLVM" popular wisdom. aarch64-apple-darwin has orders of magnitude more users than all the Gcc-exclusive targets combined.

Compiling Rust with GCC: an update

paulj — Mon, 12 Sep 2022 11:54:02 +0000

That statement isn't true though. The risk is reduced, but not eliminated.

Compiling Rust with GCC: an update

mathstuf — Sun, 11 Sep 2022 14:02:46 +0000

> Well, the Dreamcast retro gaming scene doesn't run on Linux,

No, but Linux does run on the Dreamcast hardware due to a community effort (with recent updates!): http://linuxdc.sourceforge.net/

Compiling Rust with GCC: an update

josh — Sun, 11 Sep 2022 12:18:04 +0000

> Tier 3 means Rust's CI checks this compiles, but they don't check it works, and it is only supplied with the core library.

That's tier 2 (compiles, but isn't tested). Tier 3 is "this exists in the codebase, and might work, but you'll need to build it yourself".

https://doc.rust-lang.org/nightly/rustc/platform-support....

Compiling Rust with GCC: an update

tialaramex — Sun, 11 Sep 2022 10:39:54 +0000

Rust's tier support list says armv4t-none-eabi has tier 3. That platform is described as ARMv4 with Thumb and exists particularly to make the Nintendo Gameboy Advance work.

Tier 3 means Rust's CI checks this compiles, but they don't check it works, and it is only supplied with the core library.

Obviously the kernel is comfortable in that world, you can't just TcpStream::connect() from inside the kernel either, although it won't fit on a GBA as I understand it, presumably if you've got a big enough ARMv4 system to run Linux, Rust isn't a difficult problem.

Compiling Rust with GCC: an update

tialaramex — Sun, 11 Sep 2022 09:20:12 +0000

https://open-std.org/JTC1/SC22/WG14/www/docs/n3005.pdf is the current state of TS 6010, the draft Technical Specification which is what happened to N2676

So, modulo crazy ISO problems, the C23 standard per se won't mandate this roughly PNVI-address exposed model, but there will be an ISO document separately specifying how this would work. The standard is... rough. But there is limited enthusiasm for figuring out all the fine details while it remains unclear if everybody will even implement it. This only starts to make sense once at least two major compilers (e.g. MSVC and GCC) implement it.

With TS6010 you get most of the optimisations people expect in a modern compiler, and which of course Rust is doing, but you can do a XOR doubly linked list in C, as one example of stunt pointer manipulation that some people still think is a good idea. Of course some optimisations are given up in your doubly linked list code to make this work, but you don't feel that loss in unrelated code.

Compiling Rust with GCC: an update

joib — Sun, 11 Sep 2022 07:16:48 +0000

Well, the Dreamcast retro gaming scene doesn't run on Linux, does it, so not particularly relevant to the topic of which architectures Linux supports? Still kinda cool, I would have expected Dreamcast to be long gone and forgotten.

(Likewise the above-mentioned xtensa is ubiquitous in the 'maker scene' thanks to the ESP8266/ESP32 family of chips, but those don't run Linux so again not particularly relevant to this discussion. And as a side-note, it seems Expressif is transitioning to RISC-V cores.)

Compiling Rust with GCC: an update

k8to — Sun, 11 Sep 2022 04:39:04 +0000

When i was at Wind River in the late 90s SuperH was an alive and well dev board type that people were actively developing on. I'm sure the arch made its way into various devices. I have no idea which if them are potentially still alive, of course, but I suspect it's not a trivial amount.

At the same time, it was dwarfed by powerpc, x86, mips, arm, and even sparc.

Compiling Rust with GCC: an update

mjg59 — Sun, 11 Sep 2022 01:04:19 +0000

And it's sufficient for David A Wheeler's Double Diverse Compilation - if you build rustc with rustc and also build rustc with mrustc, and then run rebuild rustc with both of those outputs, if you get identical output then there's no backdoor (or alternatively both are backdoored, but that would mean whatever you built mrustc with is backdoored, and you can verify that through another round of DDC)

Compiling Rust with GCC: an update

khim — Sun, 11 Sep 2022 00:56:24 +0000

> Seems like some have reasons on why they want to make things in this area a bit more strict.

It's simple yet pretty useful psychological trick: if you ask people to follow needlessly strict rules but promise to relax them later 90% (if not 99%) is people would be happy with them. No matter what rules would you invent. And you can talk to the tiny subset of people who want more relaxed rules and try to make them happy, too. If you try to make the rules precise then you can never reach acceptance from everyone.

Heck, the whole Rust building is built on top of that approach: that's what separation of safe and unsafe Rust does.