|
|
Log in / Subscribe / Register

Cohen: gccrs: An alternative compiler for Rust

Arthur Cohen has posted a detailed introduction to the gccrs project on the Rust Blog, seemingly with the goal of convincing the Rust community about the value of the project.

Likewise, many GCC plugins are used for increasing the safety of critical projects such as the Linux kernel, which has recently gained support for the Rust programming language. This makes gccrs a useful tool for analyzing unsafe Rust code, and more generally Rust code which has to interact with existing C code. We also want gccrs to be a useful tool for rustc itself by helping pan out the Rust specification effort with a unique viewpoint - that of a tool trying to replicate another's functionality, oftentimes through careful experimentation and source reading where the existing documentation did not go into enough detail.

(LWN last looked at gccrs in October).


to post comments

Reflections

Posted Nov 8, 2024 17:09 UTC (Fri) by xtifr (guest, #143) [Link] (23 responses)

Wow, of course we need a second implementation! Has everyone forgotten Ken Thompson's famous paper, "Reflections on Trusting Rust"? :D

Reflections

Posted Nov 9, 2024 1:00 UTC (Sat) by sadlerap (subscriber, #153640) [Link] (1 responses)

This would be the third implementation of a Rust compiler. The mrustc compiler was the second Rust compiler on the scene and can currently bootstrap rustc up to and including v1.54.0.

https://github.com/thepowersgang/mrustc

Reflections

Posted Nov 9, 2024 3:29 UTC (Sat) by rsidd (subscriber, #2582) [Link]

Technically true but mrustc is not meant to be a full-fledged rust compiler. It is primarily for bootstrapping rustc (on platforms where there isn't an existing rustc binary, I guess). gccrs is meant to be a full-fledged alternative compiler.

Reflections

Posted Nov 11, 2024 8:35 UTC (Mon) by taladar (subscriber, #68407) [Link] (19 responses)

Historically second implementations have been incredibly harmful to language standards so it is extremely questionable if a at best theoretical solution to a theoretical attack would make up for all that harm.

Second implementations harmful to language standards

Posted Nov 11, 2024 11:55 UTC (Mon) by farnz (subscriber, #17727) [Link] (12 responses)

I question your premise; second implementations, IME, are only harmful if you permit the lowest common denominator implementation to influence the standard (which is where C goes horribly wrong). You don't get harm if you're willing to say that the second place (and subsequent) implementations have to keep up with the first place - as, for example, happens with Win32, ReactOS and WINE.

Second implementations harmful to language standards

Posted Nov 11, 2024 12:23 UTC (Mon) by pizza (subscriber, #46) [Link] (10 responses)

> You don't get harm if you're willing to say that the second place (and subsequent) implementations have to keep up with the first place - as, for example, happens with Win32, ReactOS and WINE.

Uh, from Win32's (and Microsoft's) perspective, the latter two are irrelevant. They don't factor into design or implementation decisions, and for all intents may as well not exist at all.

What you're _actually_ saying is that there is that the "standard" is "whatever the first &| official implementation does".

Everyone else has to be feature-for-feature (and bug-for-bug) compatible with that "official implementation", in perpetuity.

Second implementations harmful to language standards

Posted Nov 11, 2024 13:28 UTC (Mon) by farnz (subscriber, #17727) [Link] (9 responses)

Exactly. The existence (or otherwise) of a second implementation is orthogonal to the real problem, which is designing around the lowest common denominator of all implementations, rather than designing for one implementation and expecting the others to catch up.

The existence of a "blessed" implementation that defines the standard does not mean that other implementations are impossible; see also CPython versus Jython versus IronPython.

Second implementations harmful to language standards

Posted Nov 11, 2024 23:19 UTC (Mon) by dvdeug (guest, #10998) [Link] (7 responses)

Java defines ints to be 32 bit and longs to be 64 bit. If you want a language that defines every detail and forces the environment to adapt, there are many languages that will do that. C wants to be the best low-level language on the largest supercomputers to small microprocessors where Linux is too big to run. You can't do that if there's one implementation that everyone has to follow.

Second implementations harmful to language standards

Posted Nov 11, 2024 23:47 UTC (Mon) by intelfx (subscriber, #130118) [Link]

> C wants to be the best low-level language on the largest supercomputers to small microprocessors where Linux is too big to run. You can't do that if there's one implementation that everyone has to follow.

That conclusion makes no sense.

Second implementations harmful to language standards

Posted Nov 12, 2024 6:39 UTC (Tue) by mb (subscriber, #50428) [Link] (5 responses)

>C wants to be the best low-level language

Thanks. That made my day.

>on the largest supercomputers to small microprocessors where Linux is too big to run.
>You can't do that if there's one implementation that everyone has to follow.

yet, Rust does it successfully.

Second implementations harmful to language standards

Posted Nov 13, 2024 1:15 UTC (Wed) by dvdeug (guest, #10998) [Link] (4 responses)

> yet, Rust does it successfully.

Does it? There's good compilers for C to the MOS 6502 CPU (NES, etc.) The GitHub links for the LLVM Rust port to the MOS 6502 CPU says it's 23000 commits behind, and looks to be about nine months old, a problem for a fast moving target like Rust. In practice, does Rust really successfully handle the MOS 6502, or the PDP-11?

Maybe every modern CPU anyone cares about compiling to is big enough and generic enough that we can just run Java on it. But C and most C era portable languages didn't have that luxury of assuming all math is IEEE-754 and all characters are ASCII, much less Unicode, and everything can handle 64-bit integers.

Second implementations harmful to language standards

Posted Nov 13, 2024 5:23 UTC (Wed) by mb (subscriber, #50428) [Link]

>PDP-11

You'll always find an obscure architecture that's not supported (yet) and point at it.

Let's take a less obsolete example that's actually still relevant:
AVR-8

Yes, the compiler is also a couple of months behind and it's updated irregularly.

Does it matter? Yes.
Is it a problem? Not really.
The generated code quality is basically the same as avr-gcc and it's recent enough to support all relevant Rust features.

In fact, for some things Rust is even better on AVR-8 than C. Integer promotion, which doesn't exist in Rust, is a real pain on C AVR-8.

Rust (as a language) can scale very well from tiny to very big.

Second implementations harmful to language standards

Posted Nov 13, 2024 6:50 UTC (Wed) by intelfx (subscriber, #130118) [Link] (2 responses)

>>> C wants to be the best low-level language on the largest supercomputers to small microprocessors where Linux is too big to run. You can't do that if there's one implementation that everyone has to follow.
>>
>> yet, Rust does it successfully.
>
> Does it? There's good compilers for C to the MOS 6502 CPU (NES, etc.)

You have just conjured up a strawman. The fact that there happens to be a C implementation for $obscure_arch and no Rust implementation for $obscure_arch (yet, because nobody cared enough) does not prove your original point in any way.

There is nothing preventing the (single) Rust implementation to be extended to MOS 6502, or the PDP-11, or whatever-have-you, if someone happens to care enough to do the work.

Second implementations harmful to language standards

Posted Nov 13, 2024 12:19 UTC (Wed) by dvdeug (guest, #10998) [Link] (1 responses)

> You have just conjured up a strawman. The fact that there happens to be a C implementation for $obscure_arch and no Rust implementation for $obscure_arch

I noticed you didn't offer an alternative 8 bit architecture. I suspect the MOS 6502 CPU runs some of the most common targets for new commercial software, after x86-64 and ARM. People do continue to make and sell Commodore 64 and NES games, after all.

If you want to exclude obscure architectures, then yes, I'm sure Rust runs on ARM and x86-64. Woot. CPU design had converged enough by the 1990s that using LLVM and GCC, it can run on most general purpose 32 bit and 64 bit CPUs supported by Linux. That's more limited than C, though.

Second implementations harmful to language standards

Posted Nov 13, 2024 18:51 UTC (Wed) by mb (subscriber, #50428) [Link]

>That's more limited than C, though.

Please stop building yet another straw man.
Yes, Rust is more limited in that nobody has implemented support for your favorite obsolete architecture, yet.

You can add an implementation for your favorite architecture that you deeply care about. That is not a *fundamental* problem with the language itself.

Please remember what *exactly* I commented on:

>>on the largest supercomputers to small microprocessors where Linux is too big to run.
>>You can't do that if there's one implementation that everyone has to follow.

Especially this part:

>>You can't do that

And the answer to that is: Yes, you can.
I even gave an example of an extremely tiny architecture that is very well usable with Rust (Tier 3 support).
I run Rust code on an AtTiny 26. Which has 128 Bytes of RAM (no, the "kilo" is not missing) and 2 kBytes Flash (= 1024 ASM instructions).
And you can probably go to even smaller variants of that family.

How small does it need to be to show you that Rust (the language) is capable of being used on extremely small and also extremely large architectures?

We do *not* need to support *every* big and small architecture.
We just have to support a couple to show that the language is capable to do these kind if scaling.

Second implementations harmful to language standards

Posted Nov 12, 2024 8:40 UTC (Tue) by Wol (subscriber, #4433) [Link]

> Exactly. The existence (or otherwise) of a second implementation is orthogonal to the real problem, which is designing around the lowest common denominator of all implementations, rather than designing for one implementation and expecting the others to catch up.

Which is why you really need something ABOVE the specification, which dictates what the specification is trying to specify. For example, while I don't think Rust has a formal one, it's clearly "Outside of 'unsafe' blocks, there is no UB, and your bog-standard average programmer should be able to spend their entire career without worrying about 'unsafe' ".

Which in important respects makes it easy to say "the Rust spec is defective and needs to be fixed".

Cheers,
Wol

Second implementations harmful to language standards

Posted Nov 14, 2024 22:44 UTC (Thu) by bartoc (guest, #124262) [Link]

The problem is that people start using the alternate implementations, and start depending on them continuing to work. This inherently limits how the primary implementation can evolve and puts pressure on everyone to be compatible with each-other and with some "official" standard. This sounds great but to evolve a standard you need someplace to implement proposed features where you'll get user experience, and the pressures to be compatible basically mean either the standards body will have no real power or whoever implements experimental features will forever be stuck with them as tech debt.

It's extremely dysfunctional vs an open source model, but to some extent it's the destiny of all successful languages to become dysfunctional like this. That's what you get when you have users and all changes are very global.

Reflections

Posted Nov 12, 2024 3:26 UTC (Tue) by dannyobrien (subscriber, #25583) [Link] (5 responses)

Can you expand a little on the idea that multiple (or is it just second?) implementations are harmful to standards. I think I understand what you're saying, but the opposite can also be said to be true -- that a standard isn't really a standard unless there are multiple interpretations.

Reflections

Posted Nov 12, 2024 10:33 UTC (Tue) by taladar (subscriber, #68407) [Link] (4 responses)

A second (or more, the second isn't special) implementation makes any change to the language significantly harder to decide on in the first place and to coordinate the actual implementation.

You also inevitably get different behavior, even if all implementations try very hard to implement the same standard, so in practice people will start using some sort of compile time detection (or just set it manually) of the implementation to run different code on different implementations.

Then, when it comes to making changes to the standard people will argue implementation A behavior should be standardized because their code uses that one and they don't want to change their entire code base and other people will argue implementation B behavior should be standardized because their code uses that one and they also don't want to change their entire code base and then some "compromise" is put into the standard that allows both behaviors.

Essentially once you have a second implementation further language development is severely impaired and every library author has to deal with testing against multiple implementations for very little gain.

Reflections

Posted Nov 12, 2024 16:28 UTC (Tue) by mathstuf (subscriber, #69389) [Link] (3 responses)

I'm not so sure. Without implementation experience, it is very hard to standardize large/complex features. C++ (FD: I'm on the standards committee) now vastly prefers to have actual implementations of proposals instead of just discussing abstract wording. I feel that this has vastly improved the state of merged features. Alas, the last one to "slip through" without major implementation experience is the one I'm most involved in (modules). Tooling is still catching up…

And once one wants implementation experience, it can be hard to have everyone working on one implementation. Prior implementation design decisions can ossify future language possibilities. I know there are features which have been "easier" in some C++ compilers than others and, while some might argue fewer features would improve C++, selecting a "prime" implementation may not overlap well with the set of features that are actually available.

Reflections

Posted Nov 12, 2024 18:02 UTC (Tue) by Wol (subscriber, #4433) [Link] (2 responses)

> I know there are features which have been "easier" in some C++ compilers than others and, while some might argue fewer features would improve C++, selecting a "prime" implementation may not overlap well with the set of features that are actually available.

Can't you copy Rust "Editions" :-)

Provided you can get people to agree on your prime implementation - always a big ask! - just bundle all the features you're getting rid of behind a "--deprecated-" flag, and require the compiler to spit out a diagnostic if any of them are actually used.

So they remain available, but anybody using them can't avoid noticing they might go away, and is encouraged towards stuff that doesn't spew out all these annoying messages :-)

Cheers,
Wol

Reflections

Posted Nov 12, 2024 19:25 UTC (Tue) by atnot (guest, #124910) [Link]

> Can't you copy Rust "Editions"

There was a good blog post on this which I can't find due to search engine shittification, but the summary is no, or at least not easily.

But from memory, one reason is textual includes, macros and templates, which need to continue to work for backwards compatibility, and make it very ambiguous what edition to use for what code. Modules improve this somewhat nut there's still some issues, I don't remember the details. Another is the small compilation unit size combined with C++'s stable ABI, which makes it very hard to attain any benefits. A further problem is that cargo usually being able to instantly upgrade your source code without breaking anything is an essential part of what makes edition changes seamless in Rust, which is much more difficult in C++ both because it's harder to refactor automatically and no official organizational capacity exists to write such a tool. Then there's the issue that even in Rust, edition changes are limited to relatively simple syntax tree rewrites and stuff like turning warnings into errors. This still enables one to do a lot, but only in combination with other language features, which doesn't apply cleanly to C++ and would need to be rethought.

So it definitely could be done, but it's far more difficult, and more importantly there's mo obvious path towards the kind of improvements people want from C++ using this approach, compared to the effort it would require.

> Provided you can get people to agree on your prime implementation - always a big ask! - just bundle all the features you're getting rid of behind a "--deprecated-" flag.

This effectively already exists in the form of the various warnings compilers emit, but those are not standardized of course and wouldn't be enough anyway. But as you say, the real problem is getting a very broad consensus for something like this in an organizational structure where consensus is difficult to obtain, because it's just not the way things are done by default.

Reflections

Posted Nov 12, 2024 19:31 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

Rust's editions are essentially different "spellings" of the same language. For example, one edition can reserve a keyword while another edition lets it be an identifier. But the underlying language that is being accessed is the same (e.g., a type cannot implement a trait based on the compiled edition as "does X impl T?" needs to be the same regardless of the querying context's edition). The problem with adapting this to C++ is that the specific spelling can be part of the API contract. For example, templates are essentially macros at some level in that they are not checked for correctness until instantiation is attempted and the specific spelling used can matter quite a bit. Couple this with SFINAE[1] where one can ask "is this template instantiation valid?" and change behavior based on the answer, the set of things that one can actually change under such a C++ edition is minuscule.

At least that's my understanding of being in the room when the initial "let's bring editions to C++" paper was presented; I don't think things have changed much since then on that front.

Note that profiles uses a similar technique, but it works instead by *banning* the use of various spellings rather than allowing new spellings to have new meanings.

[1] https://en.wikipedia.org/wiki/SFINAE

Reflections

Posted Nov 11, 2024 18:40 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

> "Reflections on Trusting Rust"

Groan.

Rust on Dreamcast

Posted Nov 8, 2024 23:32 UTC (Fri) by darcagn (subscriber, #168667) [Link] (4 responses)

Wow, I'm so happy to see Arthur shout-out to the Sega Dreamcast scene! So cool! Late last year I got gccrs building code for the Sega Dreamcast, though I'm using rustc_codegen_gcc now since it's further along. I've got the `std` up and running on the Dreamcast homebrew community's operating system, KallistiOS. Alternative compilers will be critical to Rust's success, and I'm so thankful to people like Arthur who are working on these implementations.

Rust on Dreamcast

Posted Nov 9, 2024 12:31 UTC (Sat) by khim (subscriber, #9252) [Link] (3 responses)

> Alternative compilers will be critical to Rust's success

Isn't the first part of your message contradicts the second?

I absolutely agree that different backends are nice thing to have, there are already three (or four? LLVM-based, GCC-based, Cranelift and I think backend for common language runtime is it's own special thing).

And “Reflections on Trusting Rust” is covered by mrustc.

This only leaves gccrs with very weak justifications for it's existence. Almost no sensible justification at all, I would say.

I'm not proposing to stop that effort, I'm all for people to spend their own time and money on anything they like… I'm just honestly not sure what makes it valuable.

P.S. Justification of the form rustc_codegen_gcc requires rustc to be available in the first place doesn't sound convincing to me: that's the exact same situation as with gnat, isn't it? Why don't we rush to create an alternative for gnat but need an alternative for rustc?

Rust on Dreamcast

Posted Nov 10, 2024 18:49 UTC (Sun) by willy (subscriber, #9762) [Link] (1 responses)

There are many Ada compilers in the world. I presume they can compile gnat.

Rust on Dreamcast

Posted Nov 10, 2024 19:05 UTC (Sun) by khim (subscriber, #9252) [Link]

> I presume they can compile gnat.

And you presume wrongly:

In order to build GNAT, the Ada compiler, you need a working GNAT compiler (GCC version 5.1 or later).

This includes GNAT tools such as gnatmake and gnatlink, since the Ada front end is written in Ada and uses some GNAT-specific extensions.

From what I understand you can bootstrap GNAT in the same way you can bootstrap rustc (start with ancient version of GNAT, build slightly newer version, repeat) – and yet that haven't spurred the need to, suddenly, create an alternate compiler.

Need I remind you that Ada is used in many security-critical avenues like programs for airplanes and military… why, then, would something that is totally not an issue for Ada justify the gccrs?

Rust on Dreamcast

Posted Nov 18, 2024 8:17 UTC (Mon) by ssokolow (guest, #94568) [Link]

(or four? LLVM-based, GCC-based, Cranelift and I think backend for common language runtime is it's own special thing)
You think correctly. rustc_codegen_clr is a backend in its own right. Its author is blogging about progress on it at https://fractalfir.github.io/

Reusing of rustc parts and bootstrapping

Posted Nov 9, 2024 2:39 UTC (Sat) by pothos (subscriber, #116075) [Link] (7 responses)

The reusing of some rustc parts like Polonius borrow checking sounds interesting but also makes me wonder how bootstrapping works then.

Is this reuse only limited to optional things such as borrow checking which can be skipped in bootstrapping, so that you would compile gccrs twice, first without borrow checking to get a bootstrap Rust compiler and then again with borrow checking to get a proper Rust compiler?
While this seems like a nice separation, the upstream rustc project might not always organize itself in a way such that non-codegen-critical parts are easily reusable - but for getting an gccrs MVP out any shortcut is a big plus…

Reusing of rustc parts and bootstrapping

Posted Nov 9, 2024 3:07 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]

> The reusing of some rustc parts like Polonius borrow checking sounds interesting but also makes me wonder how bootstrapping works then.

You don't need the borrow checker to compile Rust. Without a borrow checker, your compiler might accept incorrect programs, but that's it. So it's enough for bootstrapping the borrow checker.

Reusing of rustc parts and bootstrapping

Posted Nov 9, 2024 10:41 UTC (Sat) by matthias (subscriber, #94967) [Link] (1 responses)

The blog clearly says that they also consider reusing other modules like the trait solver. A module which is clearly necessary for codegen.

So most likely you will need a working rust compiler to bootstrap gccrs if they follow this route. This can be rustc or rustc_codegen_gcc. And if you want to port to a new architecture, you can use gccrs to cross-compile gccrs for the new architecture. mrustc might not have all the features required to bootstrap gccrs.

Reusing of rustc parts and bootstrapping

Posted Nov 10, 2024 0:23 UTC (Sun) by Tobu (subscriber, #24111) [Link]

mrustc seems like a good solution from where I stand? If it can bootstrap rustc (and it is periodically updated for that) it could bootstrap gccrs.

Reusing of rustc parts and bootstrapping

Posted Nov 9, 2024 12:26 UTC (Sat) by josh (subscriber, #17465) [Link] (2 responses)

I would expect that eventually building gccrs will require a previous version of gccrs.

Reusing of rustc parts and bootstrapping

Posted Nov 9, 2024 19:47 UTC (Sat) by Karellen (subscriber, #67644) [Link] (1 responses)

Looking at the files in https://github.com/Rust-GCC/gccrs/tree/master/gcc/rust/ and it's subdirectories, they mostly appear to be .cc/.h files, so it looks like gccrs should only need a C++ compiler to build. Given that the rest of gcc needs a C++ compiler to bootstrap, this makes bootstrapping gccrs no harder than any other tool in the project.

Reusing of rustc parts and bootstrapping

Posted Nov 9, 2024 19:50 UTC (Sat) by josh (subscriber, #17465) [Link]

From my comment:

> I would expect that eventually

Bootstrapping gccrs doesn't require gccrs *today*; I expect that it one day will, when it uses non-optional components written in Rust. This would make it comparable to other GCC frontends that include components written in the language the frontend compiles.

Reusing of rustc parts and bootstrapping

Posted Nov 12, 2024 8:17 UTC (Tue) by CohenArthur (subscriber, #173796) [Link]

We wrote a separate blogpost which covers this question and is more technical: https://rust-gcc.github.io/2024/09/20/reusing-rustc-components.html

Yes, you are correct that we will be either skipping some passes (like borrow-checking) when bootstrapping or reimplementing enough of the pass to use it (like trait-solving). This is still a huge gain in speed, as writing a trait-solver which is 95% complete takes a *lot* less time than writing one which is 99% complete.


Copyright © 2024, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds