Progress toward a GCC-based Rust compiler

Posted Dec 17, 2023 1:10 UTC (Sun) by roc (subscriber, #30627)
In reply to: Progress toward a GCC-based Rust compiler by Phantom_Hoover
Parent article: Progress toward a GCC-based Rust compiler

Reimplementing the front-end is likely to expose bugs where the rustc frontend does something unintended.

I think it will also be useful for bootstrapping.

Progress toward a GCC-based Rust compiler

Posted Dec 17, 2023 2:28 UTC (Sun) by atnot (subscriber, #124910) [Link] (11 responses)

It doesn't really change the bootstrapping story much, given that mrustc exists, which is a C++14 compiler that compiles valid rust to bad C11. So depending on whether you're okay pre-generating those sources, it is either around equally as easy or significantly easier to bootstrap than gccrs would be.

Progress toward a GCC-based Rust compiler

Posted Dec 18, 2023 1:20 UTC (Mon) by mathstuf (subscriber, #69389) [Link] (10 responses)

`mrustc` is stuck on bootstrapping 1.39 last I heard (the primary author went off to complete their degree or something). Note that its "borrow checker" implementation is "I'm sure it's fine".

Progress toward a GCC-based Rust compiler

Posted Dec 18, 2023 2:13 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

mrustc can bootstrap 1.54

Progress toward a GCC-based Rust compiler

Posted Dec 18, 2023 2:13 UTC (Mon) by tialaramex (subscriber, #21167) [Link] (8 responses)

"I'm sure it's fine" is of course completely reasonable - for a bootstrap compiler. If it's not fine then our final recursive build will discover that and fail. This is no sort of way to do routine development, but that's not what a bootstrap compiler is for.

Progress toward a GCC-based Rust compiler

Posted Dec 19, 2023 14:05 UTC (Tue) by rrolls (subscriber, #151126) [Link] (7 responses)

I was considering the question "is it safe for a bootstrap Rust compiler to not include a borrow checker" before I even read the comments, so it's pleasing to see this discussed here.

Surely the approach of "the final recursive build will discover [any invalid borrows] and fail" isn't 100% safe, because if an invalid borrow exists it's _possible_ (unlikely but possible) that it creates an edge case that allows exactly that invalid borrow through the borrow checker that was just compiled, right?

I'd describe it as 99% safe, since in practice it'd be much more likely that an invalid borrow in the bootstrap compiler would cause some noticeable effect somewhere else and _not_ prevent its own discovery... but I think for 100% safety, you'd have no option but to write the borrow-checker in a lower-level, already-proven language.

Do correct me if I'm wrong, though!

Bootstrapping and borrow-checking

Posted Dec 19, 2023 14:49 UTC (Tue) by timon (subscriber, #152974) [Link]

I don’t think you’re wrong, but I want to add another idea:

Being borrow-checked is more a property of the input (source code) rather than the output (binary), so I’d say you can reasonably treat it as orthogonal to your bootstrap chain.

The problem of having a not-borrow-checked borrow-checker is similar to the “trusting trust” problem. For countering the trusting trust problem, you can do “diverse double-compiling” [1].

I woud propose “borrowing borrow-checkers” as an even better countermeasure here. Just throw your bootstrap compiler Rust source at as many borrow-checkers as you can find, and optimally they should all agree whether your borrows are fine.

[1]: https://dwheeler.com/trusting-trust/

Progress toward a GCC-based Rust compiler

Posted Dec 19, 2023 15:17 UTC (Tue) by farnz (subscriber, #17727) [Link] (1 responses)

The bootstrap compiler can only be safely used on sources that have been built successfully by a compiler on another platform that never went through a stage without a borrow checker; that way, if there is a bug caused by an invalid borrow (noting that borrow checking is purely a safety net - it does not affect codegen at all), the pre-existing compiler on another platform will have found it.

Progress toward a GCC-based Rust compiler

Posted Dec 19, 2023 16:36 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

Note that the borrow checker itself is not part of the language, but of the implementation. Rust's rule is "shared XOR mutable"; the borrow checker is an inexact implementation of an enforcement for this. It has been improved over time from the original lexical borrow checker to non-lexical lifetimes to polonius in the future. I presume that a runtime could also enforce it (at the cost of runtime overhead). In fact, I wonder if valgrind could be enhanced to verify this for any given program at runtime…that would give any implementation more trust in its enforcement of the rule.

Progress toward a GCC-based Rust compiler

Posted Dec 19, 2023 16:37 UTC (Tue) by steveklabnik (guest, #114343) [Link] (2 responses)

Borrow checking does not affect codegen in any way. As long as the code runs successfully under a borrow checker, a compiler without one can compile it with zero issues. As rustc has been checked by rustc's borrow checker, mrustc will (modulo other bugs, of course) not miscompile it.

At one point mrustc produced a byte-identical compiler to rustc, I am not sure if that's a one-time thing or if they always do that, though.

Progress toward a GCC-based Rust compiler

Posted Dec 20, 2023 8:20 UTC (Wed) by rrolls (subscriber, #151126) [Link] (1 responses)

Aha, so basically, the solution "borrow-check it with rustc, then compile it with whatever" effectively _is_ the solution "write the borrow-checker in a lower-level language". (Since IIRC, each rustc is built with an older rustc, but at some point you'd get back to a version of rustc that wasn't written in Rust.)

One more complication, then. Is "safety" _purely_ a property of the source code, or does it depend on the environment it's being compiled in as well? For example, you use rustc to borrow-check your source code on a typical consumer-grade x86-64 linux platform, and now you compile the same code on some esoteric architecture and operating system. Presumably, the environment can affect, for example, how generics get instantiated, which might turn something that was deemed safe on the first platform into something that's not fine at all on the second? Or does Rust insist that code is only deemed safe if every possible instantiation is safe? I suppose you could request cross-compilation in your borrow-check step, which might address this, but it does add some complication.

Progress toward a GCC-based Rust compiler

Posted Dec 20, 2023 13:13 UTC (Wed) by farnz (subscriber, #17727) [Link]

The part of safety that the borrow checker is responsible for is purely a property of the source code; it confirms that (for example) that the shared XOR mutable invariant is maintained by safe Rust.

Anything which can't be machine-checked at the source level as maintaining the safety invariants is supposed to marked with unsafe, which indicates that the human programmer is responsible for checking that things are safe. This means that anything outside Rust source (such as machine-specific primitives) should be marked as unsafe, and it's on the humans writing a safe wrapper to ensure that the wrapper maintains invariants at all times.

Rust does insist that all instantiations are either safe, or marked appropriately with unsafe; additionally, the caller of an instantiation of a generic that's marked with unsafe must mark their calling block with unsafe to indicate that they're OK with this. You cannot have an unsafe instantiation of a safe generic in safe code - you need the markers to tell Rust that you've thought about this and are going to uphold the language invariants, even if the compiler can't check your working (which is true of platform interfaces, for example, where the compiler can't check the platform behaviour).

Progress toward a GCC-based Rust compiler

Posted Dec 21, 2023 18:55 UTC (Thu) by dvdeug (guest, #10998) [Link]

There was a GCC version that had a particular proprietary C compiler noted in its documentation because that compiler had a bug in its compiling of GCC that made certain optimizations NOPs. Thus a three stage bootstrap of GCC would produce a slightly broken first stage, and an unoptimized but correct second-stage, which would break bootstrap because it wouldn't match the optimized third stage.

So it's conceivable that a bad bootstrap compiler could break the borrow checker in rustc. It'd be unlikely to carry over to next generations, though.