LWN: Comments on "Committing to Rust in the kernel"

Building Rust-for-Linux on stable Rust

pizza — Wed, 02 Oct 2024 12:12:06 +0000

> Seems to me that C is a pretty immature language if you need a 9 year old compiler because some platforms are so reluctant to update that unstable mess to a newer version that they are almost 10 major versions behind the current version.

For Linux, said 9-year-old compiler is the *minimum* required version, not the *only* version that is supported/expected to work.

(I am typing these words on a system with the most recent Linux release, compiled using the most recent GCC release)

> Seems if you try even a little bit portraying the facts to make the process the other side uses look bad is actually pretty easy.

Another thing that makes you look bad is deliberately mis-stating documented facts.

Building Rust-for-Linux on stable Rust

taladar — Wed, 02 Oct 2024 10:57:17 +0000

Seems to me that C is a pretty immature language if you need a 9 year old compiler because some platforms are so reluctant to update that unstable mess to a newer version that they are almost 10 major versions behind the current version.

Seems if you try even a little bit portraying the facts to make the process the other side uses look bad is actually pretty easy.

Positive vibes

dmv — Tue, 01 Oct 2024 22:04:05 +0000

Makes me wonder how many behind-the-scenes conversations were had to arrive at the positivity.

As an onlooker, the basic idea that the kernel is developed out in the open is an attractive notion, but it’d be really interesting to have the curtains pulled back a bit to see how much actually is somewhat behind-closed-doors/out of the public eye. I would assume some of it would have to be. Given that companies have kernel developers on staff, some of that stuff is gonna be company-internal, of course. But I’m interested less in that part of it. The kernel development project is its own massive social experiment of sorts. Surely human-sensitive things are easier to manage out of the public eye. What else?

Building Rust-for-Linux on stable Rust

knewt — Fri, 27 Sep 2024 16:59:40 +0000

>> Kernel docs says minimum version is GCC v. 5.1, but it varies depending on architecture.

In coincidental timing, seems that's getting bumped up as of 6.13! This patch is in the pipeline as of recently:

========
From: Ard Biesheuvel <ardb@kernel.org>

Bump the minimum GCC version to 8.1 to gain unconditional support for
referring to the per-task stack cookie using a symbol rather than
relying on the fixed offset of 40 bytes from %GS, which requires
elaborate hacks to support.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
========

Building Rust-for-Linux on stable Rust

knewt — Fri, 27 Sep 2024 16:55:44 +0000

>> “Obviously recently ported hardware would require a newer GCC so take that with a grain of salt.” Obviously? Not at all to me! Can you elaborate on how you came to this conclusion? (Also it looks like you missed word “driver” along the lines)

I imagine the thought here is that newer processor architectures will have only gotten GCC support more recently, and as such only in newer versions of GCC, which feels reasonable.

Building Rust-for-Linux on stable Rust

andy_shev — Fri, 27 Sep 2024 12:26:15 +0000

“Obviously recently ported hardware would require a newer GCC so take that with a grain of salt.” Obviously? Not at all to me! Can you elaborate on how you came to this conclusion? (Also it looks like you missed word “driver” along the lines)

Positive vibes

Wol — Thu, 26 Sep 2024 20:06:51 +0000

> > There is a need for documentation and tutorials on how to write filesystem code in idiomatic Rust.

I'm pleased to see he wrote "documentation AND TUTORIALS".

Never forget, without tutorials, documentation is usually written in double dutch ... (says he who's just been reading a load of documentation and wondering what the hell it's going on about ...)

Cheers,
Wol

Unstable compilers

ralfj — Thu, 26 Sep 2024 20:04:42 +0000

> In Rust, undefined behaviour is viewed as a bug in the language (or as "you can only do that in an unsafe block") as far as I can tell. Completely diametric views on UB.

Yes -- mostly the latter. I think Undefined Behavior is a great tool for language designers; they can use it to enable powerusers to get the compiler to do things you'd never (well, not practically) get it to do without. But it is the language designer's responsibility to yield this tool with care.

A more lengthy discussion of this point is in my blog: https://www.ralfj.de/blog/2021/11/18/ub-good-idea.html

In C, things can be UB either because the standard says they are UB, or because the standard is entirely silent about that case. The latter is considered a spec bug in Rust, it is not acceptable for us to have things be "implicitly" UB. The former still happens, but we are doing our best to describe those cases as unambiguously as possible. Rust doesn't have a spec yet, so we still have a lot of work ahead of us, but Miri [1] helps a lot: if we understand our UB rules well enough to wire them into a tool that can automatically test for UB, that's a great first step towards an unambiguous specification! I think something like Miri would be nearly impossible to do for C, since the standard is just not sufficiently clear about many corner cases.

[1]: https://github.com/rust-lang/miri/

Positive vibes

pbonzini — Thu, 26 Sep 2024 19:29:43 +0000

Well, if Ted went from expressing frustration to:

> There is a need for documentation and tutorials on how to write filesystem code in idiomatic Rust. He said that he has a lot to learn; he is willing to do that, but needs help on what to learn.

Then it's a lot more than just "two data points".

RHEL Support

arnd — Thu, 26 Sep 2024 12:29:59 +0000

I also see hexagon and xtensa in the tier 3 list of the rustc manual, leaving only 7 architectures with no support at all, which are also the ones with very few users these days:

alpha, parisc and superh are old, with the latest CPU cores dating back to 2005, any work going on here is driven by hobbyists, but m68k shows that this may still happen.

arc, microblaze, and nios2 are still available for new designs but most users have moved on to risc-v here, and I don't expect anyone to put money into a rustc port. Nios2 and microblaze are likely to get phased out from the kernel in the next few years. Openrisc is somewhere between those, as a softcore that is not owned by a single company.

Unstable compilers

admalledd — Wed, 25 Sep 2024 18:29:16 +0000

Certainly not, but the stability of `rustc` is often much higher than many give it credit for, and that Rust gives much better ABI(*)/linking guarantees. It is often not plausible to use a different GCC version from what compiled the kernel to compile custom modules for example. Rust can provide much stronger guarantees here by default, though as mentioned in the article there is in-kernel work to be done for the module symbol naming/linking.

Rust in *most* cases will either halt compilation with an Error, and InternalCompilerError, or result in symbols that will fail at linking time instead of (used to be more) commonly "compiling" fine but crashing/failing at runtime or `insmod` time.

* Rust technically has no ABI, but you can expose/use one via various exports/macro things, commonly of course a "c-abi". This is mostly handled by `rust-bindgen` for automation with a dash of human control when required.

Unstable compilers

sunshowers — Wed, 25 Sep 2024 18:10:44 +0000

Yes, that is correct, and Rust is better for it. Under the assumption that unsafe code is correct, safe code doesn't have UB.

I would rather optimization 'alpha' be extracted through principled approaches like enabling the use of "restrict" on &mut pointers.

Unstable compilers

ballombe — Wed, 25 Sep 2024 17:56:59 +0000

> You could (especially as you approach the modern era) still often *compile* with whatever GCC (and sometimes CLANG) version so long as it was above some horribly old bare minimum, but it was not uncommon to have subtle miscompilations in more esoteric drivers, or flat out some drivers not compiling, etc.

Because all rust compilers are guaranteed to be bug-free ?

Building Rust-for-Linux on stable Rust

ojeda — Wed, 25 Sep 2024 16:56:43 +0000

To clarify: we have never recommended nightly or beta compilers (specific or not). We have always supported stable (released) compilers only, until v6.11.

Since v6.11, we support a minimum Rust version, currently 1.78.0. Thus 1.79, 1.80 and 1.81 work too. Beta and nightly compilers should also generally work now.

The Rust project and bindgen have now the kernel in their pre-merge CI. This means that unintentional changes to unstable features in Rust that break Linux do not get merged into Rust.

The #2 issue (and the sublists) definitely helps as you say (and it is one of the reasons why I started it in GitHub, since it cross-references issues), but having the CI is even better. Upstream Rust also has a tag for us, A-rust-for-linux, that we are starting to use.

This has also allowed us to cover the toolchains of a few Linux distributions too.

Please see https://rust-for-linux.com/rust-version-policy, as well as the Kernel Summit talk from LPC a few days ago (the stream/video is not available yet, as far as I understand).

Building Rust-for-Linux on stable Rust

intelfx — Wed, 25 Sep 2024 14:15:56 +0000

> The response to the "Rust tooling is still immature and doesn't cover all use cases that some folks need" *fact* should be (and according to the _actual_ R4L devs, is) "We're still working on it; we'll get there", not "waah different standards, give us special treatment instead" or "those use cases don't matter" of the _very_ annoying fanbois that keep coming out of the woodwork.

Now this is a case of shifting the goalposts.

The response is, and always was, "We're still working on it and we'll get there". Nobody has ever said "waah different standards" while asking for "special treatment". The problem with different standards is merely that certain vocal people want "getting there" to happen in zero time. C did not get there in zero time, so why does Rust have to?

Unstable compilers

tialaramex — Wed, 25 Sep 2024 12:47:54 +0000

Take for example 128-bit integer support. Rust has 128-bit integers. Code to handle these e.g. formatting them for output as text - exists but in a typical Rust application which doesn't actually have any 128-bit integers the code for them never ends up in the resulting binary.

But, Linux currently just says nope, we'll take everything just in case, because a loadable module might use it. A few kilobytes of formatting for types we never use? Give me those too.

So for Stable Rust Linux would get a bunch of 128-bit integer formatting code it has no use for, on the off chance that somebody might some day write a loadable module which expects 128-bit integer formatting to work. In Rust for Linux they've instead just made this code conditional and switched off the condition, it won't compile in your loadable module and the kernel avoids a few kilobytes of "format 128-bit integers" code.

The push back from the Rust community is they're not interested in doing a bunch of extra work to help people who want all the symbols but then wish somebody else would do the work to slim it down for them, pick a lane. So the Rust for Linux change will never land in the stable Rust in this form.

Positive vibes

Wol — Wed, 25 Sep 2024 12:16:54 +0000

The problem is, the squeaky wheel gets all the oil.

Cheers,
Wol

Positive vibes

pizza — Wed, 25 Sep 2024 12:05:41 +0000

> At the same time there's also more willingness to collaborate than what transpired from the infamous "you're not going to make us learn Rust" quote.

FWIW, sampling two points from from a locally-noisy source does not make for good generalizations.

Building Rust-for-Linux on stable Rust

pizza — Wed, 25 Sep 2024 11:56:26 +0000

> Now, that doesn't mean that then need to use bleeding version of Rust would stay with us forever, but I don't see why Rust have to be treated differently from how C was treated in the Linux bringup phase.

Because unlike Linus's original announcement of Linux 30-odd years ago, Linux is no longer a just-for-fun toy.

If Rust proponents claim that the quality standards/practices of yesteryear are wholly insufficient today then they can't claim "waaah we're being held to a higher standard than ysteryear" when the mirror is turned on them.

The response to the "Rust tooling is still immature and doesn't cover all use cases that some folks need" *fact* should be (and according to the _actual_ R4L devs, is) "We're still working on it; we'll get there", not "waah different standards, give us special treatment instead" or "those use cases don't matter" of the _very_ annoying fanbois that keep coming out of the woodwork.

Unstable compilers

Wol — Wed, 25 Sep 2024 10:42:10 +0000

> > it appears the Rust devs seem much more amenable to treating "you're optimising away necessary code" as a bug than the C folks were.

> The optimization backend in the end is the same, the difference is that the language itself is designed to make it harder to trigger undefined behavior.

Maybe that's the effect. But even today the C devs seem keen on creating new undefined behaviours. In Rust, undefined behaviour is viewed as a bug in the language (or as "you can only do that in an unsafe block") as far as I can tell. Completely diametric views on UB.

Cheers,
Wol

RHEL Support

sam_c — Wed, 25 Sep 2024 10:28:56 +0000

What ends up being problematic for us is that only certain tiers get "host tools" built which means we have to build our own binaries for distributing Rust. sparc64 is an example.

Building Rust-for-Linux on stable Rust

sam_c — Wed, 25 Sep 2024 10:03:22 +0000

Replying to both you and asahilina: thanks. I'm pleased to hear that there's a strong presumption against adding more such cases especially.

Unstable compilers

pbonzini — Wed, 25 Sep 2024 09:52:47 +0000

> it appears the Rust devs seem much more amenable to treating "you're optimising away necessary code" as a bug than the C folks were.

The optimization backend in the end is the same, the difference is that the language itself is designed to make it harder to trigger undefined behavior.

For example the level of aliasing is declared explicitly by separating shared and exclusive references (& and &mut). Developers can only turn a shared reference into an exclusive one via UnsafeCell (and then the code needs to be explicitly marked as unsafe) or wrappers thereof (which are carefully designed to avoid undefined behavior).

Unstable compilers

Wol — Wed, 25 Sep 2024 09:46:29 +0000

> I guess I'd make the distinction between new features only available in and bugginess though.

You need to add to that list "new features either enabled by default, or often enabled by force, that break the kernel".

One only has to look at all the complaints about UB, where gcc was optimising code away. Quite often (a) this stuff was NEEDED, and (b) there was no flag to disable the optimisation responsible for removing them.

So the compiler wasn't buggy, it was working as designed. And if the die-hards are going to complain and say "well Rust needs to call C to access the hardware", it was not at all unusual for C to have to call assembler to access the hardware, because the C compiler was just deleting the code as UB with no way to stop it. Hence my comments in various places about mathematics is not reality.

The difference is, it appears the Rust devs seem much more amenable to treating "you're optimising away necessary code" as a bug than the C folks were.

Cheers,
Wol

Unstable compilers

sam_c — Wed, 25 Sep 2024 09:32:40 +0000

I meant to add: ... because you can more easily backport bug fixes.

Unstable compilers

sam_c — Wed, 25 Sep 2024 09:32:13 +0000

Thanks. I guess I'd make the distinction between new features only available in and bugginess though.

RHEL Support

farnz — Wed, 25 Sep 2024 08:44:53 +0000

As a really quick summary, only x86-32, x86-64 and AArch64 are in Tier 1. Tier 2 brings in AArch32, LoongArch, RISC-V, PowerPC, SPARC64 and WASM. Tier 3 then brings in AVR, C-SKY, MIPS, SPARC32, S390x (but not plain S390) and 68k.

Very roughly, the tiers are:

Tier 1 blocks merging of PRs if any builds or tests fail in CI.
Tier 2 blocks merging of PRs if any builds fail in CI; tests are allowed to fail.
Tier 3 simply has one or more people promising to work on fixes if it fails to build in CI.

Positive vibes

pbonzini — Wed, 25 Sep 2024 08:40:38 +0000

At the same time there's also more willingness to collaborate than what transpired from the infamous "you're not going to make us learn Rust" quote.

Building Rust-for-Linux on stable Rust

asahilina — Wed, 25 Sep 2024 07:06:57 +0000

There is a backdoor in the stable compiler to enable nightly features. R4L uses this backdoor, and therefore works with stable Rust builds.

Since the features are still "nightly features" they can and do change, which can break the build with newer compiler versions, but the list keeps getting shorter and we try not to add any new required nightly features unless they are necessary or very important to achieve the R4L goals.

At some point all of the features will be promoted to stable, the backdoor will no longer be necessary, and R4L builds will be guaranteed not to break with newer compilers.

Unstable compilers

admalledd — Wed, 25 Sep 2024 05:16:59 +0000

TL;DR: Until "recent history" of about as recent as ten years ago, compiling the Linux Kernel had a very odd list of "these are the only compiler versions *known* to work correctly, try any others and they are between you and the mantissa of your FPU doing a DIV 0". Such as specific patch levels of GCC or CLANG.

You could (especially as you approach the modern era) still often *compile* with whatever GCC (and sometimes CLANG) version so long as it was above some horribly old bare minimum, but it was not uncommon to have subtle miscompilations in more esoteric drivers, or flat out some drivers not compiling, etc.

Many forget this, because distros would baseline on these known-good compiler versions, so the majority of users would never know, often even kernel developers.

Building Rust-for-Linux on stable Rust

admalledd — Wed, 25 Sep 2024 05:11:13 +0000

There are a few approaches to this, but the basic one that R4L is (mostly) taking is thus:

1. Recommend a specific known nightly or beta build.
2. Often, the specific nightly build chosen is *actually* the same as the then-just-released "stable compiler", but with the nightly features enabled.

Work-in-progress to get to (2), and really I've used a nightly version from over a year ago (mostly) just fine, the stability-or-not of the total compiler is still (nearly) production worthy, it is the specific features themselves that are more a risk. To mitigate that, Rust-For-Linux *explicitly* tracks and lists exactly which ones they use and why: https://github.com/Rust-for-Linux/linux/issues/2 and that it is (strongly) recommended to not add to that list without good reason.

Further, by linking that github issue as they have, any *changes* in the upstream Rust compiler known to change those unstable features the R4L project depends on gets ping'd to them, which they can then address. Most of the time, the features used by R4L are "user stable" in that "Rust *wants* to enable and let users use it exactly like X, but there are some corner cases that we choose to cause the compiler to error on that should work as well". IE: The features R4L are (mostly) using aren't expected to change much if at all from the developer's perspective. If they are likely to need to change though, due to github bot-magic, they will have plenty of warning and involvement before it even hits a nightly release.

RHEL Support

admalledd — Wed, 25 Sep 2024 04:57:50 +0000

FWIW, in -theory- 360, POWER, Sparc, etc, should work with Rust. Just they aren't "Tier 1 officially supported" yet since they lack commercial backing/QA/validation, and getting them into the kernel would be its own minor project.

For clarity on ISA/Platform support: https://doc.rust-lang.org/nightly/rustc/platform-support....

Though the above is more about applications, not kernels, you can (mostly) just pay attention to the first bit of a platform-triple there for what ISA is supported. That page also clarifies better than I could what T1/T2/T3/etc all mean on "officially supported".

That is all for the LLVM backend, and there are of course the two projects (codegen-gccjit and GCC-rs) to use GCC in some flavor instead.

So yea, there is concern still on missing platforms vs the Linux Kernel, but the hope is the commercial interests involved with those will either (1) support LLVM to those, or (2) support one or the other GCC project.

All that said, still likely years away for any of the missing platforms to need to care about Rust: many of the subsystems and drivers likely to be written won't ever be intended for such legacy platforms anyways.

Building Rust-for-Linux on stable Rust

khim — Wed, 25 Sep 2024 04:48:45 +0000

And when Linus presented Linux in comp.os.minix he explicitly have written: It also uses every feature of gcc I could find. And he used the most recent version of gcc he could find.

Now, that doesn't mean that then need to use bleeding version of Rust would stay with us forever, but I don't see why Rust have to be treated differently from how C was treated in the Linux bringup phase.

Building Rust-for-Linux on stable Rust

milesrout — Wed, 25 Sep 2024 02:15:24 +0000

>are considered experimental, and only enabled when using the pre-release "nightly" version of the compiler. This allows them to iterate on the feature's design, without having to worry about projects outside the compiler and its standard library depending on them.

So why is Linux going to depend on them? How is that supposed to work?

Building Rust-for-Linux on stable Rust

Wol — Wed, 25 Sep 2024 01:22:04 +0000

> Linux only requires GCC 5.1, which was released *nine years* ago. Versus rustc 1.78, which only landed *four months* ago.

istr that for many years Linux had a *maximum* version of gcc, which was positively ancient! If Linux can complain, at different times, that some gccs are too old, while other gccs are too new, then why can't it complain that some rustcs are too old, while others are too new? It is what it is. You just have to use whatever compiler that works.

At the end of the day, the version of the compiler is irrelevant. Seeing as most of the Rust code currently appears to be drivers, if the compiler is contemporaneous with the hardware or feature, what's the problem?

Cheers,
Wol

Unstable compilers

sam_c — Wed, 25 Sep 2024 01:02:04 +0000

Please elaborate. I am unfamiliar with the C you describe.

Building Rust-for-Linux on stable Rust

pizza — Wed, 25 Sep 2024 00:06:54 +0000

> I’ve said it before, and I’ll say it again: it is unfair and disingenuous that the “new contender” is immediately required to clear the bar that nothing else was subjected to for a damn long time.

It is unfair to deliberately misstate Linux's C compiler requirements as well.

When Linux 5.15 bumped the minimum version to GCC 5.1, the latter had been available for *seven years*. (As opposed to the then-brand-new GCC 11)

Building Rust-for-Linux on stable Rust

Heretic_Blacksheep — Tue, 24 Sep 2024 23:50:14 +0000

Kernel docs says minimum version is GCC v. 5.1, but it varies depending on architecture. Obviously recently ported hardware would require a newer GCC so take that with a grain of salt. If a (micro)arch was only integrated into GCC last year, that's your likely minimal version. The rest of the build tools versions vary similarly.

Building Rust-for-Linux on stable Rust

intelfx — Tue, 24 Sep 2024 22:33:32 +0000

> While what you are saying is _technically_ correct, the details paint a very different story.

Only if you conveniently ignore some of these details and highlight the others.

> Linux only requires GCC 5.1, which was released *nine years* ago. Versus rustc 1.78, which only landed *four months* ago.

Yes, so how old is the C code in Linux, and how old is the Rust code in Linux? And how much time did it take (since the inception of $LANGUAGE code in Linux) to arrive at this state of affairs?

I’ve said it before, and I’ll say it again: it is unfair and disingenuous that the “new contender” is immediately required to clear the bar that nothing else was subjected to for a damn long time.

Unstable compilers

Baughn — Tue, 24 Sep 2024 21:44:58 +0000

This describes C, for most of the history of thethe kernel.