Rustaceans at the border [LWN.net]

Rustaceans at the border

Posted Apr 16, 2022 11:04 UTC (Sat) by mpr22 (subscriber, #60784)
In reply to: Rustaceans at the border by jsakkine
Parent article: Rustaceans at the border

> when is Rust compiler/language going to start retain full backwards compatibility between the so called editions?

Do you have a concrete example of a case where the compatibility rules documented in https://doc.rust-lang.org/edition-guide/editions/index.html are being broken?

If so, has someone filed a bug report, and what was the response?

In the alternative, if you aren't claiming those rules are being broken, but rather that those rules do not satisfy your demands, perhaps you could clarify what the deficiency is.

to post comments

Rustaceans at the border

Posted Apr 16, 2022 14:19 UTC (Sat) by jsakkine (subscriber, #80603) [Link] (16 responses)

That's trivial. It's hard to orchestrate changes around the mainline kernel if a change to the language that is not backwards compatible. Direct quote:

"There are times, however, when it is useful to be able to make small changes to the language that are not backwards compatible."

This is unacceptable.

Rustaceans at the border

Posted Apr 16, 2022 16:06 UTC (Sat) by mpr22 (subscriber, #60784) [Link] (5 responses)

The paragraph following the one that contains the sentence you're reacting to says:

> When we want to release a feature that would otherwise be backwards incompatible, we do so as part of a new Rust edition. Editions are opt-in, and so existing crates do not see these changes until they explicitly migrate over to the new edition. This means that even the latest version of Rust will still not treat async as a keyword, unless edition 2018 or later is chosen. This choice is made per crate as part of its Cargo.toml. New crates created by cargo new are always configured to use the latest stable edition.

So if you write all your code in Rust 2018, and label it as being in Rust 2018, it should continue to be treated as Rust 2018 by the toolchain for as long as you keep it labelled as being in Rust 2018, and any failure by the toolchain to do so is a bug which should be reported post-haste.

And then a bit further on:

> The most important rule for editions is that crates in one edition can interoperate seamlessly with crates compiled in other editions. This ensures that the decision to migrate to a newer edition is a "private one" that the crate can make without affecting others.

So your code built as Rust 2018 can interoperate with crates that were built as Rust 2021 or Rust 2015 (Rust 2015 being the initial stable edition associated with v1.0 of the toolchain, and the edition your code is treated as if you do not explicitly specify an edition); in pursuit of this goal, they even created a "raw identifiers" mechanism when defining Rust 2018, to allow a crate which declared public interfaces using identifiers that were valid under Rust 2015 but have become keywords in Rust 2018 to continue to present those public interfaces under the same name.

Rustaceans at the border

Posted Apr 16, 2022 19:31 UTC (Sat) by jsakkine (subscriber, #80603) [Link] (4 responses)

I would like to see a clear explanation how the maintenance flow would work with stable and long-term kernels and all these editions. The oldest long-term still in maintenance was released in late 2016. This looks like initially a nightmare for backporting and bisecting stuff. I admit that this might be because my lack of experience with Rust but that's why Rust community should be able to explain this so that you cannot misunderstand instead of just referencing Rust documentation.

Rustaceans at the border

Posted Apr 16, 2022 22:53 UTC (Sat) by beagnach (guest, #32987) [Link] (1 responses)

You'll probably get a better answer in lkml, since all of this is a work in progress with issues like this still being figured out by some very dedicated people who (if you bother to follow the discussions) actually do care very much about stability, compatibility and so on.

Your chances of getting a good reply would probably be increased by reducing the amount of hostility and condescension in your tone.

Rustaceans at the border

Posted Apr 19, 2022 5:53 UTC (Tue) by jsakkine (subscriber, #80603) [Link]

I did subscribe to that now. Continuing over there. Thanks for pointing out. I did not know that vger list already exists.

Rustaceans at the border

Posted Apr 17, 2022 10:56 UTC (Sun) by khim (subscriber, #9252) [Link] (1 responses)

> I would like to see a clear explanation how the maintenance flow would work with stable and long-term kernels and all these editions.

Only the kernel can answer that. C and C++ also have editions and they also are not 100% compatible. Yet kernel contemplates them, anyway.

Rust editions happen every 3 years, their incompatibility is usually pretty limited and there are guide which helps you go from one edition to another.

They are also enabled on the per-crate basis and it's explicitly permittable to link together crates compiled for different editions.

Thus I would assume kernel folks would stick to oldest editions which are used in still supported versions of the kernel. Or maybe would allow use of the new ones in modules which are not support in older kernels.

Frankly, in practice editions are such a minor issue that it's not even worth discussing: yes, these changes are backward-incompatible but they are extremely minor (holf-dozen to dozen changes every three years) and mostly superfluous (as in: code written for the new edition can still be converted to old editions and in very mechanical fashion). A small nuisance instead of a major PITA.

Much more significant are non-edition style changes. These are numerous, happen every six week and while they are backward-compatible people embrace them quickly which means you can not easily backport new versions of important crates to old version of kernel without bumping the used version of rust compiler first.

You are barking on a wrong tree.

Rustaceans at the border

Posted Apr 17, 2022 22:40 UTC (Sun) by ssokolow (guest, #94568) [Link]

non-edition style changes

I think calling them style changes is a bit misleading. Generally, the stuff that bumps the minimum supported Rust version is the addition or stabilization of new standard library types/functions that people are then quick to jump on because they significantly reduce some boilerplate that has become near enough to ubiquitous to justify their creation.

(If you want to see the kind of things I'm talking about, caniuse.rs provides a quick overview.)

For example, one of the things introduced in Rust 1.56 was adding the rust field to Cargo.toml so libraries can specify the minimum supported Rust version in a machine-readable way.

Rustaceans at the border

Posted Apr 16, 2022 16:08 UTC (Sat) by tialaramex (subscriber, #21167) [Link] (8 responses)

> This is unacceptable

It seems that Linus Torvalds does not agree, since the kernel did in fact take a "small change to the language that is not backwards compatible" by going from C89 to C11.

Unlike this change, Rust's Editions aren't actually language version changes, as you'd have discovered if you read more than a few sentences into that document. The allowed changes can alter the written syntax but not the abstract syntax. Thus, code from Rust 2015, Rust 2018 and Rust 2021 can be expressed together in a single abstract syntax inside the compiler, even though some things you were allowed to do in Rust 2015 are prohibited in Rust 2021, and some things in Rust 2021 were impossible in Rust 2015 and the way you spell some of the things that are allowed still changed if you decided to adopt a newer edition.

For example, in C11 _Bool became a keyword, and in Rust 2021 async is a keyword. So, there's no way to name the identifier _Bool in C11, it's just gone, you'll be told this was anyway reserved and so you were wrong to use it (but the kernel uses various reserved names all over the place and does not respect C's rule about reserved names). Whereas you can name the identifier async in Rust 2021, it's just spelled r#async

Rustaceans at the border

Posted Apr 16, 2022 21:02 UTC (Sat) by jsakkine (subscriber, #80603) [Link] (7 responses)

Hmm... I would not compare something where the breaking change happens for the first time in 30 years, for legit reasons, to the situation with Rust. Quite a bad comparison IMHO.

Rustaceans at the border

Posted Apr 16, 2022 22:57 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]

gcc has _plenty_ of non-backward-compatible changes. Try compiling the kernel with gcc 3, for example.

As it is, Linux has a minimum version requirement for gcc (or clang) that gets raised pretty often. I believe it's at 5.1 right now: https://lore.kernel.org/lkml/20210910234047.1019925-2-nde...

So in practice, the _current_ Linux policy is worse than Rust. Instead of a well-defined edition, the kernel depends on an informal compiler version.

Rustaceans at the border

Posted Apr 17, 2022 11:15 UTC (Sun) by khim (subscriber, #9252) [Link] (5 responses)

> Hmm... I would not compare something where the breaking change happens for the first time in 30 years, for legit reasons, to the situation with Rust. Quite a bad comparison IMHO.

It's actually a very good comparison. The situation when you upgrade GCC and kernel stops working were so numerous that RedHat even had a special kgcc package for a time. And Android used a separate compiler just for kernel for a long time, too.

Granted, it's not because C definition, as a language, changes, but because clang/gcc and kernel disagree about certain minor features of the standard… but from a practical POV it's the same thing.

In theory, the same thing can happen in Rust, too… but in practice, because safe code doesn't contain UB (except when you exploit bugs in the compiler) it's much less of a problem. I don't think I have ever heard about the case where someone upgraded the compiler and Rust program started misbehaving. Sometimes (rarely) there are compilation issues, but they are usually easy to fix.

But with Rust we have an issue similar to C++ in the last century: it's incomplete. Certain important features are not implemented yet. Yes, that doesn't affect the code in way where you can not use new compiler to compile old code but the fact that these features are slowly-but-surely are added to the language means that many crates force you to use pretty new version of a compiler.

That is serious issue in practice. Please think what you plan to do about that, forget about editions, these are minor things.

Rustaceans at the border

Posted Apr 17, 2022 13:52 UTC (Sun) by tialaramex (subscriber, #21167) [Link] (2 responses)

> Please think what you plan to do about that

We're not talking about a feature set here, just one number which goes up. If you have Rust 1.72 then by definition all the stable stuff from Rust 1.71, 1.70, 1.69, and so on is available. You may not care very much whether checked_div() is const (Rust 1.52), but if you want IntoIterator implemented for arrays (Rust 1.53), you'll get const checked_div into the bargain.

The feature switches I think could be more contentious because at the same time some people see value in enabling some feature switch, other people will have begun to depend on it not being present in some cases. But this is already something you see for C compiler flags.

Rustaceans at the border

Posted Apr 17, 2022 14:12 UTC (Sun) by khim (subscriber, #9252) [Link] (1 responses)

The problem is not the “some number goes up”. The problem is that Rust is still getting major features like const generics or GATs.

And because these are significant, major features people often adopt in the matter of weeks and months. Some “conservative” crates only use features which are six months old, but not all crates which kernel may need or want would be “conservative”.

And even then six months is not much, kernel is accustomed for times measured in years. GCC 5.1 is still supported and that was released seven years ago!

That impedance mismatch would be much bigger problem than any hypothetical issue with Rust editions.

Yes, in theory Rust editions can wreak total chaos every three years. In practice that's a tempest in a teapot: they come rarely enough and changes are minor enough that this formal incompatibility rarely becomes a problem in practice.

But the requirement to use six months old compiler can be real PITA for may kernel users.

Rustaceans at the border

Posted Apr 17, 2022 23:13 UTC (Sun) by ssokolow (guest, #94568) [Link]

That said, if you'd like some concrete data, How often does Rust change? by Steve Klabnik is good.

Rustaceans at the border

Posted Apr 17, 2022 22:10 UTC (Sun) by bartoc (guest, #124262) [Link] (1 responses)

Rust “UB” is a smaller category than c/c++ because theres no formal standard for the language with multiple implementers. Not all ub in C is assumed to never happen by compilers (for ex: most of the preprocessor and lexer UB)

The other thing to remember is that the C standard can be obscenely vague about what implementations are allowed to do, the C++ standard tries much harder.

Rustaceans at the border

Posted Apr 19, 2022 15:44 UTC (Tue) by tialaramex (subscriber, #21167) [Link]

> Rust “UB” is a smaller category than c/c++ because theres no formal standard for the language with multiple implementers.

It's not about multiple implementations, at most that would lead to having Implementation Defined Behaviour which is much less scary, but in many cases there wouldn't be any difference since the additional implementations would just do the same thing.

C++ in particular is riddled with _intentional_ Undefined Behaviour. But even in the mundane C standard library trivial functions like abs() have Undefined Behaviour.

Rustaceans at the border

Posted Apr 19, 2022 12:34 UTC (Tue) by farnz (subscriber, #17727) [Link]

An important point here is that while Rust 2021 code will not compile with a Rust 2015-only compiler, a future Rust compiler is expected to be able to compile all past editions, and to support interface-level interoperability between the editions at the crate level. Note that the crate is the unit of compilation in Rust, just as the translation unit is the unit of compilation in C.

So, within a crate, I must stick to one edition; but within a project such as the kernel, I can freely mix editions (because I'd expect the kernel to be made of several crates, not to be one giant crate - just as in C I'd expect the kernel to be in multiple files spread around several directories, and not in a single giant source file). The only limitation I have is that I cannot use recent editions without a bump to the MSRV; but the same applies to C, where I can't use the full language without allowing for the bugs in older versions of GCC.