Rust support hits linux-next [LWN.net]

Rust support hits linux-next

Posted Mar 19, 2021 15:17 UTC (Fri) by kragil (guest, #34373) [Link]

Really interesting and awesome development. Rust could be a more productive and safer solution for a lot of driver development, but we have to wait and see.

Rust support hits linux-next

Posted Mar 19, 2021 16:12 UTC (Fri) by darwi (subscriber, #131202) [Link] (4 responses)

There is also an example character driver at drivers/char/rust_example.rs

A lot of standard kernel abstractions and APIs are not yet "rustified" at all.

Having a character driver is also not the most encouraging of examples: adding new character drivers to the kernel is really frowned upon — real drivers need way much more functionality than just FileOperations, a Mutex, and an (incomplete) SpinLock API. Nonetheless, the merge to -next is of course a very encouraging sign.

Rust support hits linux-next

Posted Mar 20, 2021 0:04 UTC (Sat) by ndesaulniers (subscriber, #110768) [Link]

I think kbuild support is more important. Those interfaces and abstractions will come over time as needed. Without build system support, you can't even try to write them. This initial patch set also attempts to put these interfaces in a single shared location in tree, so that drivers can be built in tree, and share code in tree.

Rust support hits linux-next

Posted Mar 20, 2021 2:33 UTC (Sat) by willy (subscriber, #9762) [Link] (2 responses)

I was asked what I thought of this by one of the people working on it, and I suggested they write an NVMe driver. The spec and hardware is readily available, it's relatively high performance hardware, it's something you'd actually want to have, and it doesn't have to depend on too much of the rest of the kernel (PCI services and block layer).

Maybe they're still working on it!

Rust support hits linux-next

Posted Mar 20, 2021 13:16 UTC (Sat) by atnot (subscriber, #124910) [Link] (1 responses)

I think this is intended to be an example driver, not a proof of concept driver. So it's main purpose is intended to be as a documentation reference. Requiring people to understand NVMe is kind of unhelpful for that purpose.

Rust support hits linux-next

Posted Mar 20, 2021 14:00 UTC (Sat) by willy (subscriber, #9762) [Link]

NVMe is pretty simple. The original driver was less than 2000 lines. It's only so complicated in the kernel now because some idiots decided to make the same driver support NVMoF instead of adding a separate driver for it.

The advantage of writing an actual driver for hardware that really exists is that you figure out what's missing from the kernel Rust API -- interrupts, PCI, DMA, locks that need to be shared with code written in C. That kind of thing.

Rust support hits linux-next

Posted Mar 19, 2021 22:02 UTC (Fri) by flussence (guest, #85566) [Link] (13 responses)

Well, that's one way to fend off C++/microkernel fanatics.

If I had to guess, nothing much is going to come of this for maybe 2-3 years. By then I'd expect GCC to have a Rust frontend and everything can be business as usual.

Rust support hits linux-next

Posted Mar 20, 2021 14:10 UTC (Sat) by adobriyan (subscriber, #30858) [Link] (12 responses)

C++ can't be fended off. Any source-incompatible-with-C programming language is doomed to live in separate sandbox where developers are converting unimportant drivers until the steam runs out.

Rust support hits linux-next

Posted Mar 20, 2021 19:06 UTC (Sat) by stevenrwalter (guest, #128616) [Link] (11 responses)

Are you saying you think there will be C++ in the kernel tree at some point? Also C++ is not source-compatible with C

Rust support hits linux-next

Posted Mar 20, 2021 22:18 UTC (Sat) by adobriyan (subscriber, #30858) [Link] (10 responses)

> Are you saying you think there will be C++ in the kernel tree at some point?

I can't even push trivial stuff like renaming few "this" identifiers, so no.

> Also C++ is not source-compatible with C

C++ is maximally source compatible with C which automatically makes it the best
candidate for upgrading to a more capable programming language.

Trivial sources of source incompatibility come from casts and keywords
(people like plain "new" very much) but they can be scripted away.
I mean Googlers who posted Rust to linux-next are surely capable of it.

The most difficult sources of incompatibility don't come from C99 initialisers
(which can be rearranged) but from code like this:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/...

If someone knows how C++ people deal with macrology/features above, please tell me.

Modules also have a _lot_ of icky macros as does tracing. I don't even want to think about it.

The biggest problem so far is g++ being so eager to shift compile time initialisation
to runtime. So far all oopses but one in my little Linux++ project were caused by this:
empty system call table, empty paravirt ops and empty protection_map[] array.

That one different oops was int/long confusion in ERR_PTR rewrite.

C99 caluse shuffling is very dangerous when rebasing. It is so easy to mismerge stuff and
omit or double init. Double init is tolerable because g++ complains but omission is silent.

Rust support hits linux-next

Posted Mar 20, 2021 22:24 UTC (Sat) by adobriyan (subscriber, #30858) [Link]

The best part was min/max rewrite. I liked it so much.

template<typename T, typename... Ts>
constexpr T min(T a, T b, Ts... etc)
{
	if constexpr (sizeof...(Ts) == 0) {
		return (a < b) ? a : b;
	} else {
		return min(min(a, b), etc...);
	}
}

Rust support hits linux-next

Posted Mar 21, 2021 4:00 UTC (Sun) by nyanpasu64 (guest, #135579) [Link] (1 responses)

> The biggest problem so far is g++ being so eager to shift compile time initialisation
> to runtime. So far all oopses but one in my little Linux++ project were caused by this:
> empty system call table, empty paravirt ops and empty protection_map[] array.

C++20 has the constinit keyword to enforce compile-time initialization of a data structure (designed to avoid the static initialization order fiasco), but I don't think it's coming to C anytime soon. I think it might solve your problem with gcc or g++, but I'm not confident.

Rust support hits linux-next

Posted Mar 22, 2021 15:02 UTC (Mon) by adobriyan (subscriber, #30858) [Link]

I don't think constinit as keyword helps. Asking C programmer to type "static const constinit ..." every time is too much and then there is confusion about double const and potential problem if const is omitted moving data from .rodata to .data section silently.

What's needed is compiler flag to disable dynamic initialisation or
equivalent of "nm vmlinux | grep -e _GLOBAL__sub_I_".
The latter would have saved me a lot of time. :-\

Rust support hits linux-next

Posted Mar 21, 2021 21:42 UTC (Sun) by moltonel (guest, #45207) [Link] (4 responses)

> C++ is maximally source compatible with C which automatically makes it the best candidate for upgrading to a more capable programming language.

This is debatable. Some very smart and knowledgeable Linux devs still give C++ a firm "No", while they're giving Rust a cautious "Maybe". You shouldn't ignore that hint.

Any tech shift needs a high gain-over-cost ratio. While the cost of enabling C++ build in a C project is indeed very low, the gains are (in Linux's case, and compared to Rust) not amazing either, and the inadvertent compatibility issues can negate a lot of the gains. Which leads to another counter-argument: being source-incompatible might actually be a good thing, as the clear demarcation between the C world and the Rust world could be more manageable than the fuzzy C/C++ demarcation.

The jury is still out on Linux-Rust, but Linux-C++ looks like a well-beaten dead horse.

Rust support hits linux-next

Posted Mar 22, 2021 11:10 UTC (Mon) by Wol (subscriber, #4433) [Link] (3 responses)

> This is debatable. Some very smart and knowledgeable Linux devs still give C++ a firm "No", while they're giving Rust a cautious "Maybe". You shouldn't ignore that hint.

We've had that discussion elsewhere. Linux needs a low-level language, and probably the best definition of such a language is "it doesn't give the optimiser many options". The linux devs don't like getting bitten by optimisations where the compiler "assumes facts not in evidence", and C++ adds far too many features that enable exactly that.

I gather Rust has been designed as a low-level language, and as such is currently much better than C, which has steadily been acreting hi-level features. That's not to say it'll *stay* a low-level language, but at present Rust is actually probably a far better language than C for system-level programming.

Cheers,
Wol

Rust support hits linux-next

Posted Apr 27, 2021 14:25 UTC (Tue) by nix (subscriber, #2304) [Link] (2 responses)

> I gather Rust has been designed as a low-level language, and as such is currently much better than C, which has steadily been acreting hi-level features.

That's a fairly ridiculous characterization. By any standard, Rust is much higher-level than C: you name it, language-wise it has way more advanced features and permits a higher level of abstraction than C does. What it also has is more consistency (fewer implementation-defined edge cases).

Rust support hits linux-next

Posted Apr 27, 2021 16:21 UTC (Tue) by Wol (subscriber, #4433) [Link] (1 responses)

But does it also provide more control over the machine code that's generated?

I've come to the conclusion that a good low-level language "does not give the compiler much opportunity to optimise well-written code". It seems most problems with C (for the kernel at least) are caused by the compiler optimising the hell out of it.

So long as high-level constructs map cleanly to the obvious underlying machine/assembler code, I'd call that a "good low-level language".

Cheers,
Wol

Rust support hits linux-next

Posted Apr 27, 2021 17:01 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

Depends on who you ask I guess. Is it "clear" when a chain of iterator calls are fused into a single loop over the input? Is it "clear" when tail call optimization is applied? I couldn't tell you what instructions any given line compile down to in Rust or C myself anyways, so for me none of it is "clear" anyways.

That said, I don't know Rust's guarantees around reordering statements, but given the more explicit "I'm using a single variable across threads" model, there are likely a lot fewer ways the compiler can think to order things (barring bugs in the compiler, but that's C's problem too).

Rust support hits linux-next

Posted Mar 22, 2021 0:30 UTC (Mon) by flussence (guest, #85566) [Link] (1 responses)

> C++ is maximally source compatible with C which automatically makes it the best
> candidate for upgrading to a more capable programming language.

Objective-C would be a much better candidate by that criteria.

Rust support hits linux-next

Posted Mar 22, 2021 15:17 UTC (Mon) by adobriyan (subscriber, #30858) [Link]

I don't know Objective-C. How big is the flag day patch? The one which contains the following chunk:

-CC		= $(CROSS_COMPILE)gcc
+CC		= $(CROSS_COMPILE)g++

Rust support hits linux-next

Posted Mar 20, 2021 22:12 UTC (Sat) by amarao (guest, #87073) [Link]

I see a lot of niceness there. Good error types, traits, etc. Nevertheless I feel slightly odd reading examples and pieces. It all ... not high level. I mean there is a constant flux of small things about low-level details. Rust is loved for high level feeling even when dealing with low-level details, like atomics and mutexes.

I'd like to see more polished base, where all kernel grind been enveloped in a higher level abstractions.

Rust support hits linux-next

Posted Mar 21, 2021 20:06 UTC (Sun) by julesb (guest, #139334) [Link] (8 responses)

cool, I have to learn Rust to survive in the future :)

Rust support hits linux-next

Posted Mar 24, 2021 22:06 UTC (Wed) by Subsentient (guest, #142918) [Link] (7 responses)

I bit the bullet and learned Rust, precisely because "if I know it, I don't need to fear it". I can tell you it's a lot better than I expected, but it's got its own set of issues, the biggest imho being the culture around it. That said, once you learn it, and I mean really learn it, you'll probably like it. It's an immature language with glaring gaps in documentation and standardization, but if you can get past that, it can save you a lot of pain in the way of bugs in the future. I'll always love C++, even though most hate it, but lately I can't justify using C++ to myself for projects, when I know that Rust will be so much less painful of a development experience overall.

Rust support hits linux-next

Posted Mar 25, 2021 10:05 UTC (Thu) by farnz (subscriber, #17727) [Link] (5 responses)

Out of interest, what issues do you see in the culture around Rust? I'm curious, because I like the culture around Rust a lot more than I like the culture around C and C++, but that may be bias because of how I got introduced to the two sets of cultures.

Rust support hits linux-next

Posted Apr 16, 2021 12:14 UTC (Fri) by LtWorf (subscriber, #124958) [Link] (4 responses)

probably cargo, pinning dependencies and no dynamic linking

Rust support hits linux-next

Posted Apr 16, 2021 13:15 UTC (Fri) by farnz (subscriber, #17727) [Link] (3 responses)

Of those, cargo and dynamic linking are technical, not culture. Pinning dependencies is culture, but I'd like to know where it happens because mostly people pin semver versions, not dependencies.

Rust support hits linux-next

Posted Apr 16, 2021 22:30 UTC (Fri) by LtWorf (subscriber, #124958) [Link] (2 responses)

The culture comes from the technical choice of not doing dynamic linking.

The choice was made exactly to have this culture, so it's not really a technical choice.

Rust support hits linux-next

Posted Apr 17, 2021 12:36 UTC (Sat) by mathstuf (subscriber, #69389) [Link]

> The choice was made exactly to have this culture, so it's not really a technical choice.

Hmm. Are there any links you have to back this up? AFAIK, it's because defining an ABI now would tie their hands for future improvements that are wanted (e.g., better structure packing, new sentinel representations for stdlib types, etc.), but haven't been implemented yet. I suspect a symbol versioning set of attributes would likely also be in order.

Rust support hits linux-next

Posted Apr 17, 2021 14:58 UTC (Sat) by farnz (subscriber, #17727) [Link]

All the discussions I can find on dynamic linking Rust say it works just fine, it's just that there's no stable Rust ABI yet - you have to have a C ABI if you want to be dynamically linked. They then go into technical reasons why there is no stable Rust ABI yet.

I cannot find anything to back your claim that the choice was made exactly to have this culture - rather that the choice has been made for sound technical reasons (and it's the same reason you can't dynamically link a lot of C++ code like the C++ STL too), and that if a good technical solution can be found, then dynamic linking will be revisited.

Rust support hits linux-next

Posted Mar 25, 2021 23:13 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

I'm also curious about the culture bit, but I'd like to ask about the "glaring gaps in documentation" that you're referring to. FWIW, I've found it way more helpful than Python's documentation (especially around the stdlib) at least.