Rust in the 6.2 kernel [LWN.net]

Rust in the 6.2 kernel

Posted Nov 19, 2022 9:02 UTC (Sat) by gasche (subscriber, #74946)
Parent article: Rust in the 6.2 kernel

`Either` sounds like a generic type that would have its place in any "standard library for building stuff" for the Rust language, not particularly related to kernel programming. Do I correctly understand that the Rust-in-kernel people are not using any of the existing Rust stdlib, and are in fact reinventing their own stdlib? If so, then what are the mechanisms in place to ensure the ongoing quality and consistency of the design?

to post comments

Rust in the 6.2 kernel

Posted Nov 19, 2022 9:42 UTC (Sat) by Wol (subscriber, #4433) [Link] (2 responses)

Bear in mind the kernel does not use the standard C stdlib either. Could they be re-writing the Rust stdlib for the exact same reason, namely (a) the stdlib breaks kernel invariants, and (b) the stdlib does not take into account that hardware breaks stdlib invariants?

Cheers,
Wol

Rust in the 6.2 kernel

Posted Nov 19, 2022 11:06 UTC (Sat) by gasche (subscriber, #74946) [Link] (1 responses)

I can see many reasons why kernel-side programming is a different enough environment to need its own stdlib, and there is nothing fundamentally wrong with that. But on the other hand, I would expect some principles about how to grow this kernel-stdlib. (Some people have experience growing standard libraries for languages/environments and some idea are probably worth porting over from the start.)

For example (this is just a random idea, not a suggestion or anything) Rust-in-kernel could have decided to minimize the diff with outside-kernel Rust code by sticking to subsets of standard APIs or well-known third party packages whenever possible, or taking other steps to avoid divergence and reuse existing design choices. I see no trace of such a process in the [Either patchset](https://lwn.net/ml/linux-kernel/20221110164152.26136-28-o...), which does not make any mention of pre-existing Either code in Rust outside the kernel.

Rust in the 6.2 kernel

Posted Nov 19, 2022 11:19 UTC (Sat) by khim (subscriber, #9252) [Link]

Rust doesn't have Either in it's stdlib. There are either crate with really featureful Either, and it's widely used, but from my understanding Linux kernel doesn't like to depend on external crates.

Rust in the 6.2 kernel

Posted Nov 20, 2022 14:57 UTC (Sun) by atnot (guest, #124910) [Link] (7 responses)

> `Either` sounds like a generic type that would have its place in any "standard library for building stuff"

This is somewhat unrelated to the rest of your question, but I personally really don't agree. Rust core already has types for Option and Result, which are the common case for things that can be one of two types. But to me the Either type ends up simultaneously too broad and specific compared to just defining your own enum. The generic "left" and "right" names are cryptic and confusing, there is little commonality betwen users and it has worse type safety and error reporting. It also lacks extensibility by only offering two variants, making for a painful refactoring if you notice you do need three options after all. Lastly a lot of the time when a function can take multiple things, those things usually share some property which can perhaps be better represented by a trait.

Rust in the 6.2 kernel

Posted Nov 20, 2022 20:42 UTC (Sun) by gasche (subscriber, #74946) [Link] (6 responses)

We've had this argument within the OCaml community, and actually I'm the person who ended up proposing 'either' for the standard library. I agree with your point that, very often, defining your own enum / variant type is the better approach, and also with your point that Option and Result already cover a lot of cases. But:

- The argument that "defining your own variant with domain-specific names etc." also applies as a criticism of both Option and Result, and the fact that actually we use Option and Result a lot shows that this criticism only goes so far. Sometimes there are common scenarios were having a enum of fixed shape with standard name is a good approach, and the benefits in terms of reusing other people's code (auxiliary support functions etc.) are higher than the cost of the less-specific names.

- There are generic functions for which Result could be used, but the more symmetric Either is more natural and thus a better API. (Result and Either are isomorphic so you can clearly always use one instead of the user, the question is how much of a conceptual mismatch this creates.) My main example in OCaml is

val partition_map : ('a -> ('b, 'c) either) -> 'a list -> 'b list * 'c list

It expects a function that, for any value of type 'a, will compute either a 'b or a 'c from it. Then it takes a list of elements of type 'a, and partitions it (using the function) into a list of 'b and a list of 'c. (Interestingly, this is an instance of a sort of generic operation that would split the 'a into an arbitrary sum of possible types, and return as many lists; but that function cannot be expressed easily in the OCaml type system, while 'partition_map' above can.)

Rust in the 6.2 kernel

Posted Nov 21, 2022 17:06 UTC (Mon) by steveklabnik (guest, #114343) [Link] (2 responses)

A small amount of history here: Rust *did* include Either in its standard library long, long ago. https://doc.rust-lang.org/0.7/std/either.html

My recollection of its removal was a survey of usage, and Result was used instead by 99.9% of the existing code instead.

That said, it is true that the crates.io has high usage; it is the dependent of two *extremely* popular packages, itertools and rayon. I'm not aware of any movement to move it back into the standard library, though.

Rust in the 6.2 kernel

Posted Nov 21, 2022 21:16 UTC (Mon) by micka (subscriber, #38720) [Link] (1 responses)

Looking at the Either crate api, it’s awful. At least Result gives you a meaningful name for each variant (ok, err).
Either gives you a... left and a right. Which is which? If both have the same content type, how do you differentiate?
By the way, some people (like me) can’t tell the right from the left. Or rather can’t give them a name (but never will for example drive on the wrong side of the road). I had to work on codd which used an Either type in another language and could never manage to understand any of it even after multiple years.
I guess it’self OK as write-only code...

Rust in the 6.2 kernel

Posted Nov 22, 2022 14:11 UTC (Tue) by TheGopher (subscriber, #59256) [Link]

Agree. Having worked with either in scala I can say that the overhead of a dedicated struct/union/variant is minimal - and the readability is much higher! This is the wrong place to be lazy.

Rust in the 6.2 kernel

Posted Nov 23, 2022 17:17 UTC (Wed) by khim (subscriber, #9252) [Link]

> - The argument that "defining your own variant with domain-specific names etc." also applies as a criticism of both Option and Result, and the fact that actually we use Option and Result a lot shows that this criticism only goes so far.

How? Option and Result are domain-specific.

Option is the fix for the billion-dollar mistake: it handles the case where object may or may not be present.

Result is for the case where function may return “normal” result or “error result”. Open POSIX specifications (or practically any API specification) and you'll find many such functions.

> - There are generic functions for which Result could be used, but the more symmetric Either is more natural and thus a better API.

It's not enough to have one such function. One function is always better server with ad-hock type. You need series of functions which may share a common type. It's easy to imagine such for Options: lot's of data structures have “leaf nodes”. It even easier to do that with Result: almost all functions which deal with files or network may suffer from the same errors (it doesn't matter whether you are creating file or removing it, if you don't have permission to do that error would be the exact same one).

Just what class of functions do you have in mind where you may have symmetrical two choices and these are the same across the whole range of functions?

Yes, partition_map example makes sense (and that's exactly where itertools are using Either), but Rust is imperative language with for. It's really not clear how often do you even need such thing in Rust.

Rust in the 6.2 kernel

Posted Nov 24, 2022 1:48 UTC (Thu) by atnot (guest, #124910) [Link] (1 responses)

> the benefits in terms of reusing other people's code (auxiliary support functions etc.) are higher than the cost of the less-specific names.

That is my problem with Either though. It's too vague to enable any reuse at all. I think a look at the associated functions of Option and Result in rust compared to the Either crate illustrate that quite clearly.

Option and Result have dozens of combinators each that compose usefully. You can turn Results into Options, Options into Results, Options into Iterators, Results of Options into Results, Iterators of Options into Options, Iterators of Results into Results...

Meanwhile on the Either side we get:
(https://docs.rs/either/1.8.0/either/enum.Either.html)
- A few map variations, duplicated for left and right of course
- Some forwarding of inner traits
- A bunch of methods that turn it back into the more useful Option and Result types
- Flip, that swaps the sides

This is not a swipe against the authors. It just shows you just can't really do very much with types that are completely unconstrained and semantically meaningless.

Rust in the 6.2 kernel

Posted Nov 26, 2022 14:21 UTC (Sat) by gasche (subscriber, #74946) [Link]

Another type that is "too vague" is the type of pair `(A, B)` -- and there are not that many standard-library functions defined on it. But we use it all the time! Honestly I think that the criticism against `Either` here comes from a lack of familiarity, rather than a deep reason. We programmers tend to be more familiar with product types than with sum types; for sums the most well-known is `Option<A>` that is the sum (A + 1), and we still need to time to collectively get used to using anonymous sum types when they make sense.

Rust in the 6.2 kernel

Posted Nov 30, 2022 12:07 UTC (Wed) by mlindner (guest, #162450) [Link] (1 responses)

I don't really understand the need of adding the `Either` type, it's just a very very thin wrapper around "enum" with functionality limited to only two types. I wonder why they added it.

Rust in the 6.2 kernel

Posted Dec 3, 2022 0:58 UTC (Sat) by nybble41 (subscriber, #55106) [Link]

`Either` is one of the two combinators needed to represent arbitrary arithmetic data types, together with the zero/void (`!`) and one/unit (`()`) atomic types. `Either` represents the capacity for sums (enums). Pairs / tuples are their dual, representing product types (structs). They're both very abstract, but sometimes that's exactly what you want.

It's true that you could just create your own custom enum type instead (which would be isomorphic to some tree of nested `Either` types) but your custom type won't benefit from any functions the standard library might provide for working generically with values which might be "either" one type or another. For example, partitioning a list of `Either<A,B>` into a list of `A` and a list of `B`. I'm not as familiar with what Rust provides for working with `Either` values but in Haskell the `Either` type is a key component of the Arrow abstraction, a more powerful (albeit less popular) alternative to monads, where they are used to represent conditional processing (`ArrowChoice`). It also benefits from ready-made instances of `Bifunctor`, `Bifoldable`, and `Bitraversable`.