|
|
Subscribe / Log in / New account

Rust heads into the kernel?

Rust heads into the kernel?

Posted Apr 21, 2021 5:55 UTC (Wed) by kunitz (subscriber, #3965)
In reply to: Rust heads into the kernel? by xinitrc
Parent article: Rust heads into the kernel?

I agree with that statement. Write code that works, does something useful, and allows users to achieve a goal that is otherwise not possible. Instead of writing announcements, the Rust people should rewrite three drivers from different areas and prove that their statements about better safety are justified.

I'm skeptical because of statements like this:

Secondly, modules written in Rust should never use the C kernel APIs directly. The whole point of using Rust in the kernel is that we develop safe abstractions so that modules are easier to reason about and, therefore, to review, refactor, etc.

This sounds not very kernel-like. Creating abstractions in such a way that they bear no costs will be hard. Giving the tendency of internal kernel APIs to change over time, there must be automation to support them. All of that will become clear if one would try to write actual drivers in Rust.


to post comments

Rust heads into the kernel?

Posted Apr 21, 2021 7:19 UTC (Wed) by matthias (subscriber, #94967) [Link] (2 responses)

> I agree with that statement. Write code that works, does something useful, and allows users to achieve a goal that is otherwise not possible.

Definitely.

> Instead of writing announcements, the Rust people should rewrite three drivers from different areas and prove that their statements about better safety are justified.

On the other hand, getting feedback early is not bad. If the only complaints are: there are no drivers, yet. The Rust people can continue and write some drivers. If there would have been big complaints about the interface and the way they try to integrate Rust into the kernel, then having already implemented some drivers would have been a waste of time.

> I'm skeptical because of statements like this:
> Secondly, modules written in Rust should never use the C kernel APIs directly. The whole point of using Rust in the kernel is that we develop safe abstractions so that modules are easier to reason about and, therefore, to review, refactor, etc.
> This sounds not very kernel-like. Creating abstractions in such a way that they bear no costs will be hard.
Nobody said that kernel development would be easy. On the other hand, getting all users of some kernel API to be correct in C is also not easy. In fact it is so hard, that people usually do not even try to prove things like memory-safety.
> Giving the tendency of internal kernel APIs to change over time, there must be automation to support them.
The Rust safe abstraction will be one user of an API like every other user. And it will have to be adapted when an API changes just like every other user of the API. In some cases, it might be enough to change the implementation of the safe abstraction. In other cases, also the Rust interface of the abstraction will have to be changed and therefore Rust code using the interface will also have to be refactored. But this would also be true for any C code using the interface.

> All of that will become clear if one would try to write actual drivers in Rust.

And this is just what should happen now, after there was no clear objection to the way Rust will be integrated into the kernel.

Rust heads into the kernel?

Posted Apr 21, 2021 8:34 UTC (Wed) by mkubecek (guest, #130791) [Link] (1 responses)

> The Rust safe abstraction will be one user of an API like every other user. And it will have to be adapted when an API changes just like every other user of the API. In some cases, it might be enough to change the implementation of the safe abstraction. In other cases, also the Rust interface of the abstraction will have to be changed and therefore Rust code using the interface will also have to be refactored. But this would also be true for any C code using the interface.

But "will have to be adapted" says nothing about who is going to be responsible for that. As of now, the practice is that if you want to change internal API, you are responsible for adapting all its in tree users. This is fine as long as all of kernel is written in C. But when another language is added, it is a natural question if you still expect anyone who would touch any internal API to also update your abstractions - which also means learning rust to a level sufficient for such work. Or if we can simply break the rust stuff at will and you will fix it later. Or if anyone who wants to touch the internal API has to find (in advance) someone who will take care of the rust dependants. Neither of the options looks very appealing to me.

It is very unfortunate that while rust people were very active in other parts of all the LKML and LWN discussions, nobody cared to reply to these concerns raised by Peter (and me). For some of us, this is really a fundamental question and we would like to have an idea where this path is heading before taking the first step. So I'm asking again: do you expect that anyone seriously involved in kernel development will have to get proficient (also) in rust? If not, how do you plan the coexistence to work?

Rust heads into the kernel?

Posted Apr 21, 2021 10:36 UTC (Wed) by tux3 (subscriber, #101245) [Link]

If the Rust side does go ahead with its plan of building wrappers around C APIs vs. allowing Rust drivers to directly interract with C, you at least avoid a scenario where a C refactor patch has to touch hundreds of Rust callers written in idiomatic style that can be very much unreadable to pure C people.

I imagine the wrappers will restrict themselves to very simple, easily understood code. Not only for the sake of people who should not have to read TRPL cover to cover, but also because unsafe wrappers ought to prioritize readability and 'obviously correct'ness.

Now would we still need dedicated people to take your C patches and write the corresponding Rust part?
Maybe. I'd like to hope not!

Rust heads into the kernel?

Posted Apr 21, 2021 7:28 UTC (Wed) by taladar (subscriber, #68407) [Link] (43 responses)

At the very least I would expect Rust wrappers around kernel APIs to get rid of that error-prone C inline error reporting with negative numbers that is so common in C.

Rust heads into the kernel?

Posted Apr 21, 2021 7:48 UTC (Wed) by pbonzini (subscriber, #60935) [Link] (6 responses)

Yes, you can expect that at some point the Result<T, E> will be converted to a Result<(), Errno> or Result<&SomeStruct, Errno>, and then unsafe code will convert that to a negative errno.

Rust heads into the kernel?

Posted Apr 21, 2021 7:56 UTC (Wed) by josh (subscriber, #17465) [Link] (5 responses)

Long-term, we're hoping to teach Rust that inside the kernel a valid reference will never have a value less than 4096 (carving out a "niche" in Rust terms), so that a Result<&T, Errno> (or a Result with any success type that uses the space of valid pointers, such as Box) will be exactly one machine word in size and have the same layout as a C ERR_PTR.

Rust heads into the kernel?

Posted Apr 21, 2021 12:50 UTC (Wed) by pbonzini (subscriber, #60935) [Link]

You still probably want to use other error enums than errno in Rust code, so that should be well into diminishing returns area though.

Rust heads into the kernel?

Posted Apr 21, 2021 22:59 UTC (Wed) by roc (subscriber, #30627) [Link] (3 responses)

Won't you also want to make Result<T, Errno> a single machine word where T is u32 or even u63?

Rust heads into the kernel?

Posted Apr 22, 2021 8:31 UTC (Thu) by pbonzini (subscriber, #60935) [Link]

It would be a bit more complicated by Rust being much more strict on integer casts. It means that you'd use usize in many places where C would use unsigned (or more likely, use an int that would always be positive).

Rust heads into the kernel?

Posted Apr 22, 2021 19:03 UTC (Thu) by josh (subscriber, #17465) [Link] (1 responses)

That's theoretically possible but would take either some more language work or some library work, beyond the language work we'd need to do to allow storing enums in the 0-4095 niche of pointers.

I'm assuming that you mean "a single machine word" on a 64-bit system, because on a 32-bit system you can't store a general u32 and something else in the same 32-bit word.

Rust, today, has the property that a type is always stored the same way no matter where you put it, even when taking advantage of niches and similar. That's important because if you have a Result<T, Errno>, if it's an Ok you can get a &T from it, and if it's an Err you can get a &Errno from it. So, that &Errno needs to actually reference a thing that looks like an Errno in memory, not something that's been tweaked to use different enum values.

That works fine for a reference or box or other type that cannot have a value in the range 0-4095 (once we have the ability to declare that "not in the first page of memory" niche). An Errno can use that niche.

However, if you have some type T that has a niche that isn't 0-4095 (random example: a "userspace pointer" that can't point into kernel memory but *can* potentially be 0-4095), such that T's valid values and Errno's valid values would overlap, you can't tweak the Errno's values to store them in T's niche. That would mean you couldn't get a valid reference to an Errno, because it needs translation before extracting it.

In the specific case of a u32, the easiest solution is just to let Rust store the u32 and the Errno alongside each other, and they should take up no more than a 64-bit word.

To solve the general case of niches that would require transforming the values of Errno when storing/extracting them, we could handle that in two ways, one at the library level or one at the language level:

At a library level, we could just have a dedicated type (not Result) that stores an Errno and another type, which *doesn't* let you get a &Errno from it, and instead only lets you get an Errno. (Errno itself is smaller than a machine word, so we might as well just use its value.) We could then teach that dedicated type to store the errno in the niche of the T, and re-extract it later. That would solve the specific problem, with some effort, but would require writing code specifically to handle that transformation.

Or, at a language level, we can add a concept of "types that you can't have a direct reference to", make a version of Errno with that attribute, and then allow such types to take advantage of arbitrary niches in other types. Because you can't have a reference to those types, they don't have to be stored in a way that looks exactly like they'd be stored if standalone. You'd typically use it for types that are so cheap to copy (because they're smaller than a machine word) that the inability to reference them isn't an issue.

That language-level approach is the kind of thing that you can teach the Rust language to do *once*, and then it can automatically do the kinds of optimizations that in C you'd have to manually manage in each data structure.

Rust heads into the kernel?

Posted Apr 23, 2021 0:43 UTC (Fri) by roc (subscriber, #30627) [Link]

I think there's actually a conundrum here. Someone is going to have to decide how much language and compiler work to do before you really push on Rust being used in the Linux kernel --- because before you do the latter, you will need to settle once and hopefully for all the idiomatic way to handle Errno results for various common T types.

Rust heads into the kernel?

Posted Apr 21, 2021 15:44 UTC (Wed) by wtarreau (subscriber, #51152) [Link] (35 responses)

That's not necessarily more error-prone than having to check a composite result. It's trivial to return composite results in C, it's just rarely used because most of the time it brings little value and brings its own class of bugs as well.

Rust heads into the kernel?

Posted Apr 21, 2021 18:48 UTC (Wed) by pbonzini (subscriber, #60935) [Link] (34 responses)

The difference is having language-level support for the goto-based error-checking idioms that Linux uses. For example something like
  r = func();
  if (r < 0)
    goto some_label;
  ...
some_label:
  // chain of kfree and unlocks here
  return r;
would be just
func()?;
Likewise, this
  r = -EINVAL;
  if (func() < 0)
    goto some_label;
would be
func().map_err(|_| errno::EINVAL)?;

Rust heads into the kernel?

Posted Apr 22, 2021 20:18 UTC (Thu) by wtarreau (subscriber, #51152) [Link] (33 responses)

And there are really people who can parse such horrors ?

Also not being able to figure exactly what that function is doing in my back after the question mark bothers me. How do you decide to atomically increment an error counter on the return path with that method ? How do you increment different counters depending on the case you've met ? I.e. oversized_frame or undersized_frame ?

This looks like eye-candy for dummies to me, just to attract beginners by claiming they'll write easier code but it's hard to imagine it can ever be useful outside of school books.

Rust heads into the kernel?

Posted Apr 22, 2021 20:34 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

`?` is an early return in the error case. If you want to increase an error counter before returning, I would match and on the `Err` arm increment and return. Same thing with the different counters case.

You might call it eye candy, but my eyes can't follow goto spaghetti and match up allocation location/free call pairings on a whim. the RAII makes early returns in-place doable. If you have other things to do, don't use `?`, but do your thing and return there.

Rust heads into the kernel?

Posted Apr 22, 2021 22:04 UTC (Thu) by pbonzini (subscriber, #60935) [Link] (23 responses)

Are there people who can parse C function pointer casts or ternary operator? Sure, they just have to study the language.

Why should it be a surprise that not all of Rust can be guessed without studying the language?

Honestly, the last time I heard this kind of complaint I was teaching first year university students.

Rust heads into the kernel?

Posted Apr 23, 2021 9:02 UTC (Fri) by Wol (subscriber, #4433) [Link] (18 responses)

Weren't C - and POSIX - originally designed as a glorified assembler for writing a games machine to run on a PDP-11?

So it wasn't even designed as a language to write an OS in! As those of us who can remember the - WELL DESIGNED - commercial systems that came before, to us C and *nix are basically crap. The problem is they were "good enough, and cheap enough" to sweep aside the opposition. And now all this extra stuff is being bolted on to get round design decisions that made sense for a single-user games machine, but are rubbish for actually writing real software, but can't actually be designed away because so much old software relies on them!

Whereas Rust has been designed as a language for writing low level systems. The first OS I used was originally written in FORTRAN. Then they wrote chunks of it in PL/1, both the official and their own unofficial versions. It worked pretty well. Then they re-wrote it in C and I think it was one of the major reasons the company tanked.

Rule 0 of health and safety - MAKE DOING THE RIGHT THING EASY. That's a design goal of Rust, which C appears never even to have heard of!

You know I'm on the raid mailing list, and the number of bugs and lockups and whatever that seem to be creeping out of the woodwork because memory safety, spinlock, and the similar assorted mistakes have been made seems awful. Okay, they only seem to hit when users are doing esoteric things - just running a raid array is very well tested - but it would be nice if the language made those sort of bugs HARD, rather than trying to guarantee they will lurk in pretty much any poorly-tested error path.

Cheers,
Wol

Rust heads into the kernel?

Posted Apr 23, 2021 12:04 UTC (Fri) by wtarreau (subscriber, #51152) [Link] (16 responses)

Sincerely I'm not seeing any difference between the examples above and my old horrible memories of Perl, where everything was written as awful regex matches, that was well-known for being a "write only language", and that resulted in massive vulnerabilities everywhere since it was so difficult to figure what was *really* going to happen behind the curtains.

Memory safety is the argument circling in loops all the time about this language. I'd like *CONTROL* safety. By obfuscating controls, I hardly see how someone may assess what is really going to be done. A number of issues come from compilers moving code around to optimize it while it reads fine on the screen, forcing us to add compiler barriers, READ_ONCE() and so on. You can add all the amount of "memory safety" you want, if that results in a totally mangled non-sequential syntax, you can be certain that such issues will be even harder to spot, especially with the the arrogant attitude I've seen from some people around the language constantly claiming "bugs are C, we can't make bugs in rust". Bugs are human. Humans need to understand what is being done. C is far from being perfect, we all know it, but it translates to instructions and does not add magic everywhere to make the developer feel proud of writing code using smileys. This is essential to me and way more than "memory safety at compile time" and "panic in unexpected situations".

Rust heads into the kernel?

Posted Apr 23, 2021 12:21 UTC (Fri) by hummassa (subscriber, #307) [Link] (8 responses)

Sorry, but the whole "Perl is a write-only language" reads to me like "I can't read Perl, how can people be so mean writing in a language that uses symbols!!! Sigils!!! They are programming with smileys!!!"

Yes. I understand that some people have problems with a highly symbolic language. But the "I am in control if I can see every GOTO" is just an illusion. Many parts of the kernel code will be data-driven. And just like the regular expressions you seem to abhor, there are many, many ways to write correct programs that are perfectly readable... by those who can read the language. Even if those programs are as succint as possible.

The fact is that whenever you force kernel devs to trace each and every exception that they will have to redirect flow (goto) to some place where the exact amount of resource acquisition you did up to now can be undone, you are just introducing opportunities for bugs to creep in.

Memory safety is ONE of the concerns -- resource freeing/ RAII is a bigger concern that is addressed by higher-level languages like Rust and C++.

Rust heads into the kernel?

Posted Apr 23, 2021 14:13 UTC (Fri) by wtarreau (subscriber, #51152) [Link] (7 responses)

> And just like the regular expressions you seem to abhor, there are many, many ways to write correct programs that are perfectly readable... by those who can read the language.

Yes, they're all listed on cve.mitre.org

Rust heads into the kernel?

Posted Apr 23, 2021 18:15 UTC (Fri) by hummassa (subscriber, #307) [Link] (6 responses)

> Yes, they're all listed on cve.mitre.org

PLEASE PRETTY PLEASE show me ONE example of a CVE caused by a regular expression. Let me make some popcorn while I wait for you to try.

Rust heads into the kernel?

Posted Apr 23, 2021 19:02 UTC (Fri) by hummassa (subscriber, #307) [Link] (2 responses)

Now, even if I am charitable and see that what you meant was "oh ultra-terse, symbolic code causes CVEs", this is provably false, also.

Rust heads into the kernel?

Posted Apr 23, 2021 19:47 UTC (Fri) by Wol (subscriber, #4433) [Link] (1 responses)

J (or APL), anyone :-)

Cheers,
Wol

Rust heads into the kernel?

Posted Apr 29, 2021 16:58 UTC (Thu) by ejr (subscriber, #51652) [Link]

I learned APL2 in high school. I learned Perl 4 from the man page. I find these discussions curious.

Rust heads into the kernel?

Posted Apr 23, 2021 19:02 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

Here you go!

http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-23354

"The package printf before 0.6.1 are vulnerable to Regular Expression Denial of Service (ReDoS) via the regex string /\%(?:\(([\w_.]+)\)|([1-9]\d*)\$)?([0 +\-\]*)(\*|\d+)?(\.)?(\*|\d+)?[hlL]?([\%bscdeEfFgGioOuxX])/g in lib/printf.js. The vulnerable regular expression has cubic worst-case time complexity. "

Rust heads into the kernel?

Posted Apr 23, 2021 19:59 UTC (Fri) by hummassa (subscriber, #307) [Link]

Point conceded! Oh man, I've been proven wrong TWICE already on this thread! I must be turning into a Real Boy™!

Rust heads into the kernel?

Posted Apr 27, 2021 23:28 UTC (Tue) by ras (subscriber, #33059) [Link]

I realise this is just a bit of fun, but I'd say that is not the regex's fault. It's the fault of the underlying re library using an NFA to recognise it. I've been bitten by NFA's going rouge of some input so many times now, I'd say a regex library using a NFA is a bug that leads to CVE's like the one you found.

DFA's might occasionally take exponential space for their compiled form and you have to incur the expense of compiling the entire thing, but you get to find out about your bug the first time the regex is compiled, not some at some random time later in production.

Rust heads into the kernel?

Posted Apr 23, 2021 12:23 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (6 responses)

> Sincerely I'm not seeing any difference between the examples above and my old horrible memories of Perl, where everything was written as awful regex matches, that was well-known for being a "write only language", and that resulted in massive vulnerabilities everywhere since it was so difficult to figure what was *really* going to happen behind the curtains.

The new syntax is the lambda parameter bit (`|_| expr` is a lambda that takes one argument, names it `_` to ignore it, then evaluates to `expr`) and the `?` which is the "if an error occurred, return it from the current function, otherwise evaluate to its success value.

The other bit you need to remember is:

> I'd like *CONTROL* safety

RAII is what you want then. You don't have to thread your own cleanup-on-error cases together based on where they can occur from within the function. If you allocate a lock in Rust, it has two destinies: passing it off to another function or unlocking when the function returns. There's no other choice[1].

> but it translates to instructions and does not add magic everywhere to make the developer feel proud of writing code using smileys.

Rust does the same thing. I feel like saying "?" hides control flow is like saying "?: hides lazy evaluation" or "how can I see short-circuiting boolean operator logic" in C. It's part of the language. If you don't want to learn it, fine. But why does your resistance to learning something new block others from using what they know?

[1] Well, I suppose you could manually drop it, but that is such an oddity that it'd be like seeing "// intentionally don't unlock here; it will be done in $foobar later in the logic that uses this function" except you *have* to spell it out instead of being lucky someone was kind enough to leave a comment for you.

Rust heads into the kernel?

Posted Apr 23, 2021 12:25 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

> If you allocate a lock in Rust

Sorry, this should read "take a lock" (was thinking lock guard, but then edited around it).

Rust heads into the kernel?

Posted Apr 23, 2021 12:40 UTC (Fri) by hummassa (subscriber, #307) [Link] (2 responses)

> > I'd like *CONTROL* safety

This person does not want control safety, they want safety control. They want to SEE the "if condition bail out thru such and such path" because they are under the impression that they can do better than the compiler in determining which "such and such path" is appropriate.

As you said (and I also did, in another comment), it's ok to like "things written as words" in software. And it's ok if a person likes C, knows C, and has no interest in learning C++ or Rust. But it's just silly to block others from using what they already know.

Rust heads into the kernel?

Posted Apr 23, 2021 17:24 UTC (Fri) by Wol (subscriber, #4433) [Link] (1 responses)

And I get the impression he is also assuming that the language ENFORCES "bail on error". I don't have any knowledge of Rust but I can't believe that the designers would be so stupid as to stop you identifying, handling, and cleaning up errors manually.

Cheers,
Wol

Rust heads into the kernel?

Posted Apr 24, 2021 20:29 UTC (Sat) by farnz (subscriber, #17727) [Link]

Rust indeed does not enforce bail on error. The core library uses a Rust enum called Result<T, E> (where T is a type parameter for the "happy" path, and E is a type parameter for the "failure" path) to handle errors. It has two variants: Ok(T) for the happy path, and Err(E) for the failure path; you can use Rust's match or if let primitives to dissect an error manually, if you wish.

To return a "happy" value, you return Ok(value), where value has to be of type T; similarly, to return a failure, you return Err(error) where error has to be of type E. There is a special operator, previously spelt try!(...) and now spelt ...? which for Result is equivalent to the following Rust:

match ... {
    Ok(v) => v,
    Err(e) => return Err(e.into()),
}

Of course, if you write it out in full, you don't have to return the error, or do the conversion (.into() uses the Into trait to convert from one type to another, allowing you to convert errors in this case if a suitable conversion is implemented. In the kernel, you might use this to convert from a driver-specific error to a KernelError type, for example, so that you can retain semantic details of your error while in Rust). You can implement a different pattern match that does whatever you want with the Result type - maybe not bailing at all, but doing something different.

That said, you do still end up with Rust doing some work when you exit a scope, beyond just deallocating the stack frame. If a type implements the Drop trait, the code in there is run when the object is deallocated.

Rust heads into the kernel?

Posted Apr 23, 2021 14:21 UTC (Fri) by wtarreau (subscriber, #51152) [Link] (1 responses)

> The new syntax is the lambda parameter bit (`|_| expr` is a lambda that takes one argument, names it `_` to ignore it, then evaluates to `expr`) and the `?` which is the "if an error occurred, return it from the current function, otherwise evaluate to its success value.

Thanks for explaining. I find this overly complicated to add an argument that must be ignored.

> The other bit you need to remember is:
> > I'd like *CONTROL* safety
>
> RAII is what you want then. You don't have to thread your own cleanup-on-error cases together based on where they can occur from within the function.

No, please no, that the worst horror of modern languages in my opinion. As I explained, it encourages in *not* handling abnormal conditions and letting someone else deal with them. Abnormal conditions are best handled *where* they occur, not by lazily returning to the caller which will do the same.

> If you allocate a lock in Rust, it has two destinies: passing it off to another function or unlocking when the function returns. There's no other choice[1].

While most of the time I do prefer locks to be symmetric, I've already entered into situations where it was needed *not* to unlock, and yes, that requires clean code and a big fat comment above the function (and if possible a name that suggests it).

> but it translates to instructions and does not add magic everywhere to make the developer feel proud of writing code using smileys.

Fortunately another member here showed how to handle errors with if/else that allows to properly take care of them instead of ignoring them.

> Rust does the same thing. I feel like saying "?" hides control flow is like saying "?: hides lazy evaluation" or "how can I see short-circuiting boolean operator logic" in C. It's part of the language. If you don't want to learn it, fine. But why does your resistance to learning something new block others from using what they know?

It's not a matter of me refusing to learn, it's that the arguments that are presented to defend it are exactly those which I take as counter-arguments: let's encourage developers not to care about anything anymore and ignore who will process their errors since the compiler will surely know better.

> [1] Well, I suppose you could manually drop it, but that is such an oddity that it'd be like seeing "// intentionally don't unlock here; it will be done in $foobar later in the logic that uses this function" except you *have* to spell it out instead of being lucky someone was kind enough to leave a comment for you.

Comments are precisely made to document non-obvious stuff like this. Someone who writes a function that plays with non-obvious locks or locks in a non-obvious way and doesn't mention it is looking for trouble anyway.

Rust heads into the kernel?

Posted Apr 23, 2021 14:51 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

I find this overly complicated to add an argument that must be ignored.
It's not "add[ing] an argument that must be ignored". The .map_err passes the function the error. If you want to ignore it, you use `|_|`. If you want to handle it, give it a name. It's not like C where a N-ary function can be passed to a callback passing M arguments as log as M < N. The arity must match, so the passed closure must take an argument (here, ignored).
No, please no, that the worst horror of modern languages in my opinion. As I explained, it encourages in *not* handling abnormal conditions and letting someone else deal with them. Abnormal conditions are best handled *where* they occur, not by lazily returning to the caller which will do the same.
I think we agree, but we're talking on different levels of abstraction. Of course, where to *best* handle an error depends on the error itself. Bad flags? Caller's fault. Lock contention? Probably my issue. Can't open a file? Do I have a contract to retry? Maybe the kernel *is* different, but most of the time I can wrap up an error like this:
let output = process::Command("git") // the executable
  .arg("--version") // build up an argument list
  .output() // I want the output of the command
  .map_err(|err| MyErr::GitExec("--version", err))?;
if output.status.success() {
  return Err(MyErr::Git("--version", output.status.code()));
}
where `MyErr::GitExec` indicates: - `git` failed to *launch* - the "--version" describes what was being done - the err contains the error that occurred `MyErr::Git` indicates: - `git` executed, but returned failure - "--version" acts largely the same - it contains the exit code as well My code doesn't particularly care *why* it failed, but it can annotate why it was trying to do what it was doing when the error happened.
Fortunately another member here showed how to handle errors with if/else that allows to properly take care of them instead of ignoring them.
Sure. When it calls for it, expand it out. I think I still usually prefer a closure for Result<> handoffs like above, but it's a style difference. The code is basically the same (in fact, I *think* that you can use cargo-expand to see what `?` desugars to in your code as well, but maybe that was when it was spelled `try!()`).

Rust heads into the kernel?

Posted Apr 23, 2021 23:19 UTC (Fri) by anselm (subscriber, #2796) [Link]

Weren't C - and POSIX - originally designed as a glorified assembler for writing a games machine to run on a PDP-11?

Nope. You may wish to read up on the early history of Unix.

In a nutshell, the game was written in assembly language for the PDP-7 (using a cross-assembler on the GE 635) and preceded Unix. Unix came along later as a rudimentary operating system for software development on the PDP-7. The early Unix hackers around Ken Thompson only got access to a PDP-11 in 1970 when they promised to produce a system for editing and typesetting text. This was eventually used by the Bell Labs patent department and gradually evolved into what we would think of as Unix. Various higher-level programming languages were tried and discarded, and eventually the Unix kernel was rewritten in C in 1973.

Unix was actually quite innovative in its time (especially the file system was miles ahead of the competition as far as versatility was concerned), and the fact that Unix was written in C did a lot to make it portable to many of the evolving computer architectures of the 1970s and 1980s. It is safe to say that without Unix and C, the computing landscape today would look a lot different and – in view of the various clunky manufacturer-specific systems that Unix eventually replaced – not necessarily better. Ken Thompson and Dennis Ritchie didn't receive their Turing award for their awesome beards.

Rust heads into the kernel?

Posted Apr 23, 2021 11:52 UTC (Fri) by wtarreau (subscriber, #51152) [Link] (3 responses)

Sorry, but for me, even with a lot of good faith and mind stretching, I cannot figure how something spelled "func().map_err(|_| errno::EINVAL)?;" could translate to "if (func() < 0) goto leave;". Using smileys to write expressions will sooner or later either strike you in the back when you're tired late at night, or result in something different being done to "help" you.

Ultimately you *want* the processor to emit a test and a branch to a place where some things are undone. Why would you force yourself to express it in a totally different way ? I could also take a pencil, draw animals and expect the compiler to figure what I'm trying to do and emit code, and there will possibly be fans of this, but I'm not one of them, I'm sorry.

Rust heads into the kernel?

Posted Apr 23, 2021 12:10 UTC (Fri) by hummassa (subscriber, #307) [Link] (2 responses)

Seriously? You can't read "call func(), and if it returns any error (the less than zero C status code), map this error to errno::EINVAL and return that EINVAL error instead, unwinding any initializations that had been done up to now -- which includes potentially releasing locks, file handles, and other resources that if I forget to release manually will cause a leak?" I don't like Rust very much, but even I can read that
  func()?
is much less error-prone than
  r = func();
  if( r < 0 )
    goto UNWIND_EXACTLY_WHAT_HAS_BEEN_DONE_TILL_NOW;
where you have to manually sprinkle your function with a dozen UNWIND1, UNWIND2, etc...

Rust heads into the kernel?

Posted Apr 23, 2021 14:32 UTC (Fri) by wtarreau (subscriber, #51152) [Link] (1 responses)

> Seriously? You can't read "call func(), and if it returns any error (the less than zero C status code),

No, sorry, I can't parse that this smiley "|_|" means "less than zero".

> I don't like Rust very much, but even I can read that
> func()?
> is much less error-prone than
> r = func();
> if( r < 0 )
> goto UNWIND_EXACTLY_WHAT_HAS_BEEN_DONE_TILL_NOW;
> where you have to manually sprinkle your function with a dozen UNWIND1, UNWIND2, etc...

And the simple fact that you write that is *exactly* what worries me about such practices. Just getting rid of abnormal failures seems to be the default way of developing. With the example involving the "goto" above it's simple to add an error counter for the specific type of condition that was met there, and possibly call other stuff to propagate that.

With the default "unwind" practice, well, it's just that. "OK got an error, let's report above that something happened, surely someone will figure what it was". When reading data from a NIC for example you can face a number of different "errors" (which are in fact abnormal but totally expected conditions) for which you have to write code to increase their respective counters and certainly not just "unwind exactly what has been done". In some circumstances you'll even need *not* to unwind everything, because for example you'd have pre-allocated and initialized a buffer and will prefer to keep it around for the next use instead of putting it back into the dirty pool so that it gets initialized again on next call. This is just an example of course, but you probably understand what I mean. And if not, anyway, I sense that we'll never agree because we don't have the same expectations.

Rust heads into the kernel?

Posted Apr 23, 2021 18:14 UTC (Fri) by hummassa (subscriber, #307) [Link]

No, sorry, I can't parse that this smiley "|_|" means "less than zero". It doesn't. The "less than zero" part is the C construct where you use positive return values to indicate that func did its job ok (and return some useful result, e.g., number of bytes written etc)

Just FYI: The "smiley" |_| is the equivalent to lambda x: in python (where x will be ignored from now on) and map_error just substitutes the error part of a Result for the evaluation of said lambda.

As for the rest of your argument, you are still sustaining that you and all other devs are better in knowing what has to be unwound than the compiler is. So, I'll refer to my other answer at this other comment.

Rust heads into the kernel?

Posted Apr 23, 2021 1:03 UTC (Fri) by roc (subscriber, #30627) [Link] (7 responses)

And there are really people who can parse such horrors ?
Yes. It's far less "horrifying" than, say, C type syntax.
How do you decide to atomically increment an error counter on the return path with that method ?
You can always manually write out
  if func().is_err() {
    increment_error_count();
    return Err(errno::EINVAL);
  }
or
  if let Err(e) = func() {
    if ... something involving e ... {
      increment_oversized_frame_count();
    } else if ... something involving e ... {
      increment_undersized_frame_count();
    }
    return Err(errno::EINVAL);
  }
This looks like eye-candy for dummies to me, just to attract beginners by claiming they'll write easier code but it's hard to imagine it can ever be useful outside of school books.
People write lots of real code using Rust, including Rust "Result" and "?". I have a product with 200K lines of Rust code written over the last 5 years and we use these features everywhere. I also have 30 years experience writing C and C++ code if that matters to you.

Rust heads into the kernel?

Posted Apr 23, 2021 12:07 UTC (Fri) by wtarreau (subscriber, #51152) [Link] (6 responses)

Well at least your examples look more readable in terms of control.

Rust heads into the kernel?

Posted Apr 23, 2021 12:30 UTC (Fri) by hummassa (subscriber, #307) [Link] (5 responses)

The problem is exactly that. You are arguing that the kernel shouldn't have
  long x = input & SOME_FLAG ?
    (input & SOME_MASK) + SOME_INITIAL_VALUE :
   other_input & SOME_OTHER_FLAG || OTHER_INITIAL_VALUE;
because idiomatic C is "too terse" and "hides control flow". After all, how can you see the two if that are lurking in this smiley-laden code?

Rust heads into the kernel?

Posted Apr 23, 2021 14:01 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (4 responses)

Not to mention with that particular code, I have to dredge up my C operator precedence to figure out if that `|| OTHER_INITIAL_VALUE` is a bug or not. Is it supposed to be `|`? Is it supposed to be with the false branch or the whole conditional expression?

Rust heads into the kernel?

Posted Apr 23, 2021 14:34 UTC (Fri) by wtarreau (subscriber, #51152) [Link]

I totally agree with you and am not writing C code like this either, just like I don't like seeing assignments inside conditions.

Rust heads into the kernel?

Posted Apr 23, 2021 18:09 UTC (Fri) by hummassa (subscriber, #307) [Link] (2 responses)

The short-circuiting logical-or operator || is defined as "evaluate LHS, if it's a truth value (none of false, 0, '\0', nullptr) return that, else return RHS". It binds more tightly than the ?: ternary conditional. The whole statement, explicitly written, would be:
  long x;
  if( input & SOME_FLAG ) {
    x = (input & SOME_MASK) + SOME_INITIAL_VALUE;
  } else {
    x = other_input & SOME_OTHER_FLAG;
    if( !x ) {
      x = OTHER_INITIAL_VALUE;
    }
  }

Rust heads into the kernel?

Posted Apr 23, 2021 18:30 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (1 responses)

This makes me think it's a typo because `expr || OTHER_INITIAL_VALUE` always returns 0 or 1, not `OTHER_INITIAL_VALUE`.

#include <stdio.h>

#define p(x) printf(#x " = %d\n", x)

int main() {
p(0 || 2);
return 0;
}

outputs:

0 || 2 = 1

Rust heads into the kernel?

Posted Apr 23, 2021 18:37 UTC (Fri) by hummassa (subscriber, #307) [Link]

That one was MY mistake. Wrong language. It's not a typo, it's an honest-to-god mistake and I would have introduced a kernel bug, if the code review didn't see it. The || operator always return a bool and it's zero or one.

Rust heads into the kernel?

Posted Apr 21, 2021 7:38 UTC (Wed) by LtWorf (subscriber, #124958) [Link]

Well since this seems pushed by google. They have all the interest in having the kernel not change API so that the android vendors can easily port their drivers across versions.

Rust heads into the kernel?

Posted Apr 21, 2021 20:08 UTC (Wed) by tbelaire (subscriber, #141140) [Link]

I was just a undergrad student and I worked wiht a grad student to re-write (maybe translate would be a better word) a block device driver into Rust 4 years ago, and gave a talk at Linux con.

So we've been working on it for a while now, including writing drivers.

(Turns out Rust wasn't quite ready back then, but the hard part is definitely the safe abstraction layer.)


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds