Rust in the Linux kernel (Google security blog)

Posted Apr 15, 2021 19:35 UTC (Thu) by mss (subscriber, #138799)
In reply to: Rust in the Linux kernel (Google security blog) by Cyberax
Parent article: Rust in the Linux kernel (Google security blog)

> In many cases it's faster than C++ because safe C++ would require suboptimal constructs.

That's a very general statement.

> In C++ you have to pessimise and use shared_pointer if you want the structure to be useful in a generic case because there's no way to tell
> "this object's lifetime is now controlled by the parent structure".

This is normally controlled by object (class) hierarchy.

Usually, when a class needs to store a pointer to an object it is because it has been transferred ownership of that object.
Then unique_ptr is enough.

Sometimes a class needs access to something in one of its parent classes.
Then either a raw pointer to that object should be stored (by the child class constructor, to force proper construction order at the parent class) or, alternatively, the child class should store a (raw) pointer to its parent object which implements a getter method for that thing.
In that case, one has to think whether using inheritance instead of composition would make more sense there.

A shared_ptr is necessary only in the case a class wants to "pin" an object of unspecified lifetime (with respect to its own lifetime).
But at that point reference counting is pretty much always necessary anyway.

Blindly using shared_ptr everywhere, besides the performance impact, is a circular reference bug waiting to happen.

> > Pretty much every large C++ project does not use exceptions so there is no requirement for kernel to use them, either.
> Exactly. And this is a problem.

Are you suggesting that kernel *should* make use of C++ exceptions?

Rust in the Linux kernel (Google security blog)

Posted Apr 15, 2021 20:17 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (20 responses)

> This is normally controlled by object (class) hierarchy.
Nope. C++ does offer a somewhat streamlined model for ownership of data (copy/move constructors), but it fails utterly with flexible ownership. There are third-party tools (e.g. boost::pointer_container) but they are all kinda awkward.

And sorry, but inheritance and OOP have nothing to do with that.

> A shared_ptr is necessary only in the case a class wants to "pin" an object of unspecified lifetime (with respect to its own lifetime). But at that point reference counting is pretty much always necessary anyway.
It can be avoided in Rust most of the times.

> Are you suggesting that kernel *should* make use of C++ exceptions?
No, that C++ should be redesigned to actually be fully usable without exceptions.

Rust in the Linux kernel (Google security blog)

Posted Apr 15, 2021 21:36 UTC (Thu) by bvanassche (subscriber, #90104) [Link] (19 responses)

>> Are you suggesting that kernel *should* make use of C++ exceptions?
> No, that C++ should be redesigned to actually be fully usable without exceptions.

I think that C++ is fully usable without exceptions. A great and efficient alternative for exceptions and error codes is available in the Abseil library. See also Abseil Status.

Rust in the Linux kernel (Google security blog)

Posted Apr 15, 2021 22:41 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (18 responses)

Note that this means giving up on some core bits (`new`) and needing a new algorithms library (because the STL algorithms' only "please stop, we found a problem" is an exception). Ranges might help with this, but I've not used them myself.

Rust in the Linux kernel (Google security blog)

Posted Apr 22, 2021 7:32 UTC (Thu) by ncm (guest, #165) [Link] (17 responses)

There are in fact no algorithms in <algorithm> that rely on throwing exceptions. Users may provide callbacks that throw exceptions, but that is wholly voluntary and anyway, in my experience, never necessary.

We don't need falsehoods here.

Rust in the Linux kernel (Google security blog)

Posted Apr 22, 2021 11:36 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (5 responses)

> There are in fact no algorithms in <algorithm> that rely on throwing exceptions.
That's false. Plenty of algorithms in <algorithm> rely on exceptions. E.g. std::copy_n with a back_inserter will throw on out-of-memory.

Rust in the Linux kernel (Google security blog)

Posted Apr 23, 2021 17:06 UTC (Fri) by ncm (guest, #165) [Link] (4 responses)

std::back_inserter is not an algorithm.

If you choose to specialize std::copy_n on it—as for any callback—exceptions are your responsibility. Normally, if one does not want exceptions from std::back_inserter, one calls reserve() on the container, first, to establish enough storage, which is anyway more efficient; or, for nodal containers, construct it with an allocator and make sure the allocator had enough in reserve.

Taking responsibility for what your code does is something we call programming.

Falsehoods do not improve the discourse here.

Rust in the Linux kernel (Google security blog)

Posted Apr 23, 2021 18:55 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

> Normally, if one does not want exceptions from std::back_inserter, one calls reserve() on the container, first, to establish enough storage, which is anyway more efficient; or, for nodal containers, construct it with an allocator and make sure the allocator had enough in reserve.
"reserve" requires an upfront knowledge of the resulting size. But even knowing the resulting size is not enough, because copy constructor itself might throw (e.g. copying a string and getting an OOM).

C++ absolutely relies on exceptions and working around them results in a lot of ugly code.

> Taking responsibility for what your code does is something we call programming.
If everybody actually took responsibility for what they write, C/C++ programmers would be in prison, serving lifetime sentences for the reckless endangerment.

Rust in the Linux kernel (Google security blog)

Posted Apr 23, 2021 19:40 UTC (Fri) by mss (subscriber, #138799) [Link] (2 responses)

> C++ absolutely relies on exceptions and working around them results in a lot of ugly code.

It's STL that you are talking about, not the core C++ language.

This distinction is important when considering C++ for kernel usage - we would probably only use parts of STL,
after adapting it for this environment.

> If everybody actually took responsibility for what they write, C/C++ programmers would be in prison, serving lifetime sentences for the reckless endangerment.

That kind of opinion doesn't help keeping the discussion technical and civil.

Rust in the Linux kernel (Google security blog)

Posted Apr 23, 2021 19:45 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link] (1 responses)

> It's STL that you are talking about, not the core C++ language.
I disagree. Core C++ also relies on exceptions. It's the only way to return errors from constructors, for example. With copy constructors being the worst offenders.

It's fair to say that it's possible to work around that issue, but then you'll lose most of the STL (including std::string). And you code will quite often not be idiomatic C++.

> That kind of opinion doesn't help keeping the discussion technical and civil.
Yet it's true. And I say that as a C++ developer.

I believe that writing in C++ is absolutely irresponsible at this point, and the whole safety culture needs to change.

Rust in the Linux kernel (Google security blog)

Posted Apr 23, 2021 19:59 UTC (Fri) by mss (subscriber, #138799) [Link]

> It's the only way to return errors from constructors, for example.

Many C++ projects manage to use this language without resorting to exceptions.

It depends on a class, but it it often possible to initialize object to a dummy state in case of an unexpected error in a constructor
(like a NULL pointer in case of a smart pointer template, empty string for a string template, etc.).

Rust in the Linux kernel (Google security blog)

Posted Apr 22, 2021 13:27 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (10 responses)

I didn't say *rely*. I said that if you have errors that can occur while processing the iterator range, you need to throw an exception to *abort the algorithm*. They support gathering all the errors, but if you're just caring about *any* error, stopping the algorithm from doing useless work is pretty important.

As a concrete example, say I have a fallible function call that I want to call on all elements within an iterator range, bailing on the first problem and returning the good values into an output iterator. What <algorithm> do I use for this? AFAIK, all you can do is throw an exception to escape the algorithm. Basically, I want the behavior of this open-coded loop using `std::transform`. AFAIK, turning the `break` into a `throw` is all you have.

std::vector<T> output;
output.reserve(xs.size());
bool err = false;
for (auto const& x : xs) {
  if (!validate(x)) {
    err = true; // probably richer in real code
    break;
  }
  output.emplace_back(process(x));
}
if (err) {
  // handle
}

In Rust, I have:

let res = xs.iter().map(|x| if !validate(x) { Err(()) } else { Ok(process(x)) }).collect::<Result<Vec<_>>>();
// res Ok/Err status gives me all results or the first error (tossing good results that have already been made)

Rust in the Linux kernel (Google security blog)

Posted Apr 22, 2021 16:39 UTC (Thu) by hummassa (subscriber, #307) [Link] (5 responses)

C++20 has take_while, and your example would be like

auto output = xs | take_while(&validate) | transform(&process);

One could argue, though, if your Err() indicates a real error condition, throwing would be the right thing to do™ and that there are lightweight (zero-overhead) implementations of exceptions for freestanding C++ programs (e.g. kernels).

Rust in the Linux kernel (Google security blog)

Posted Apr 22, 2021 16:54 UTC (Thu) by hummassa (subscriber, #307) [Link]

Oh, and obviously, you can always implement kernel::outcome and do

auto output = xs | transform([](const auto& x){ return validate(x) ? process(x) : ValidationError(x); }) |
  collect_or_first_error();

Rust in the Linux kernel (Google security blog)

Posted Apr 22, 2021 17:54 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (3 responses)

Ah, that is ranges (which I'm not that well-versed in). That does indeed vastly improve things. Unfortunately such things aren't available until C++20, but that's just how it is sometimes (most of the C++ projects I work on are only *considering* C++14 right now due to their compiler support matrix).

Rust in the Linux kernel (Google security blog)

Posted Apr 22, 2021 21:22 UTC (Thu) by hummassa (subscriber, #307) [Link] (1 responses)

The following questions arise: Isn't a recent Rust compiler the minimum to compile Rust modules? Why should C++ be different?

Rust in the Linux kernel (Google security blog)

Posted Apr 23, 2021 0:46 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

Various popular ones? Sure. Given that the kernel is likely in `no_std` land, the minimum is probably way lower than "typical" Rust is likely to have.

Rust in the Linux kernel (Google security blog)

Posted Apr 22, 2021 21:24 UTC (Thu) by hummassa (subscriber, #307) [Link]

(Ah, and ranges are a library-only update, so, no problem in implementing them even in C++11 for a freestanding kernel toolchain)

Rust in the Linux kernel (Google security blog)

Posted Apr 23, 2021 17:24 UTC (Fri) by ncm (guest, #165) [Link] (3 responses)

I have used std::find_if to walk a sequence and stop short, with no exceptions thrown or caught. It is quite easy and clean with a lambda.

And, of course, it is very, very easy to code your own variations, typically needing not more than one or two lines in the body. (Such an algorithm used very frequently should be proposed for the library in an upcoming Standard.)

This is a practice we call programming. It is not fundamentally different from in other languages, but in C++ is generally more fun.

When you have had to reach so far off your center of gravity to try to support an argument, and fail anyway, it is usually better to concede the point.

Rust in the Linux kernel (Google security blog)

Posted Apr 23, 2021 18:15 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

I guess I could have made the example more robust. But if `validate` is a condition on the transformation result, what is the recommended way? Say the body was:

{ auto transform = process(x); if (is_valid(transform)) { v.emplace_back(std::move(transform)); } else { break; } }

I suspect it needs something like Rust's `try_for_each` kind of algorithms, but I don't know for sure. Maybe std::outcome or whatever it ends up being will help here, but I suspect it'll just be "use ranges".

> but in C++ is generally more fun.

To each their own. But I also find Python not all that *fun* to work in, so tastes are all over the place.

> When you have had to reach so far off your center of gravity to try to support an argument, and fail anyway, it is usually better to concede the point.

Ranges just haven't been in my purview. Yes, they probably (basing on what I'm seeing at least) do make things as easy to compose and work as Rust's Iterator trait. But, C++ is C++ and getting everyone to move to the new version is difficult (and bringing in dependencies to polyfill isn't easy either). I'm glad that there's something new, but I think my point stands that if you have error handling you want in <algorithm> you're either stuck with iterating multiple times (which might not work if your iterator isn't over a view, but instead consumes like a stream iterator) or throwing an exception for early termination (or open-coding which I guess works too, but I find named algorithms easier to read).

Rust in the Linux kernel (Google security blog)

Posted Apr 24, 2021 23:46 UTC (Sat) by flussence (guest, #85566) [Link] (1 responses)

Then please do us all a favour and concede. This entire thread tree was painful to read and reminds me of the OpenOffice ones.

If this kind of broken-record conversational sandbagging is representative of the C++ community then I'm glad the language is being pushed out.

Rust in the Linux kernel (Google security blog)

Posted Apr 25, 2021 16:59 UTC (Sun) by corbet (editor, #1) [Link]

Perhaps everybody involved should let this conversation go at this point...I don't set it shedding any real light if goes any further.