LWN: Comments on "Scope-based resource management for the kernel"

Scope-based resource management for the kernel

ksandstr — Thu, 13 Jul 2023 22:26:08 +0000

>As I said here in May it would be nice if the defer feature did make to the next C standard after C23 - it would allow implementing these automatic cleanups without having to resort to non-standard compiler extensions.

The problem with defer, as it was proposed for C23, was that it was conjoined with a nonlocal exit syntax ("panic") which were effectively C++/Java/Ada exceptions by a different name. And it makes sense: with autocleanup it'd be desirable, and not a large leap, to prefer acquisition code to run without introducing silly errors in the error checks which must occur at every (re-)turn. Not that routine unlikely()[0] didn't take care of the performance issue already.

The downside was that this would also pull in lambda syntax which makes it three features instead of one, the bonus ones being a nonlocal exit mechanism that can't be audited against by grepping for <setjmp.h>, and all the fun and games that follow from first-class anonymous functions in a world where trampolines can't be emitted on the stack for security reasons. (imagine arguments about elevator controllers with 256 bytes of modifiable RAM, here.) All three put together would've made that proposal of C23 so radically different, and so hitherto unexplored, that those ideas were tabled until the fashionable feature fever had passed -- presumably to be reintroduced in the C3x process.

[0] ... and also the CPU advances of 1996, i.e. branch prediction in combination with an instruction window.

Scope-based resource management for the kernel

ksandstr — Thu, 13 Jul 2023 21:45:28 +0000

Alternatively one can place resource acquisition and release in a function, and the operation on those resources in a second function, which is then at liberty to early-exit as much as it jolly pleases. A single-caller static function will be inlined by the compiler, yielding much the same result as a block of "Enomem: ret = -ENOMEM; goto cleanup;" lines.

More related to the automatic cleanup as discussed in the article, I have to wonder whether the utility of autocleanup is commensurate with the number of possible errors in usage that these structures harbour, and the length of time it'll take before kernel review is up to the task of catching them as well as mistakes in non-compiler cleanup. Forgetting the __highly__((__underscored__)) mess of attributes, which must go after each variable declaration[0] where autocleanup is desired, is just the tip of the iceberg. Could there please be some kind of a linter program to highlight suspicious cases such as "return ptr;" when "return_ptr(ptr);" was intended, so that this glass tower doesn't become a source of the next dozen years' worth of exploitable use-after-frees? A cleanup sanitizer pass in GCC perhaps?

To contrast, in Ada[1] the equivalent of an autocleanup property is associated with a type by specifying it as an extension of a "controlled" base type, and is thereafter transparent[2] to the programmer whether such controlled objects are passed around by copy, out-parameter, array slice, or whatever. This entirely eliminates usage errors in cases where the object is treated as though it contained no pointers, if its implementations of the "dispose" and "post-copy rejigger" methods are correct. Seeing that the amount of diddle involved in approximating something similar in C is about the same, only to give up some of C's flexibility and transparency, I have to wonder if this metaprogramming exercise is worth using "in anger".

[0] which appears to restrict variable declarations to one per line, not unlike the coward's maladaptation to C declaration-follows-use pointer syntax.
[1] please note that this is not in support of its use in the kernel; far from it.
[2] in the sense of "invisible but tangible".

Scope-based resource management for the kernel

bluss — Sun, 02 Jul 2023 17:05:49 +0000

Non-trival C++ functions have two exit paths: the return statement (if a single one) and when unwinding an exception. To be fully explicit, one needs to write out destructor calls that happen in both those cases(!).

Scope-based resource management for the kernel

anton — Mon, 26 Jun 2023 14:36:09 +0000

Compiler optimizations in C and C++ aren't fundamentally different

My impression is that they are, and that the pretence that programs do not exercise standard-undefined behaviour comes, for a large part, from C++ programmers, while those who use options like -fno-strict-aliasing to get rid of "optimizations" that are based on this pretence are, for a large part, C programmers.

In earlier discussions on the topic, several advocates of these "optimizations" argued that they are very useful for not-quite-source code coming from expanding the original source code (typically when I pointed out that nobody writes programs in a way that the advocate just showed as an example of the usefulness of the "optimization"), and for the same reason that it's not a good idea to warn when the "optimization" strikes (supposedly there is so much not-quite-source code where the "optimization" strikes that one supposedly would be flooded with warnings). Both claims are very doubtful for even the most macro-laden C code, so I assumed the advocates had C++ code in mind; or maybe the advocates were just bullshitting me.

Anyway, I wonder how much the fact that GCC and Clang are written in C++ contributes to adding these kinds of "optimizations" (especially adding them (at first) without a kill switch).

Scope-based resource management for the kernel

ssmith32 — Mon, 26 Jun 2023 06:13:21 +0000

Then it should all be implemented in the editor.

Scope-based resource management for the kernel

kreijack — Sat, 24 Jun 2023 08:31:12 +0000

> It is not classic resource releasing. The compiler verifies that a destructor with extra arguments is called exactly at places in the code where an automatic destructor is called.

> If there are several things that need to be destructed, then the compiler still inserts automatically calls to parameter-less destructors before or after the explicit call in the reverse order of construction.

This seems more error prone than any other solution. A mix of actions of the developers and the compiler. Still doesn't solve the issue of calling a destructor with a wrong parameter.

Scope-based resource management for the kernel

ibukanov — Fri, 23 Jun 2023 04:55:24 +0000

It is not classic resource releasing. The compiler verifies that a destructor with extra arguments is called exactly at places in the code where an automatic destructor is called.

If there are several things that need to be destructed, then the compiler still inserts automatically calls to parameter-less destructors before or after the explicit call in the reverse order of construction.

Scope-based resource management for the kernel

marcH — Thu, 22 Jun 2023 21:02:46 +0000

> There are some problems to which object-orientation maps well but C++ went really "all-in" and kept functional programming techniques out of developers' mindsets for much too long of a time, holding back the entire industry for decades.

Forgot the mandatory reference to the masterpiece, sorry for the extra email:

http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom-of-nouns.html

Yes this one is not about C++ but Java which is even worse. But: close enough.

Scope-based resource management for the kernel

marcH — Thu, 22 Jun 2023 20:51:05 +0000

> So you really think that inheritance and virtual functions would not be one of the first next C++ features that people would demand, if we would compile the kernel with a C++ compiler?

Of course it would; implementing classes in C is not fun at all and it's done all over the kernel already: https://lwn.net/Articles/444910/

BTW I suspect this is another reason people want to keep C++ out: Object-Oriented Obfuscation. There are some problems to which object-orientation maps well but C++ went really "all-in" and kept functional programming techniques out of developers' mindsets for much too long of a time, holding back the entire industry for decades.

Funny how Javascript out of all languages helped functional programming finally break out.

Rust is of course more functional than OO.

Scope-based resource management for the kernel

marcH — Thu, 22 Jun 2023 20:42:35 +0000

> If that's your main issue,

Not "mine"; the issue of everyone who does not want C++

> then luckily it's a non-issue.

Must feel good to be right when so many people are wrong :-)

> my point is that for this feature, C++ is a better tool than C with non-standard extensions.

Yes! But: see above.

> Discussions which C++ features are allowed in the kernel is no different than any other coding style discussion.

Qualitatively yes. Quantitatively no. I know: I don't have any metrics to show you; these things are very hard to measure. But you don't have any either and the consensus / educated guess in Linux (and some other projects) is that the trade-off is not worth it. That is of course based on observations in other, actual C++ projects, I mean the lack of metrics does not mean it's based on thin air either.

BTW: which features are allowed is IMHO a small part of code style discussions. A very important part of course, maybe the most important one but small in "email volume". I'm just saying "code style" is not a great choice of words here, never mind.

> That's a straw man. Let's discuss the language rather than discussing theoretical arguments from hypothetical C++ fanboys

Oh come on: "you're holding it wrong" is the typical answer of _every_, _real_ C++ fanboy! Very vocal and hopefully not representative as usual on the Internet but certainly not "hypothetical".

> I know very well that C++ is hard, and I admit it's a complex mess of a language.

Thank you!

Scope-based resource management for the kernel

kreijack — Thu, 22 Jun 2023 19:52:19 +0000

> I’m surprised, but excited, that the kernel is open for this feature. (I believe systemd extensively uses it, for instance.)

I am curious if they considered some prior art (like systemd) and a compatible implementation. For example systemd use take_ptr, where in this example it is used free_ptr.

This would help the portability of the code and pushing to standardize some features.

Scope-based resource management for the kernel

kreijack — Thu, 22 Jun 2023 19:47:07 +0000

> Any extra arguments you would want to pass to a destructor function (i.e. besides the pointer to the object being destroyed) can be transformed to/from a model where those arguments are kept around as members of the object. Both approaches make an appearance in the POSIX environment:

> 1. void *p = mmap(..., z, ...); /* workworkwork */; munmap(p, z);
> 2. void *p = malloc(z); /* workworkwork */; free(p);

> It's a tradeoff. #2 needs memory in each object to store the 'z' used during construction, while #1 needs CPU instructions to load 'z' from some memory location into the function call.

I am not sure that these can be called 'destructor'; a destructor is a way to release automatically the resource. If you have to call explicetely it is not a destructor but is a 'classic' resource releasing.

If the releasing is automatic, you cannot forgot to do that.

Instead if you have to pass a parameter to a destructor you are introducing another potential mistake: what if in your example you call munmap(p, y) (note y as last argument) ?

Using a destructor, or __attribute__((__cleanup__)) are not magic bullet that solve all the issues. A more interesting example is what if two abject are linked and have to be released together (i.e. explicitly). A destructor cannot solve this kind of issue, because it assume that the objects are independent each others, and the destructorS can operate independently.

Scope-based resource management for the kernel

mb — Thu, 22 Jun 2023 19:41:25 +0000

>but the code flow is the same as with other ways of doing dynamic dispatch (no matter if C++, C or Rust).

No it isn't.

Scope-based resource management for the kernel

make — Thu, 22 Jun 2023 19:35:03 +0000

> Nice that you agree.

No, I don't agree. Virtual functions don't affect the code flow; they are just syntactic sugar for dynamic dispatch, but the code flow is the same as with other ways of doing dynamic dispatch (no matter if C++, C or Rust). But, uh, we're now even more off-topic, you're dragging me away from the topic (=RAII) so much, that I'm confident you're just a troll, and I've already been feeding you too much, shame on me. That was unexpected on LWN.

Scope-based resource management for the kernel

mb — Thu, 22 Jun 2023 19:19:39 +0000

>only to prove that code flow in C++ and Rust is different after all

Nice that you agree.

Scope-based resource management for the kernel

make — Thu, 22 Jun 2023 19:07:28 +0000

> No. But you didn't read my full text, before answering on the first paragraph.

Oh yes, you were missing the point, because instead of replying to the point I made (about RAII), you made up a new story and replied to this story of yours.

Now you came up with a new story, this time about virtual functions, only to prove that code flow in C++ and Rust is different after all, but that again misses my point. Don't you get it? This article and this thread is about RAII and nothing else.

Scope-based resource management for the kernel

mb — Thu, 22 Jun 2023 18:55:56 +0000

>You're missing the point.

No. But you didn't read my full text, before answering on the first paragraph.

>What makes you think policies about acceptable C++ features are different?

So you really think that inheritance and virtual functions would not be one of the first next C++ features that people would demand, if we would compile the kernel with a C++ compiler?

Converting all manual implementations of virtual function calls is the next obvious transistion we would have.
And there goes your same code flow.

Scope-based resource management for the kernel

make — Thu, 22 Jun 2023 18:38:38 +0000

> No, not at all. Rust has no inheritance. Code flows between ideomatic C++ and ideomatic Rust are quite different.

You're missing the point. The article and this thread is about using RAII in the Linux kernel, not about inheritance. RAII works the same in C++ and Rust (and in C with GNU extensions, as it's going to be used soon).

The fact that C++ has features that Rust has not (which I didn't suggest to use and which do not affect the code flow) does not falsify my argument - it doesn't even have anything to do with this thread.

> It does not work to say that we're going to switch to C++ but only use RAII from it.

It works with other C language features or Rust language features or coding style policies or anything else. What makes you think policies about acceptable C++ features are different? Just because of "arguing over and over again"? Why is arguing over C++ features different? Bikeshedding discussions have existed as long as Linux exists, and will always exist. This argument seems rather arbitrary, because you can use the same argument to reject C11, C17, Rust (or anything else).

> The C implementation might not be as pretty as C++

Agree. We don't agree on the rest, but that's fine.

Scope-based resource management for the kernel

rwmj — Thu, 22 Jun 2023 11:10:17 +0000

The defer feature as proposed for C23 was massive over-engineering and not even good. What the C committee needs to do is standardize __attribute__((cleanup)).

Scope-based resource management for the kernel

mb — Thu, 22 Jun 2023 10:32:43 +0000

>because the source code flow in C++ and Rust is similar.

No, not at all.
Rust has no inheritance. Code flows between ideomatic C++ and ideomatic Rust are quite different.

It does not work to say that we're going to switch to C++ but only use RAII from it.
People *will* start using more and more of C++'s features.

It's better to keep that huge amount of stuff out of the kernel to begin with.
The C implementation might not be as pretty as C++, but pulling in C++ comes with a tremendous cost of arguing over and over again why this and that C++ feature shall not be used in the kernel.

Scope-based resource management for the kernel

make — Thu, 22 Jun 2023 09:24:39 +0000

> This is the main issue: define "subset" and "desirable". Everyone will have different answers (and C++ has everyone's favorite features).

If that's your main issue, then luckily it's a non-issue.

The premise of the article is that the kernel developers have already decided they want (something like) RAII, and my point is that for this feature, C++ is a better tool than C with non-standard extensions.

Discussions which C++ features are allowed in the kernel is no different than any other coding style discussion. There are already discussions on which C standard shall be used and which new C standard features are acceptable in the kernel (like atomics). Nothing new here, just more of the usual.

> Another top problem is: "C++ is not hard, your developers are just holding it wrong".

That's a straw man. Let's discuss the language rather than discussing theoretical arguments from hypothetical C++ fanboys, because that's not relevant for the language choice.

About C++: I know very well that C++ is hard, and I admit it's a complex mess of a language. Many reasonable things can be said about the ugliness of C++.

If you're starting with C, and you want to use certain advanced features like RAII, C++ is a good choice, because the migration is so simple. It is certainly reasonable to say Rust is better than C++, and I don't argue C++ should be used *instead* of Rust - I only say C++ could easily be used *now* to improve code quality where a Rust migration is not possible currently. Additionally, a future rewrite to Rust is easier from C++ with RAII than from plain C, because the source code flow in C++ and Rust is similar. C++ is actually a good intermediate language for a smoother migration to Rust.

Scope-based resource management for the kernel

marcH — Thu, 22 Jun 2023 08:28:17 +0000

> On the other hand, allowing a C++ (subset) would make all desirable C++ features...

This is the main issue: define "subset" and "desirable".
Everyone will have different answers (and C++ has everyone's favorite features).

Another top problem is: "C++ is not hard, your developers are just holding it wrong". I've called this the "mythical workplace" argument.

Scope-based resource management for the kernel

mb — Thu, 22 Jun 2023 06:28:32 +0000

>if a bug in the compiler

Is there any sign that such a bug ever existed w.r.t. cleanup routines?

Scope-based resource management for the kernel

irogers — Thu, 22 Jun 2023 05:39:16 +0000

I like scope-based resource management but this is only solving a problem with local variables within the scope of functions. Full RAII would allow destructors to run say when a variable in the heap was overwritten by NULL. We can't have a factory returning memory with a cleanup on the return type, and the caller must be aware to do this.

I worry about the cleanup attribute. Weak is another attribute, outside of the C spec, that is widely used. However, in contexts like the perf tool, it breaks (legitimately) gcc's LTO. The perf tool has had bugs with weak and const on global variables allowing optimizations that break code. The bugs with weak have always been subtle and I can imagine that cleanup's bugs will be similar. Moving to the cleanup approach means that if a bug in the compiler does occur the changes may need to be reverted or only the latest compilers can build the kernel, neither option sounds appealing.

Fwiw, a similar macro magic hack that is related is the reference count checking work in the perf tool. The approach allocates memory on a get, the contents of the memory pointing to the original object, and on a put it releases the memory. A get without a put is a memory leak. A use after put is a use-after-free/fault. Leak sanitizer will identify all places where the gets with no puts occur with their stack traces. Unlike cleanup it works for any kind of variable but it is invasive, requires effort by the user and leans heavily on leak sanitizer.

Scope-based resource management for the kernel

rahulsundaram — Mon, 19 Jun 2023 12:32:47 +0000

> You already know the answer to that question, so why ask it? Your trolling doesn't add anything useful to the discussion.

Bringing up C++ when it is never going to be used in the Linux kernel adds nothing to the discussion in the first place.

Scope-based resource management for the kernel

zorro — Mon, 19 Jun 2023 11:38:56 +0000

You already know the answer to that question, so why ask it? Your trolling doesn't add anything useful to the discussion.

Scope-based resource management for the kernel

geert — Mon, 19 Jun 2023 09:15:44 +0000

modern = gcc-4.1? ;-)

That's the reason I kept on compiling kernels with gcc-4.1 for a while, even after the bar was raised to gcc-4.6...

Scope-based resource management for the kernel

alison — Mon, 19 Jun 2023 03:58:52 +0000

>As I said here in May it would be nice if the defer feature did make to the next C standard after C23 - it would
>allow implementing these automatic cleanups without having to resort to non-standard compiler extensions.

Given that the kernel has only recently updated to C11, it's unlikely that C23 would have been adopted even if it included defer. It's too bad that defer has been reinvented as a long series of macros, although everyone will be glad to see functions that are 2/3 error-handling goto's and associated labels get shorter. Perhaps the new macros could significantly reduce lines of code in source files that are full of error handlers.

Scope-based resource management for the kernel

tialaramex — Mon, 19 Jun 2023 00:32:09 +0000

Linear aka Use-exactly-once types. C++ isn't very well suited for this kind of types. Neither is Rust. There are languages which thrive on linear types and a few newer ones are purpose built for these types.

Scope-based resource management for the kernel

nksingh — Sun, 18 Jun 2023 21:13:12 +0000

I would instead prefer a paradigm like WordPerfect's 'reveal codes', where the compiler and editor can display the code that the compiler is injecting as an option rather than needing to have the code visible at all times.

The advantage of producing the code as needed for review/debugging rather than requiring it to be explicit: when editing, your code remains flexible and modifiable with the compiler doing the bookkeeping at the end, rather than you having to compensate for changes manually before you can even test your code out. I think for domains like embedded, high performance, and kernel code, this transform between 'debuggable' and 'editable' source code would help satisfy both people who want productivity and those who want to know exactly what will happen in the CPU.

Scope-based resource management for the kernel

ibukanov — Sun, 18 Jun 2023 14:46:36 +0000

The idea is that the compiler requires explicit destructor calls exactly at the places where presently it inserts it automatically.

Scope-based resource management for the kernel

tialaramex — Sun, 18 Jun 2023 13:38:22 +0000

This "subset" is a trick. It's a good conjuring trick, but it isn't real. You are actually obliged to have the whole C++ language, you can choose not to use all of it, but it's all still there anyway dragging you down. It reminds me of the excuse used for C++ move like fifteen years ago when it was just an idea not a language feature yet.

See, C++ move is just strictly worse than Rust's "destructive" move which was already a known idea. But you can construct an argument where the C++ move looks more fundamental, you say well this "destructive move" is move + destroy, if we wanted that in C++ we just do move and then destroy, so that's better 'cos we had the choice. Except, it turns out that's not actually how it works. Move is destructive by its nature on real computers, so in C++ you must have code to construct a dummy value so as to achieve the non-destructive move, whereupon of course if you did want it destroyed you also then need to destroy your dummy value.

Scope-based resource management for the kernel

tialaramex — Sun, 18 Jun 2023 12:38:40 +0000

Sentiments from WG21 (the C++ committee) members don't seem far removed from what you're describing here as "hostility". Their conclusions are different but the reported facts seem similar.

The remaining C++ proponents continue to believe in a sort of Phlogiston Theory for programming languages. If they can just add the right things to C++ then it will not be an over-complicated language any more. It's true that in C++ 11, C++ 14, C++ 17, C++ 20 and C++ 23 the additions did make the language even more complicated each time, but surely this time it will work.

Scope-based resource management for the kernel

make — Sun, 18 Jun 2023 06:01:09 +0000

> much longer build time

Switching GCC from C to C++ mode alone has no effect on the build time. There are C++ features that do add to the build time, like excessive use of templates, but so do C features like excessive use of macros. It depends on which C++ subset you allow.

I bet there's no build-time difference between Peter's pseudo-RAII using macros and the GNU extensions and "true" C++ RAII (using no other C++ feature).

Scope-based resource management for the kernel

tialaramex — Sun, 18 Jun 2023 02:24:38 +0000

So where is it then? Where is your C++ Linux kernel?

Scope-based resource management for the kernel

Cyberax — Sat, 17 Jun 2023 23:11:17 +0000

> There has been a few patch sets posted to allow C++ usage in the Linux kernel

It used to be possible to compile Linux with C++ enabled, back in the early 2.6 days. I remember because I worked with a hardware device that had a C++ kernel module.

Then the kernel started using "class" and "operator" as regular variables.

Scope-based resource management for the kernel

mss — Sat, 17 Jun 2023 22:29:02 +0000

There is no hostility.

Expressions like C++ is a horrible language or the crap that is C++ or (more recent) It really is a crap language certainly sound like hostility.

Scope-based resource management for the kernel

mb — Sat, 17 Jun 2023 21:22:02 +0000

There is no hostility.
It's just that the advantages of using C++ are not seen by kernel developers as being significant enough to justify the downsides of having C++. (e.g. much longer build time).

Scope-based resource management for the kernel

mss — Sat, 17 Jun 2023 20:57:33 +0000

The Linux source code is readily available though, so you can try this out for yourself and make sure not to go talking about this fantasy again until you've actually found out what the reality is like and have something to show for your effort.

There has been a few patch sets posted to allow C++ usage in the Linux kernel - the most recent one AFAIK was posted as an April Fools' joke 5 years ago.

Although, given the well-known upstream hostility to C++, people find it hard to justify the amount of work required to prepare a serious RFC.

Scope-based resource management for the kernel

make — Sat, 17 Jun 2023 19:36:20 +0000

> There speaks the user-space programmer who thinks the solution for a slow program is to throw hardware at it.

Your whole post sounds awfully arrogant (and wrong). It doesn't add any content to the discussion, you only bash somebody you don't know with bogus and strawman arguments.

Compiler optimizations in C and C++ aren't fundamentally different, and aren't fundamentally different between userspace and kernel space. Yes, there are differences in both aspects, but not in a way that adds to the difficulty like you pretend it does.

> How much effort / often times have you had to optimise a program for speed? It's an art that most young people just don't understand.

Every day for several decades, and I wouldn't call it "art", because it's deterministic and mechanical, the simplest part of my job. Yes, many people don't value fast/lean code, that's a big problem that annoys me a lot, but your way of ranting at me says more about yourself. You sound like "confused old man yells at cloud" rather than "wise old man".