Moving the kernel to modern C [LWN.net]

Moving the kernel to modern C

Posted Mar 3, 2022 3:22 UTC (Thu) by marcH (subscriber, #57642)
In reply to: Moving the kernel to modern C by dvdeug
Parent article: Moving the kernel to modern C

> Every step in computer history has reduced the programmer's ability to know what code will run

You're mixing up operator overloading with totally unrelated things. Yes of course we have no idea what machine code runs, what micro code underneath that and of course with code-reuse (at last...) we don't know what code runs in libraries. But operator overloading and inheritance are very different: with them you don't know _which source tab in your own editor you should look at_! In other words, you get lost in _your own code_.

That's a huge difference. There is a very simple solution for all the rest: only use compilers and libraries versions that everyone else uses too. Not only it reduces the number of bugs massively, but when you hit one you can just google it. Not knowing which _code source_ runs in your own project is a totally different problem.

> a+b should add two items. If it's not obvious what it's doing, then that's a bug in the library.

I agree there's only so much damage that can be done by overloading '+'. But once again the problem is the same: where is the compiler / linter flag that limits operator overloading to only "sensible" use cases? Where do you even draw that line?

> Java's "everything is a class" is an annoying box, but when OO is useful, it's really useful.

To a hammer, everything looks like a nail..

to post comments

Moving the kernel to modern C

Posted Mar 3, 2022 19:05 UTC (Thu) by dvdeug (guest, #10998) [Link] (9 responses)

> But operator overloading and inheritance are very different: with them you don't know _which source tab in your own editor you should look at_! In other words, you get lost in _your own code_.

It's easy to know where a + b comes from; if you know that a is of type Quaternion, then look in Quaternion."+" or whatever the specific notation is. (If you were running Java, that would be in quaternion.java; in C, of course, even if you know the function name, it could be anywhere, so if you're getting lost in your own code, you might want to look into Java or at least Java-like file naming conventions.) If you don't know what type a is, you have bigger problems than operator overloading.

Inheritance is more complex, yes. Ultimately, if it's your own code, you should know what type is getting passed in and where to look for that code. Whether you're using inheritance or switches, you're going to need to know what the value is to see what code is being run, and if you know what that value is, you can find the code.

> But once again the problem is the same: where is the compiler / linter flag that limits operator overloading to only "sensible" use cases?

You write in a language that has all code go through a general-purpose macro processor before the compiler, one that has long been shown to do evil, evil stuff (the Bourne shell being a notorious example of horrors that can be done by the well-intentioned.) You trust the programmer to be sensible, or (arguably) you run a more locked down language like Java.

> To a hammer, everything looks like a nail..

In response to the statement that sometimes a Phillips head screwdriver is useful? If you don't believe that OO is useful for GUIs, point to alternatives and explain why OO languages are so popular for GUIs. KDE was done in C++ back before libstdc++ existed.

Moving the kernel to modern C

Posted Mar 3, 2022 22:19 UTC (Thu) by Wol (subscriber, #4433) [Link] (3 responses)

> In response to the statement that sometimes a Phillips head screwdriver is useful? If you don't believe that OO is useful for GUIs, point to alternatives and explain why OO languages are so popular for GUIs. KDE was done in C++ back before libstdc++ existed.

Except that this article is not about GUIs. Maybe it should be - why do we keep on having to buy newer, faster computers just to stop the guis slowing everything down? Do we really want to let that bloat into the kernel - pretty much bringing our super-fast new CPUS to a grinding halt ...

Cheers,
Wol

Moving the kernel to modern C

Posted Mar 3, 2022 22:49 UTC (Thu) by dvdeug (guest, #10998) [Link] (2 responses)

This subthread started with a generalized bashing of OO languages, which is why a mention of where OO languages are useful came up.

I'd be interested to see good measurements of why modern GUIs are so much slower; I'm curious if once you compensate for the increase in screen size and the increase in data sizes, if they really are all that much slower.

Moving the kernel to modern C

Posted Mar 3, 2022 23:54 UTC (Thu) by Wol (subscriber, #4433) [Link] (1 responses)

I thought it started with a suggestion that OO languages (C++) be used in the kernel.

In which case, OO-bashing is justified. OO *IS* useful, but in the kernel when a few extra bytes of object code can cost you dearly in page faults etc, those faults are pretty fatal for the concept. THAT was the purpose of the bashing ...

Cheers,
Wol

Moving the kernel to modern C

Posted Mar 4, 2022 9:42 UTC (Fri) by mpr22 (subscriber, #60784) [Link]

The kernel embraced the size overheads of OO back in the 1990s, and C++'s OO model is just as much "pay for what you use" as the one the kernel uses.

If a class has no virtual members, it doesn't need a function table and doesn't have function-pointer dispatch overheads.

If it has some virtual members, only the members that are virtual have to be identified in the function table and called through function pointers.

Moving the kernel to modern C

Posted Mar 4, 2022 6:27 UTC (Fri) by marcH (subscriber, #57642) [Link] (4 responses)

> > But operator overloading and inheritance are very different: with them you don't know _which source tab in your own editor you should look at_! In other words, you get lost in _your own code_.

> Ultimately, if it's your own code, you should know what type is getting passed in and where to look for that code.

I should not have written _your own code_, I meant: you get lost _in the code base that you are working on_ as opposed to some external library, compiler or other abstraction you don't care about.

Most developers spend most of their time reading and debugging code they did not write themselves. Which is _exactly_ why kernel maintainers don't want C++

> If you don't believe that OO is useful for GUIs

I was just rephrasing you.

Moving the kernel to modern C

Posted Mar 4, 2022 14:57 UTC (Fri) by dvdeug (guest, #10998) [Link] (3 responses)

> I should not have written _your own code_, I meant: you get lost _in the code base that you are working on_

And it still stands; if it's the code base that you are working on, you should know what type is getting passed in and where to look for that code. Assuming you're not using Ruby, the type of the variable should be named nearby in the code.

And again, "I can't find which source tab to look at" is weird coming from someone bashing Java. In a.b(...), the code is found in file named after the type of a. In C, the definition of the function may be found in one of the files you #included in this code (or not; you can always declare external functions directly), but that header file name may have no relation to the file where the code is written. Of course, with the handy -D option to the C compiler, who knows what the function is actually named.

Moving the kernel to modern C

Posted Mar 4, 2022 17:28 UTC (Fri) by marcH (subscriber, #57642) [Link] (2 responses)

> Assuming you're not using Ruby, the type of the variable should be named nearby in the code. [...] In a.b(...), the code is found in file named after the type of a.

The type of the variable does not provide vtable resolution. That's done only at run-time. Same problem for explicit .ops function pointers in C. You cannot just "grep" through vtables / .ops because they're resolved at run-time. That's the whole OO proposition.

> Assuming you're not using Ruby, the type of the variable should be named nearby in the code.

Afraid you lost me there.

> In C, the definition of the function may be found in one of the files you #included in this code (or not; you can always declare external functions directly), but that header file name may have no relation to the file where the code is written.

Code indexers can do a surprisingly good job most of the time. For the rest there is always git grep. It works as long as nothing is resolved at run-time.

> Of course, with the handy -D option to the C compiler, who knows what the function is actually named.

I've seen many -D and I've never seen one redefining function names. As already wrote above, no one like the pre-processor and its abuse is very easily and routinely caught at code review. The entire problem with C++ is "where do you draw the line?". That's really not a problem with the pre-processor: it's always very obvious whether you use it or not and I've seen "no pre-processor here" review comments countless times.

Moving the kernel to modern C

Posted Mar 4, 2022 20:22 UTC (Fri) by marcH (subscriber, #57642) [Link]

BTW a great trick to unravel (evil) cpp macros is to deliberately insert a compilation error. gcc then shows the entire stack of macros involved. Much easier than cc -E or something.

Same trick with build systems and many other situations with "too many layers of indirections": deliberately injecting errors is often a great shortcut.

(none of that available at run-time of course)

Moving the kernel to modern C

Posted Mar 5, 2022 2:42 UTC (Sat) by dvdeug (guest, #10998) [Link]

> The entire problem with C++ is "where do you draw the line?".

That's a problem with C++, not operator overloading or object orientation.

> That's really not a problem with the pre-processor: it's always very obvious whether you use it or not and I've seen "no pre-processor here" review comments countless times.

It is quite similar to operator overloading; you can see where the macro or operator is defined, but not necessarily where it is used.

Moving the kernel to modern C

Posted Mar 3, 2022 22:20 UTC (Thu) by nybble41 (subscriber, #55106) [Link] (12 responses)

"Operator overloading" is no different than any other overloading. Operators are just functions with symbols for names rather than alphanumeric strings. If you don't have an issue with `add(a, b)` where `add` can be overloaded then there isn't much rational basis for objecting to `a + b` where `+` can be overloaded.

> But once again the problem is the same: where is the compiler / linter flag that limits operator overloading to only "sensible" use cases? Where do you even draw that line?

Sensible languages (such as Haskell) only allow overloading though the implementation of an interface (typeclass), which narrows the problem down: if you want to override `+` in Haskell then you need to provide an instance of `Num` for that data type, which implies that you need to implement the other numerical operations like `*`, `negate`, `fromInteger`, etc. which make up the minimal complete definition. If you try to define `+` at global scope outside of a `Num` instance without explicitly suppressing `Prelude.+` you'll get an error due to the conflicting names.

Typeclasses, in turn, generally come with "laws" which specify how their members should relate to each other, in addition to declaring types for each member which instances must share. (For historical reasons `Num` does not have any official typeclass laws itself, but there are certain basic expectations regarding associativity, commutivity, distributivity, and other properties[0] which amount to informal laws.) The laws, unlike the types, are not automatically enforced, but unlawful instances are strongly discouraged and it is usually simple to write property-based tests to verify that they are met. The key is that the instances of an interface are related by more than just a common name. Implementing a typeclass serves as a declaration of intent that the instance will follow the typeclass laws and generally behave as expected for an instance of that typeclass.

It helps that Haskell allows almost any sequence of symbols as an operator name, so there is no pressure to abuse the left-shift operator for stream output. For example. You can also use any function name as an infix operator by putting it in backquotes—you can even define precedence and fixity for named functions used with infix syntax, just as you can for symbolic operators.

Rust uses essentially the same model, with traits in place of typeclasses, though the set of numerical traits in Rust is a bit more nuanced than Haskell's catch-all `Num` typeclass and the syntax unfortunately doesn't allow for custom operators.

[0] https://hackage.haskell.org/package/base-4.16.0.0/docs/GH...

Moving the kernel to modern C

Posted Mar 4, 2022 8:12 UTC (Fri) by wtarreau (subscriber, #51152) [Link] (5 responses)

> If you don't have an issue with `add(a, b)` where `add` can be overloaded then there isn't much rational basis for objecting to `a + b` where `+` can be overloaded.

No, that's precisely the opposite. If you don't have issue with "add(a, b)", the please by all means use that and leave the operators to their original meaning so that the vast majority of the characters in the code you are reading do what you were always taught they do. Nobody imagines that when you're doign your errands and see "3 + 1 offered", this "+" means "perform a database access and do some special operation to return a different value" nor "concatenate them and say 31". No, you imagine an addition, with all the simplicity that comes with it, but within some technical constraints imposed by computers (e.g. domain limitations causing wrapping, saturation or overflows). When I read "a + b", I hear "a plus b" and nothing else. I'm not hearing "a, the first argument of a function using a symbol looking like the plus I know, and a second argument b, now let's check if such a function exists otherwise I'll assume it's in fact a regular plus".

It is important to be able to read code the most naturally possible. It's a matter of efficiency and reliability. And most exploited security flaws in software are found by careful code review and could be spotted by their developers if the code was not constantly cheating on them doing nasty tricks that do not ressemble what it seems to do. That's the same reason why many developers perfer to use upper case for macros. It's a signal that you should look it up and that it might evaluate your arguments more than once, for example.

Moving the kernel to modern C

Posted Mar 4, 2022 9:38 UTC (Fri) by mpr22 (subscriber, #60784) [Link] (1 responses)

I do have an issue with add(a, b) anywhere that isn't the definition of a type's implementation of the Arithmetic interface, though.

When I'm trying to understand the calculations a piece of code is doing on data entities for which arithmetic is a "natural" concept, but which are not Sacred Primitive Types of the implementation language, wading through the resulting vast piles of "x = add_foo(a, multiply_foo(b, c))" etc is the kind of chore that degrades my attention span.

Moving the kernel to modern C

Posted Mar 4, 2022 13:04 UTC (Fri) by wtarreau (subscriber, #51152) [Link]

It's also more difficult to read for me, but way less than if I cannot trust any of the most elementary operators anymore.

A good example are the mmx/sse/avx* API with all those complicated functions that map 1-to-1 to the underlying instructions. It's particularly hard to read, but it would be even worse if operators were abused to perform some of them. And there are still much less operators than possible functions anyway, so being a bit more explicit doesn't hurt.

Moving the kernel to modern C

Posted Mar 4, 2022 15:59 UTC (Fri) by dvdeug (guest, #10998) [Link] (1 responses)

> Nobody imagines that when you're doign your errands and see "3 + 1 offered", this "+" means "perform a database access

In all the programming languages under discussion, 3 + 1 returns 4. In mathematics, a + b is an arbitrary function, almost always associative and commutative; 3 + 1 can equal 0, in Z mod 4. I've certainly seen + used as a concatenation operator in real life, like in rebuses.

> if the code was not constantly cheating on them doing nasty tricks that do not ressemble what it seems to do. That's the same reason why many developers perfer to use upper case for macros. It's a signal that you should look it up and that it might evaluate your arguments more than once, for example.

Non-hygenic macros are insane. Had you said

> That's exactly why I despise [C's macro system]. You cannot trust anymore what you're reading.

I wouldn't have disagreed. So why the difference? Why can developers be trusted with macros and not operator overloading?

Moving the kernel to modern C

Posted Mar 4, 2022 17:02 UTC (Fri) by marcH (subscriber, #57642) [Link]

> > That's the same reason why many developers perfer to use upper case for macros. It's a signal that you should look it up and that it might evaluate your arguments more than once, for example.

> Why can developers be trusted with macros and not operator overloading?

They can't with either. No one likes the pre-processor. It's always seen as a necessary evil to work around C's limitations. Macros are subject to especially high review scrutiny and new ones can be introduced only if they solve a very generic problem and only when they are used all across the board (which ensures a lot of test coverage).

Moving the kernel to modern C

Posted Mar 4, 2022 17:06 UTC (Fri) by marcH (subscriber, #57642) [Link]

> And most exploited security flaws in software are found by careful code review and could be spotted by their developers if the code was not constantly cheating on them doing nasty tricks that do not ressemble what it seems to do.

I wonder where you got that from. Sure there's Spectre and alike but most security flaws I ever looked at (admittedly not that many) were all "mundane" out of bounds accesses, uninitialized use, use after free, integer overflows, etc. All the usual and mundate memory corruption features of C which according to Microsoft and Google account for 70% of all security issues in C/C++ projects (I don't know what the other 30% are)

Moving the kernel to modern C

Posted Mar 4, 2022 14:37 UTC (Fri) by foom (subscriber, #14868) [Link] (5 responses)

> Operator overloading" is no different than any other overloading.

It's different in a really key way in most languages (not Haskell): operators have a more convenient infix syntax, but there's a limited number of them which you can use, and that causes people to prefer to overload them for inappropriate operations.

Let's pretend c++ had no infix operators whatsoever (overloaded or not). So instead of `1 << 2` you'd say `1.left_shift(2)`. When creating an output stream type nobody would _even consider_ having programmers spell "write to output" as `cout.left_shift("hello world")`. That's just ridiculous on it's face! Of course you'd use a more appropriate name.

Yet, because of the limited operator vocabulary that is available to work with, there's a great temptation to use operator overloads for these nonsensical operations. An absolutely irresistible temptation, I'd say. And that's why operator overloading unfortunate in practice -- even though it shouldn't be in theory.

For example: C++20 just repeated this mistake, introducing an overload of bitwise_or `|` for (effectively) function composition in the new ranges library. With the weak rationale that `|` means pipe in Unix shell which is kinda the same thing so it "makes sense". But it doesn't. That's not what the operator | is supposed to mean in c++.

Moving the kernel to modern C

Posted Mar 4, 2022 16:07 UTC (Fri) by dvdeug (guest, #10998) [Link] (4 responses)

> operators have a more convenient infix syntax, but there's a limited number of them which you can use, and that causes people to prefer to overload them for inappropriate operations.

Good point in general. Scala mitigates that by letting methods with one argument be used infix (in version 2, in Scala 3 you have to declare it infix), and allowing pretty much arbitrary operator/method names (for better and worse, :^$*+ is a valid method name that can be used infix in Scala.)

I'd point out your C++ examples are self-inflicted; unlike programmers, the C++ designers could have added operators to the system.

Moving the kernel to modern C

Posted Mar 4, 2022 16:44 UTC (Fri) by atnot (guest, #124910) [Link]

> I'd point out your C++ examples are self-inflicted; unlike programmers, the C++ designers could have added operators to the system.

This is a really long tradition in C++ land. For example, afaict the only reason C++ attempts to use operators for streams in the first place is because the language itself was not rich enough to be able to express generic, type-safe string formatting well. It just so happened that the existing function overloading and a convenient left-associative operator let you cobble together a hacky workaround which then became standard.

Meanwhile nearly every other language decided to go and do what needed to be done to enable type safe string formatting instead. Including C++, which accidentally eventually enabled it to be written anyway, giving rise to {fmt} and then std::format.

Then they went and did the same thing again with iterators, sfinae, variant/visit/optional, etc.

I think the moral of iostreams is less that operator overloading is a bad idea, but that people will do absolutely horrible things if a language is not interested in finding ways to adequately address their needs.

Moving the kernel to modern C

Posted Mar 4, 2022 21:29 UTC (Fri) by foom (subscriber, #14868) [Link]

> I'd point out your C++ examples are self-inflicted

Indeed, the C++ standard library designers often (but not always) act as if they have no ability to influence the core language design. And they may be correct, to some degree -- the language and library changes are done by different working groups within the standards committee, so there's going to be a greater organizational friction to get your change in, if you need to modify both.

Moving the kernel to modern C

Posted Mar 5, 2022 17:13 UTC (Sat) by wtarreau (subscriber, #51152) [Link] (1 responses)

> I'd point out your C++ examples are self-inflicted; unlike programmers, the C++ designers could have added operators to the system.

That was my point. Having a set of "free to use" operators that are never defined by default and are always type-specific would be perfect because they're sufficient to ring a bell when you read that code. But as the previous commenter said, using explicit names instead of left-shifting string still remains quite better. After all in some languages (BASIC for example) we were not shocked by reading "OR", "AND" or "XOR" as operators between two numbers. I'd be fine with a "CAT" operator to concatenate two strings, and remove the ambiguity that you have in certain languages like JS where a=b+1 is ambiguous when b="1" where you don't know if you'll get string "11" or integer 2.

Moving the kernel to modern C

Posted Mar 5, 2022 17:45 UTC (Sat) by dvdeug (guest, #10998) [Link]

> After all in some languages (BASIC for example) we were not shocked by reading "OR", "AND" or "XOR" as operators between two numbers. ... remove the ambiguity that you have in certain languages like JS where a=b+1 is ambiguous when b="1" where you don't know if you'll get string "11" or integer 2.

Does OR do a bitwise OR or a boolean OR? I don't see any saving in ambiguity there. Likewise for the left shift; for all the fuss over it, in practice I've never seen it be the least bit ambiguous.

As for JavaScript, as I said, I was thinking of statically typed languages, and if you're programming JavaScript, you should know the answer to that. (It's "11".) But it is a little more confusing than it would be in other languages, since "1" / 1 implicitly converts "1" to a number, which is problematic, since lossy conversions should generally be avoided, especially when you're shoving a round peg into a square hole. The problem is not with operator overloading, so much as it's with lossy conversions. "String" / 1 should be caught at compile time, not run time.

Moving the kernel to modern C

Posted Mar 4, 2022 7:42 UTC (Fri) by wtarreau (subscriber, #51152) [Link] (2 responses)

> But operator overloading and inheritance are very different: with them you don't know _which source tab in your own editor you should look at_! In other words, you get lost in _your own code_.

Exactly. More than a decade ago, one of our project died after the ruby developer who was leaving admitted he was totally unable to tell us which assignments or operations were causing database accesses! Because doing a=b+c could *possibly* perform requests ina database to fetch some values or store the results. It became so deep that the code was a living being of itself that noone could tame anymore, and fixing bugs became totally impossible. Definitely a very bad idea. The only thing it provides is ease of *writing* code, but we must never forget that we write once and read it many times.

Moving the kernel to modern C

Posted Mar 4, 2022 14:30 UTC (Fri) by ianmcc (guest, #88379) [Link]

How is that fixed by spelling it a=add(b,c) ?

Moving the kernel to modern C

Posted Mar 4, 2022 14:36 UTC (Fri) by dvdeug (guest, #10998) [Link]

= or + doing database requests is insane, but so is naming the function f() or add() or shelia(). I was not familiar that any dynamically typed language had operator overloading; in a statically typed language, the types of a, b and c would have made automatic renaming possible. Clarity of naming is important, and I'd argue that there's a conservation of information; the more ambiguous the types, the more explicit the function naming needs to be to compensate.

> The only thing it provides is ease of *writing* code

One of the recent Scheme additions was writing (add (times a c) b) as {a * c + b}, because it actually is easier to read code like that, and

"hi " + name.toString + " and good morning"

is also easier to read than

"hi".append(name.toString).append ("and good morning")