Moving the kernel to modern C [LWN.net]

Moving the kernel to modern C

Posted Mar 3, 2022 22:20 UTC (Thu) by nybble41 (subscriber, #55106)
In reply to: Moving the kernel to modern C by marcH
Parent article: Moving the kernel to modern C

"Operator overloading" is no different than any other overloading. Operators are just functions with symbols for names rather than alphanumeric strings. If you don't have an issue with `add(a, b)` where `add` can be overloaded then there isn't much rational basis for objecting to `a + b` where `+` can be overloaded.

> But once again the problem is the same: where is the compiler / linter flag that limits operator overloading to only "sensible" use cases? Where do you even draw that line?

Sensible languages (such as Haskell) only allow overloading though the implementation of an interface (typeclass), which narrows the problem down: if you want to override `+` in Haskell then you need to provide an instance of `Num` for that data type, which implies that you need to implement the other numerical operations like `*`, `negate`, `fromInteger`, etc. which make up the minimal complete definition. If you try to define `+` at global scope outside of a `Num` instance without explicitly suppressing `Prelude.+` you'll get an error due to the conflicting names.

Typeclasses, in turn, generally come with "laws" which specify how their members should relate to each other, in addition to declaring types for each member which instances must share. (For historical reasons `Num` does not have any official typeclass laws itself, but there are certain basic expectations regarding associativity, commutivity, distributivity, and other properties[0] which amount to informal laws.) The laws, unlike the types, are not automatically enforced, but unlawful instances are strongly discouraged and it is usually simple to write property-based tests to verify that they are met. The key is that the instances of an interface are related by more than just a common name. Implementing a typeclass serves as a declaration of intent that the instance will follow the typeclass laws and generally behave as expected for an instance of that typeclass.

It helps that Haskell allows almost any sequence of symbols as an operator name, so there is no pressure to abuse the left-shift operator for stream output. For example. You can also use any function name as an infix operator by putting it in backquotes—you can even define precedence and fixity for named functions used with infix syntax, just as you can for symbolic operators.

Rust uses essentially the same model, with traits in place of typeclasses, though the set of numerical traits in Rust is a bit more nuanced than Haskell's catch-all `Num` typeclass and the syntax unfortunately doesn't allow for custom operators.

[0] https://hackage.haskell.org/package/base-4.16.0.0/docs/GH...

to post comments

Moving the kernel to modern C

Posted Mar 4, 2022 8:12 UTC (Fri) by wtarreau (subscriber, #51152) [Link] (5 responses)

> If you don't have an issue with `add(a, b)` where `add` can be overloaded then there isn't much rational basis for objecting to `a + b` where `+` can be overloaded.

No, that's precisely the opposite. If you don't have issue with "add(a, b)", the please by all means use that and leave the operators to their original meaning so that the vast majority of the characters in the code you are reading do what you were always taught they do. Nobody imagines that when you're doign your errands and see "3 + 1 offered", this "+" means "perform a database access and do some special operation to return a different value" nor "concatenate them and say 31". No, you imagine an addition, with all the simplicity that comes with it, but within some technical constraints imposed by computers (e.g. domain limitations causing wrapping, saturation or overflows). When I read "a + b", I hear "a plus b" and nothing else. I'm not hearing "a, the first argument of a function using a symbol looking like the plus I know, and a second argument b, now let's check if such a function exists otherwise I'll assume it's in fact a regular plus".

It is important to be able to read code the most naturally possible. It's a matter of efficiency and reliability. And most exploited security flaws in software are found by careful code review and could be spotted by their developers if the code was not constantly cheating on them doing nasty tricks that do not ressemble what it seems to do. That's the same reason why many developers perfer to use upper case for macros. It's a signal that you should look it up and that it might evaluate your arguments more than once, for example.

Moving the kernel to modern C

Posted Mar 4, 2022 9:38 UTC (Fri) by mpr22 (subscriber, #60784) [Link] (1 responses)

I do have an issue with add(a, b) anywhere that isn't the definition of a type's implementation of the Arithmetic interface, though.

When I'm trying to understand the calculations a piece of code is doing on data entities for which arithmetic is a "natural" concept, but which are not Sacred Primitive Types of the implementation language, wading through the resulting vast piles of "x = add_foo(a, multiply_foo(b, c))" etc is the kind of chore that degrades my attention span.

Moving the kernel to modern C

Posted Mar 4, 2022 13:04 UTC (Fri) by wtarreau (subscriber, #51152) [Link]

It's also more difficult to read for me, but way less than if I cannot trust any of the most elementary operators anymore.

A good example are the mmx/sse/avx* API with all those complicated functions that map 1-to-1 to the underlying instructions. It's particularly hard to read, but it would be even worse if operators were abused to perform some of them. And there are still much less operators than possible functions anyway, so being a bit more explicit doesn't hurt.

Moving the kernel to modern C

Posted Mar 4, 2022 15:59 UTC (Fri) by dvdeug (guest, #10998) [Link] (1 responses)

> Nobody imagines that when you're doign your errands and see "3 + 1 offered", this "+" means "perform a database access

In all the programming languages under discussion, 3 + 1 returns 4. In mathematics, a + b is an arbitrary function, almost always associative and commutative; 3 + 1 can equal 0, in Z mod 4. I've certainly seen + used as a concatenation operator in real life, like in rebuses.

> if the code was not constantly cheating on them doing nasty tricks that do not ressemble what it seems to do. That's the same reason why many developers perfer to use upper case for macros. It's a signal that you should look it up and that it might evaluate your arguments more than once, for example.

Non-hygenic macros are insane. Had you said

> That's exactly why I despise [C's macro system]. You cannot trust anymore what you're reading.

I wouldn't have disagreed. So why the difference? Why can developers be trusted with macros and not operator overloading?

Moving the kernel to modern C

Posted Mar 4, 2022 17:02 UTC (Fri) by marcH (subscriber, #57642) [Link]

> > That's the same reason why many developers perfer to use upper case for macros. It's a signal that you should look it up and that it might evaluate your arguments more than once, for example.

> Why can developers be trusted with macros and not operator overloading?

They can't with either. No one likes the pre-processor. It's always seen as a necessary evil to work around C's limitations. Macros are subject to especially high review scrutiny and new ones can be introduced only if they solve a very generic problem and only when they are used all across the board (which ensures a lot of test coverage).

Moving the kernel to modern C

Posted Mar 4, 2022 17:06 UTC (Fri) by marcH (subscriber, #57642) [Link]

> And most exploited security flaws in software are found by careful code review and could be spotted by their developers if the code was not constantly cheating on them doing nasty tricks that do not ressemble what it seems to do.

I wonder where you got that from. Sure there's Spectre and alike but most security flaws I ever looked at (admittedly not that many) were all "mundane" out of bounds accesses, uninitialized use, use after free, integer overflows, etc. All the usual and mundate memory corruption features of C which according to Microsoft and Google account for 70% of all security issues in C/C++ projects (I don't know what the other 30% are)

Moving the kernel to modern C

Posted Mar 4, 2022 14:37 UTC (Fri) by foom (subscriber, #14868) [Link] (5 responses)

> Operator overloading" is no different than any other overloading.

It's different in a really key way in most languages (not Haskell): operators have a more convenient infix syntax, but there's a limited number of them which you can use, and that causes people to prefer to overload them for inappropriate operations.

Let's pretend c++ had no infix operators whatsoever (overloaded or not). So instead of `1 << 2` you'd say `1.left_shift(2)`. When creating an output stream type nobody would _even consider_ having programmers spell "write to output" as `cout.left_shift("hello world")`. That's just ridiculous on it's face! Of course you'd use a more appropriate name.

Yet, because of the limited operator vocabulary that is available to work with, there's a great temptation to use operator overloads for these nonsensical operations. An absolutely irresistible temptation, I'd say. And that's why operator overloading unfortunate in practice -- even though it shouldn't be in theory.

For example: C++20 just repeated this mistake, introducing an overload of bitwise_or `|` for (effectively) function composition in the new ranges library. With the weak rationale that `|` means pipe in Unix shell which is kinda the same thing so it "makes sense". But it doesn't. That's not what the operator | is supposed to mean in c++.

Moving the kernel to modern C

Posted Mar 4, 2022 16:07 UTC (Fri) by dvdeug (guest, #10998) [Link] (4 responses)

> operators have a more convenient infix syntax, but there's a limited number of them which you can use, and that causes people to prefer to overload them for inappropriate operations.

Good point in general. Scala mitigates that by letting methods with one argument be used infix (in version 2, in Scala 3 you have to declare it infix), and allowing pretty much arbitrary operator/method names (for better and worse, :^$*+ is a valid method name that can be used infix in Scala.)

I'd point out your C++ examples are self-inflicted; unlike programmers, the C++ designers could have added operators to the system.

Moving the kernel to modern C

Posted Mar 4, 2022 16:44 UTC (Fri) by atnot (guest, #124910) [Link]

> I'd point out your C++ examples are self-inflicted; unlike programmers, the C++ designers could have added operators to the system.

This is a really long tradition in C++ land. For example, afaict the only reason C++ attempts to use operators for streams in the first place is because the language itself was not rich enough to be able to express generic, type-safe string formatting well. It just so happened that the existing function overloading and a convenient left-associative operator let you cobble together a hacky workaround which then became standard.

Meanwhile nearly every other language decided to go and do what needed to be done to enable type safe string formatting instead. Including C++, which accidentally eventually enabled it to be written anyway, giving rise to {fmt} and then std::format.

Then they went and did the same thing again with iterators, sfinae, variant/visit/optional, etc.

I think the moral of iostreams is less that operator overloading is a bad idea, but that people will do absolutely horrible things if a language is not interested in finding ways to adequately address their needs.

Moving the kernel to modern C

Posted Mar 4, 2022 21:29 UTC (Fri) by foom (subscriber, #14868) [Link]

> I'd point out your C++ examples are self-inflicted

Indeed, the C++ standard library designers often (but not always) act as if they have no ability to influence the core language design. And they may be correct, to some degree -- the language and library changes are done by different working groups within the standards committee, so there's going to be a greater organizational friction to get your change in, if you need to modify both.

Moving the kernel to modern C

Posted Mar 5, 2022 17:13 UTC (Sat) by wtarreau (subscriber, #51152) [Link] (1 responses)

> I'd point out your C++ examples are self-inflicted; unlike programmers, the C++ designers could have added operators to the system.

That was my point. Having a set of "free to use" operators that are never defined by default and are always type-specific would be perfect because they're sufficient to ring a bell when you read that code. But as the previous commenter said, using explicit names instead of left-shifting string still remains quite better. After all in some languages (BASIC for example) we were not shocked by reading "OR", "AND" or "XOR" as operators between two numbers. I'd be fine with a "CAT" operator to concatenate two strings, and remove the ambiguity that you have in certain languages like JS where a=b+1 is ambiguous when b="1" where you don't know if you'll get string "11" or integer 2.

Moving the kernel to modern C

Posted Mar 5, 2022 17:45 UTC (Sat) by dvdeug (guest, #10998) [Link]

> After all in some languages (BASIC for example) we were not shocked by reading "OR", "AND" or "XOR" as operators between two numbers. ... remove the ambiguity that you have in certain languages like JS where a=b+1 is ambiguous when b="1" where you don't know if you'll get string "11" or integer 2.

Does OR do a bitwise OR or a boolean OR? I don't see any saving in ambiguity there. Likewise for the left shift; for all the fuss over it, in practice I've never seen it be the least bit ambiguous.

As for JavaScript, as I said, I was thinking of statically typed languages, and if you're programming JavaScript, you should know the answer to that. (It's "11".) But it is a little more confusing than it would be in other languages, since "1" / 1 implicitly converts "1" to a number, which is problematic, since lossy conversions should generally be avoided, especially when you're shoving a round peg into a square hole. The problem is not with operator overloading, so much as it's with lossy conversions. "String" / 1 should be caught at compile time, not run time.