Moving the kernel to modern C [LWN.net]

Moving the kernel to modern C

Posted Feb 24, 2022 18:47 UTC (Thu) by adobriyan (subscriber, #30858)
Parent article: Moving the kernel to modern C

Yay!

Don't forget this one too:

# warn about C99 declaration after statement
KBUILD_CFLAGS += -Wdeclaration-after-statement

Moving the kernel to modern C

Posted Feb 24, 2022 20:30 UTC (Thu) by marcH (subscriber, #57642) [Link] (45 responses)

Yes please, finally! Combined declarations and initializations like every other programming language. More 'const' and fewer "this variable 'may' be used uninitialized" guessing/silliness. No more reverse Christmas trees.

In even more advanced languages 'const' is the default but let's not get carried away; too much maths that could scare hardware engineers emotionally attached to their registers.

Moving the kernel to modern C

Posted Feb 25, 2022 9:29 UTC (Fri) by geert (subscriber, #98403) [Link] (6 responses)

> In even more advanced languages 'const' is the default

"const" is the default for /var/iables?!?

Moving the kernel to modern C

Posted Feb 25, 2022 9:37 UTC (Fri) by ncm (guest, #165) [Link]

Let us not confuse the name with the thing named.

Moving the kernel to modern C

Posted Feb 26, 2022 0:14 UTC (Sat) by camhusmj38 (subscriber, #99234) [Link] (1 responses)

Const is the default for named values. Mutability is opt in in Rust. C and C++ have const as the opt in.
Scala has different words for mutable and non-mutable values (var and val respectively.)

Constant v Immutable

Posted Feb 28, 2022 19:10 UTC (Mon) by tialaramex (subscriber, #21167) [Link]

Rust distinguishes constants from variables which simply can't be mutated. By default you get a variable but it can't be mutated. You can declare constants instead with "const" or, if you annotate your variable with "mut" you allow the variable to be mutated subsequently.

let cannot_change = some::expression(with_variables_if_you, want);

const COMPILE_TIME: u32 = my::WayToGet::a_constant_value(PERHAPS_FROM_OTHER_CONSTANTS);

let mut count = 0; /* We will change this, presumably when counting stuff */

At compile time the constants must be well, constant, (over time the amount of labour the compiler is willing to undertake to determine what that constant *is* has increased, as it has in C++ to a much greater extent) but the ordinary immutable value is not known at compile time, yet, it is immutable (of course if it's inside the scope of a loop, it will be conjured into existence, perhaps with a new value, each time the loop runs)

This means Rust programs naturally do Kate Gregory's first step from maintaining a C++ codebase, marking everything immutable and then only marking as mutable the stuff that actually changes, so now the maintenance programmer has some idea what's actually going on.

If you're familiar with C, Rust's const is like a type-safe improvement on #define and Rust's default immutable variables are more like C's const. You may notice that the types were elided from my variable examples but not the constant, Rust insists on being explicitly told the type of constants but it will infer types for many variables from how they're defined or used.

Moving the kernel to modern C

Posted Feb 28, 2022 3:36 UTC (Mon) by marcH (subscriber, #57642) [Link] (2 responses)

> > In even more advanced languages 'const' is the default
>
> "const" is the default for /var/iables?!?

Sorry, I should used the standard name "non-modifiable lvalue" /s

More seriously, you're highlighting a serious "const" problem in the programming languages and culture and especially in C. It's very confusing to call "constant" something in a local scope that does not change after initialization but that is _different_ everytime the including function is called. So yes, an "immutable variable" is the unfortunate and confusing name used to make that difference.

Rust does make a formal difference between 1) constant, 2) immutable and 3) mutable variables:https://doc.rust-lang.org/book/ch03-01-variables-and-muta...

Consider this example:

some_function()
{
...
z = f(g(x1) + h(x2)) / (j(x3) - k(x4)) - l(x5) + ... ;
...
}

There is a simple reason why most people don't write code like this and why they break it down into multiple steps: readability. Not just to avoid very long lines but to simply give a good NAME to carefully chosen checkpoints in the middle:

some_function()
{
... some code, including of course some statements and not just declarations ...

const meaningful_name1 = f(g(x1) + h(x2) ;
const meaningful_name2 = j(x3) - k(x4);
etc.
...
}

Funny enough, I've sometimes seen this lack of intermediate "variables" being abused by people new to functional languages ("look Ma, no variables!). It's especially tempting when you have a ternary operator more readable than " cond ? A : B". I digress.

It's sad that many programming languages seem to care so little about the difference between read-only and read/write when mutability is in fact the most critical programming concept for both correctness (unintended side effects) and concurrency:
https://doc.rust-lang.org/book/ch16-00-concurrency.html

Every documentation about concurrency, locking, RCU and what not uses the words READ and WRITE every other line. Yet C does not care and calls everything "a variable". Can you see a problem / gap here? C, the low level language supposedly in charge of managing memory accessed concurrently by devices and multicores got a formal memory model in... 2011! After Java and I believe by basically borrowing the C++ one. RCU and locking experts aside, the vast majority of kernel developers underestimates or even ignores the ridicule of that C-tuation.

And of course the more read-only variables you have, the less likely you are to modify them by mistake. Can't hurt when coding in _the_ language of memory corruptions.

C has been influenced too much by the hardware engineering perspective where a variable is a memory location / register and not enough by the more "mathematical" view where a variable is just a name given to the result of some computation. Allowing declarations after statements is a baby step but into the right direction. All grown-up languages have already taken this step.

Moving the kernel to modern C

Posted Feb 28, 2022 8:17 UTC (Mon) by geert (subscriber, #98403) [Link]

Thanks, I do like the mathematical view!
And allowing declarations after statements is a requirement for making intermediate results of non-trivial processing const.

Moving the kernel to modern C

Posted Mar 1, 2022 0:09 UTC (Tue) by marcH (subscriber, #57642) [Link]

Forgot the classic reference: https://queue.acm.org/detail.cfm?id=3212479

C Is Not a Low-level Language
Your computer is not a fast PDP-11.
David Chisnall

> Caches are large, but their size isn't the only reason for their complexity. The cache coherency protocol is one of the hardest parts of a modern CPU to make both fast and correct. Most of the complexity involved comes from supporting a language in which data is expected to be both shared and mutable as a matter of course. Consider in contrast an Erlang-style abstract machine, where every object is either thread-local or immutable

Etc.

Moving the kernel to modern C

Posted Feb 25, 2022 9:39 UTC (Fri) by wtarreau (subscriber, #51152) [Link] (37 responses)

> Combined declarations and initializations like every other programming language.

Please no! That's the most horrible thing I hate in modern C.

Normally when reviewing code and looking for a variable, you just need to glance at
the top of each upper level opening brace and nothing more. With those insane
declarations after statement, you work like in bash: you have to read *ALL* lines
above where you are, hoping you didn't miss the right one. This serves absolutely
no purpose, and only has for effect to complicate code reviews and ease introduction
of new bugs.

Moving the kernel to modern C

Posted Feb 25, 2022 11:14 UTC (Fri) by Wol (subscriber, #4433) [Link] (14 responses)

You're confusing "use" and "initialise".

Don't allow mixing USE and declaration. But DO allow the *compiler* to set the initial value.

Cheers,
Wol

Moving the kernel to modern C

Posted Feb 25, 2022 15:13 UTC (Fri) by wtarreau (subscriber, #51152) [Link] (13 responses)

I'm not sure we're speaking about the same thing. I'm speaking about not making this monstrosity possible, where I'd say "good luck" for figuring the type of "i" depending on the line you're reading, and its bounds:

#include <stdio.h>
#include <unistd.h>

int blah(long x, int j)
{
long i = x ? x : -1;
int k = i;

for (int i = 1; i < j; i++) {
k += i * 2;
char i = (k & 1) ? 'O' : 'E';
int pid = getpid();
printf("i=%d j=%d pid=%d\n", i, j, pid);
}
return k;
}

PS: sorry for the formatting, I didn't find how to make a code block.

Moving the kernel to modern C

Posted Feb 25, 2022 16:16 UTC (Fri) by farnz (subscriber, #17727) [Link] (1 responses)

Making a code block on LWN needs two tags in HTML formatting: <pre> to indicate that formatting matters, and <tt> to indicate that you want monospaced fonts. Below is <pre><tt> followed by your code (with indentation added by my brain), followed by </tt></pre> - I've also had to escape special characters with HTML escapes (but it's a simple matter to write code to do this for you).


#include <stdio.h>
#include <unistd.h>

int blah(long x, int j)
{
    long i = x ? x : -1;
    int k = i;

    for (int i = 1; i < j; i++) {
        k += i * 2;
        char i = (k & 1) ? 'O' : 'E';
        int pid = getpid();
        printf("i=%d j=%d pid=%d\n", i, j, pid);
    }
    return k;
}

Moving the kernel to modern C

Posted Feb 28, 2022 10:15 UTC (Mon) by wtarreau (subscriber, #51152) [Link]

> I've also had to escape special characters with HTML escapes

Thanks. That was the thing that made me think I was heading the wrong direction and that possibly there was something simpler in order to just paste a piece of code.

Moving the kernel to modern C

Posted Feb 25, 2022 16:51 UTC (Fri) by Wol (subscriber, #4433) [Link] (1 responses)

I thought that was allowed in ancient C ...

Unless you mean actually declaring inside the "if" statement ... but I thought declaring after a { was permitted anywhere. I dunno, it's ages since I've programmed C in anger.

But it would be nice to say you can ONLY declare after a {, but that includes things like "int i = 1". You shouldn't be able to do things like "int i; i=1; int j; j=2;", though.

Cheers,
Wol

Moving the kernel to modern C

Posted Feb 25, 2022 18:44 UTC (Fri) by nybble41 (subscriber, #55106) [Link]

In C89 and GNU89 declarations must occur before statements within each block. C89 additional requires initializers to be compiler-time constants. However, it's not as if this equivalent GNU89 code is any easier to follow:

#include <stdio.h>
#include <unistd.h>

int blah(long x, int j)
{
    long i = x ? x : -1;
    int k = i;
    {
        int i;
        for (i = 1; i < j; i++) {
            k += i * 2;
            {
                char i = (k & 1) ? 'O' : 'E';
                {
                    int pid = getpid();
                    printf("i=%d j=%d pid=%d\n", i, j, pid);
                }
            }
        }
    }
    return k;
}

The real lessons here are "use meaningful names" and "avoid shadowing".

Moving the kernel to modern C

Posted Feb 26, 2022 0:21 UTC (Sat) by camhusmj38 (subscriber, #99234) [Link]

This is called variable shadowing and is possible in C89 as well. It can be very ugly which is why it is discouraged although a good compiler should warn on shadowing.
A good practice in modern languages is to combine declaration and initialisation so that you reduce the chance of accessing an uninitialised value. It also encourages locality in the code which makes it easier to comprehend.

Moving the kernel to modern C

Posted Feb 28, 2022 11:48 UTC (Mon) by ianmcc (guest, #88379) [Link] (7 responses)

main.cpp:11:6: error: redeclaration of ‘char i’
   11 | char i = (k & 1) ? 'O' : 'E';
      |      ^
main.cpp:9:10: note: ‘int i’ previously declared here
    9 | for (int i = 1; i < j; i++) {
      |          ^

Moving the kernel to modern C

Posted Feb 28, 2022 13:56 UTC (Mon) by jem (subscriber, #24231) [Link] (6 responses)

You seem to have a faulty C compiler, or you didn't copy the code correctly. The curly bracket at the end of line 9 starts a new block, and it's perfectly legal to declare a new 'i' variable inside that block.

Moving the kernel to modern C

Posted Feb 28, 2022 17:06 UTC (Mon) by ianmcc (guest, #88379) [Link] (4 responses)

That might be valid C (although I don't know why, but it doesn't give any errors in an online C compiler). It isn't valid C++. The scope of the control variable declared in the for loop is the loop itself, so you can't declare another variable with the same name in the same scope.

for (int i = ..)
{
int i = 2; // not valid C++. There is already a variable 'i' declared in this scope
}

Moving the kernel to modern C

Posted Feb 28, 2022 20:10 UTC (Mon) by nybble41 (subscriber, #55106) [Link] (3 responses)

> The scope of the control variable declared in the for loop is the loop itself, so you can't declare another variable with the same name in the same scope.

What you say agrees with the C++ standard, but it makes me wonder why the standard authors appear to have been competing to come up with the most Byzantine special cases and exceptions they could think of to integrate into the standard rather than taking the simplest and least surprising route. Syntactically, the body of the for loop is a single statement which may be a compound statement. That part is the same as C. The braces are *not* part of the syntax for the loop. If the scope of the control variable were in fact the loop itself, and the body were treated the same as any other statement, then the compound statement would be an independent scope nested *within* that for-loop scope, and declarations within the compound statement would shadow any declarations scoped to the for loop (as they do in C). Instead the standard pierces the abstraction and treats compound statements in a for loop body differently than compound statements located elsewhere. There is no logic to this that I can see, just a bald statement that "If a name introduced in an init-statement or for-range-declaration is redeclared in the outermost block of the substatement, the program is ill-formed."

Moving the kernel to modern C

Posted Mar 1, 2022 16:27 UTC (Tue) by ianmcc (guest, #88379) [Link] (2 responses)

In C++ the declaration and the body of the loop are the same scope. In C, initializer in the for loop establishes its own scope, so there are actually two scopes created with a C for loop. This was unintended behavior in C, and a defect report was raised about it, but it seems it wasn't seen as important enough. http://www.open-std.org/jtc1/sc22/wg14/www/docs/n2257.htm...

After all, who would write such code anyway? Its a strange thing to take issue with.

Moving the kernel to modern C

Posted Mar 1, 2022 21:58 UTC (Tue) by nybble41 (subscriber, #55106) [Link] (1 responses)

> In C++ the declaration and the body of the loop are the same scope.

for (init-statement condition[opt] ; expression) statement

The body of the for loop is just *statement*. If the declaration were in that scope it wouldn't survive from one iteration of the loop to the next or be visible in *condition* or *expression*. Declarations in *init-statement* are scoped over the entire for loop, not just the body.

Normally a compound statement within *statement* would introduce its own separate block scope *below* the level of *statement*, but in C++ the lines are blurred between the body of the for loop and the *inside* of the compound statement. In other words, I would expect this to be a redeclaration error, because `char i` and `int i` are declared in the same scope (note that all the examples in the standard are of this form):

for (int i = 0; i < N; ++i)
    char i = 7;

but not this, because `char i` is declared in the new *nested* scope created by the compound statement and not directly in the body of the for loop:

for (int i = 0; i < N; ++i) {
    char i = 7;
}

Contrast this with the following code which the standard (C++20 draft) claims is "equivalent" to the second example "except that names declared in the init-statement are in the same declarative region as those declared in the condition, and except that a continue in statement (not enclosed in another iteration statement) will execute expression before re-evaluating condition":

{
    int i = 0;  /* init-statement */
    while (i < N  /* condition */) {
        { char i = 7; }  // statement
        ++i;
    }
}

In the "equivalent" while loop version there is clearly no redeclaration error—the `char i` declaration is within not just one but two levels of compound statements under the while loop and the scope where `int i` was declared.

> After all, who would write such code anyway? Its a strange thing to take issue with.

Whether you would write that by hand or not, it's an unnecessary (and IMHO completely pointless) complication which moreover breaks compatibility with C. Redeclaration conflicts could appear as a result of macro expansion or other code generation, not just in hand-written code.

Moving the kernel to modern C

Posted Mar 2, 2022 8:56 UTC (Wed) by ianmcc (guest, #88379) [Link]

You've got the history the wrong way around. The behaviour of C++ here hasn't changed since it was first standardized in 1998. At that time, C didn't allow a declaration in a for statement. C99 borrowed the wording from the C++ Annotated Reference Manual, without realizing that the wording had been updated during the C++ standardization process. So C introduced an incompatibility with C++, not the other way around. The C standards committee documents are very clear that this was accidental, not intentional.

The bottom line is that C++ will flag an error in some instances of very dubious code that is most likely a bug anyway (i.e. declaring a variable that shadows the loop control variable) where C99 would allow it. None of the standards committee see it as something worth the bother of fixing. If you really did intend to introduce a shadow declaration, the simple fix is to enclose it in another compound statement.

Moving the kernel to modern C

Posted Feb 28, 2022 17:14 UTC (Mon) by ianmcc (guest, #88379) [Link]

See also http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1865.htm

Moving the kernel to modern C

Posted Feb 25, 2022 11:28 UTC (Fri) by jem (subscriber, #24231) [Link] (4 responses)

You can add this to the list of things you hate about Rust, too. In Rust you can even do this:

fn main() {
    let i = 0;
    println!("{}", i);    // Prints 0.
    
    let i = i+1;
    println!("{}", i);    // Prints 1.
}

Note that the i variables are immutable ("const") and there are two of them. The second let introduces a new variable which is initialized with the value i+1, where i refers to the first variable.

Moving the kernel to modern C

Posted Feb 28, 2022 8:09 UTC (Mon) by geert (subscriber, #98403) [Link] (3 responses)

So is this really any safer in the end?
I still cannot rely on the constant i being constant over the full range of the block, as anyone can have inserted one or more redefinitions in the middle of the block.

Moving the kernel to modern C

Posted Feb 28, 2022 9:38 UTC (Mon) by marcH (subscriber, #57642) [Link] (1 responses)

I believe every compiler for pretty much every language can detect shadowing and warn about it. C included.

Moving the kernel to modern C

Posted Feb 28, 2022 10:09 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link]

Go is a notable exception.

Moving the kernel to modern C

Posted Feb 28, 2022 11:26 UTC (Mon) by taladar (subscriber, #68407) [Link]

It is quite idiomatic in Rust to check a Result<T, E> value and reuse the same name for the content of type T after checking.

The alternative isn't really any better, coming up with extra names for what is logically the same value.

Moving the kernel to modern C

Posted Feb 25, 2022 19:30 UTC (Fri) by ballombe (subscriber, #9523) [Link] (14 responses)

> Please no! That's the most horrible thing I hate in modern C.

I am glad to see I am not alone.

Usually, when it is used, it is a sign the function is too large and should be split, which would resolve the scoping issue.
Of course since C17 still does not support gnu89 nested functions, sometime splitting the function require passing an inordinate amount of parameters.

Moving the kernel to modern C

Posted Feb 25, 2022 19:53 UTC (Fri) by marcH (subscriber, #57642) [Link] (9 responses)

> > Please no! That's the most horrible thing I hate in modern C.

You meant: in _any_ vaguely modern language. What rock have you been living under?

> Usually, when it is used, it is a sign the function is too large and should be split, which would resolve the scoping issue.

Exactly. If you can't find a variable declaration then the function is simply too long.

Of course with https://en.wikipedia.org/wiki/Type_inference (1958) you don't even need to declarations _at all_ but again, let's not get carried away and scare ancient species...

Moving the kernel to modern C

Posted Feb 25, 2022 20:53 UTC (Fri) by mpr22 (subscriber, #60784) [Link] (8 responses)

If I can't find the variable declaration, I'm using generic or weakly language-aware editing tools instead of strongly language-aware editing tools.

Moving the kernel to modern C

Posted Feb 26, 2022 14:07 UTC (Sat) by Wol (subscriber, #4433) [Link] (7 responses)

Are you telling me strongly language-aware editing tools can find non-existing declarations?

Okay, I find the lack of declarations ON OCCASION a complete pain, but don't blame the tool if the language doesn't require declarations. There's plenty of languages like that out there ...

(Which is why I tend to use Option Explicit in VB, or -DCLVAR in FORTRAN, etc etc.)

Cheers,
Wol

Moving the kernel to modern C

Posted Feb 26, 2022 14:55 UTC (Sat) by mpr22 (subscriber, #60784) [Link]

> There's plenty of languages like that out there ...

There are, but in the small handful that I use (or have used in the past), I have never written a program where losing track of where I first initialized a variable would be a serious concern.

Moving the kernel to modern C

Posted Feb 26, 2022 15:19 UTC (Sat) by nix (subscriber, #2304) [Link]

> Are you telling me strongly language-aware editing tools can find non-existing declarations?

Well, no, but they can tell you what types the compiler has inferred for those variables (assuming the program is currently syntactically valid enough to do so, which in my experience is usually true if you're doing maintenance rather than writing a new function: if you're writing a new function, I hope you already know what types the variables you're using in your new code are. :) )

Moving the kernel to modern C

Posted Feb 26, 2022 22:10 UTC (Sat) by camhusmj38 (subscriber, #99234) [Link] (4 responses)

Type Inference is not the same as no declaration. Type Inference just means types don’t have to be explicitly stated where the compiler can infer it from the expression used to initialise the variable.

Moving the kernel to modern C

Posted Feb 27, 2022 16:04 UTC (Sun) by marcH (subscriber, #57642) [Link] (3 responses)

Why would be the purpose of a declaration without an explicit type?

How would a typeless declaration help the people in this thread who complain about not finding declarations? If not the type, what else are they looking for?

Moving the kernel to modern C

Posted Feb 27, 2022 18:55 UTC (Sun) by camhusmj38 (subscriber, #99234) [Link] (1 responses)

The declaration has a type - it’s inferred from the initialiser. It’s really only appropriate with local variables.

Moving the kernel to modern C

Posted Feb 27, 2022 20:49 UTC (Sun) by marcH (subscriber, #57642) [Link]

I see nothing factually wrong in your last two comments above but I really don't understand why they're posted as replies to mine. I'm afraid you're misunderstanding my points and questions.

Moving the kernel to modern C

Posted Feb 27, 2022 19:06 UTC (Sun) by mathstuf (subscriber, #69389) [Link]

In Rust, it can move the lifetime of a variable around. So if I have some code that needs to live for the outer scope, but is only "known" in some nested place, I can use `let somename;` on a block in the right place, initialize it where the data is available, and continue on. I've used this before to keep a string alive long enough for a context where it may have been either allocated (and therefore need a place to hang its destructor) or came from some other string that lived longer than the function anyways.

Moving the kernel to modern C

Posted Feb 25, 2022 20:31 UTC (Fri) by abatters (✭ supporter ✭, #6932) [Link] (3 responses)

> gnu89 nested functions

Which require your entire program to have an executable stack.

Moving the kernel to modern C

Posted Feb 25, 2022 20:55 UTC (Fri) by pbonzini (subscriber, #60935) [Link] (2 responses)

They only do if they are used as function pointers.

Moving the kernel to modern C

Posted Feb 25, 2022 22:12 UTC (Fri) by ncm (guest, #165) [Link] (1 responses)

Of course in C++ and Rust you have function literals. In C++,


  auto f = [](auto a, auto b) { return a + b; };
  assert(f(3, 4) == 7);
  assert(f(3.25, 3.75) == 7.0);
  assert(f("3"s, "4"s) == "34");

Rust lambdas might be less versatile.

Moving the kernel to modern C

Posted Feb 26, 2022 0:18 UTC (Sat) by camhusmj38 (subscriber, #99234) [Link]

No, Rust Lambdas generally work the same as C++ lambdas - subject to the usual Rust rules about lifetime.

Moving the kernel to modern C

Posted Feb 25, 2022 23:25 UTC (Fri) by NYKevin (subscriber, #129325) [Link]

As a vim user, I just use the asterisk and hash keys to find the declaration. Not sure what your editor supports, but if it can't do that, you might want to consider switching to a different editor (not necessarily vim, of course, whatever works for you).

Moving the kernel to modern C

Posted Feb 26, 2022 10:40 UTC (Sat) by niner (guest, #26151) [Link]

I have fixed far too many bugs caused by people having to restructure perfectly readable code to appease the C89 gods. Some of those bugs I had introduced myself. On the other hand I have yet to come across a single bug in our code that was clearly caused by any confusion supposedly allowed by having variables declared where they are actually going to be used. Indeed the latter can make things even clearer, because it allows for more usage of const as in:

{
    foo();
    void * const x = must_be_called_after_foo();
    ...
}