LWN: Comments on "C considered dangerous"

C considered dangerous

mathstuf — Fri, 21 Sep 2018 18:14:40 +0000

> C++ does still have integer overflow UB, as does Rust

No, Rust defines integer overflow as twos-complement (though in debug builds it will panic). You can use an always-panicking addition however, but that's not the default.

C considered dangerous: VLAs

nix — Thu, 13 Sep 2018 18:51:28 +0000

Good points. VLAs might still be useful if your maximum depends on some other variable, *and* you are confident you have enough space, but frankly by this point either you're in a hot inner loop or you should be using malloc() anyway.

C considered dangerous: VLAs

excors — Thu, 13 Sep 2018 13:06:52 +0000

The lack of error handling means it's only safe to use VLAs (or alloca) if you know a static upper bound on the size of the array, and it's small enough that you're confident it's never going to overflow the stack. But if you know an upper bound, why not just always allocate that much? Overallocating seems unlikely to have any performance impact in typical cases. And if the stack is too small for your constant-sized array, you'll find out immediately and can fix it, rather than the failure being dependent on (possibly-attacker-controlled) variables, so it makes the behaviour more predictable and easier to debug.

C considered dangerous: VLAs

nix — Thu, 13 Sep 2018 11:53:53 +0000

Complaining that VLAs are expensive seems wrong. If you don't need a dynamically-sized array, why are you using a VLA? If you do need one, the only alternative is heap allocation, and that is more expensive and error-prone than stack allocation.

Quite. But of course the problem with both alloca() and VLAs is the same as the problem with automatic variables: error handling. If you run out of stack space, there is nothing you can do: it's game over. In a language without exceptions, there aren't even any edges control flow could propagate down, and no way to get the error out. Meanwhile, malloc() has always been able to return NULL (though, again, since nobody other than sqlite tests these failure cases systematically, it is questionable whether this is terribly useful).

C considered dangerous: VLAs

DHR — Tue, 11 Sep 2018 17:04:46 +0000

I don't really like VLAs. They don't fit well into the C type system. They are not part of C++. They are now only an optional part of C, if I remember correctly (the C Committee seems to regret their addition).

Having said that, they were an attempt to cleanly replace alloca (an abomination). Stack allocation is so much cheaper than heap allocation. But of course there is a risk when stacks must be tightly bounded.

Complaining that VLAs are expensive seems wrong. If you don't need a dynamically-sized array, why are you using a VLA? If you do need one, the only alternative is heap allocation, and that is more expensive and error-prone than stack allocation.

C considered dangerous

DHR — Tue, 11 Sep 2018 16:53:25 +0000

"An interesting side effect that came about while investigating this was a warning he got about unreachable code when he enabled the auto-initialization. There were two variables declared just after a switch (and outside of any case), where they would never be reached."

A declaration does not have to be reached unless it has an initializer (or is a VLA). So these would probably be correct C, and perhaps meaningful.

Any executable code between a switch head and the first case label (or other label) is unreachable and a good compiler should flag it. Adding an auto-initializer carelessly triggers this problem.

If a declaration in the switch body (and not further nested) has a scope covering the rest of the switch body. So all cases.

The unfortunate choice of -Wdeclaration-after-statement means that placing a declaration at the start of a switch body is sometimes a good choice.

A compiler should flag any unused variable.

Undefined behaviour

anselm — Mon, 10 Sep 2018 22:28:22 +0000

The main difference between an arithmetic shift to the right and a logical shift to the right is that the logical shift will make the MSB zero while the arithmetic shift will keep the MSB (the sign of a signed number) whatever it was before the shift. This doesn't make a huge amount of sense when shifting to the left, because the LSB in a signed number isn't special, so processor makers often didn't bother with arithmetic shifts to the left (especially with the simpler late-20th-century-vintage CPUs).

Undefined behaviour

mgb — Mon, 10 Sep 2018 19:17:26 +0000

> You mention SRA and SRL. Therefore you should have a matching SLA and SLL.

How would you have expected the functionality of these hypothetical SLA and SLL to have differed?

Undefined behaviour

Wol — Mon, 10 Sep 2018 18:14:35 +0000

As I heard it, it was undocumented BECAUSE it didn't work.

You mention SRA and SRL. Therefore you should have a matching SLA and SLL. The story as I heard it was that some programmers used that exact logic, worked out what the opcode for SLL should be, and discovered what I called "shift left and increment".

In other words, a perusal of the documentation led to people deducing a "missing" opcode, and when they tried it, it ALMOST did what was expected.

Cheers,
Wol

Undefined behaviour

anselm — Mon, 10 Sep 2018 15:08:46 +0000

I would have thought that the C spec should have said "When behaviours are undefined because different hardware behaves differently, it is down to the compiler writers to provide a definition for their implementation".

The C standard, at least back when I was still programming in C, did in fact make that distinction. When dealing with a C implementation, you distinguish between behaviour that is specified in the standard, “implementation-defined behaviour” that is not specified in the standard but requires a consistent (and documented) approach by the implementation, and “undefined behaviour”, where anything can happen, including unpredictable and inconsistent effects.

Undefined behaviour

jem — Sun, 09 Sep 2018 15:40:43 +0000

The Z-80 did have a working shift left instruction (SLA), in addition to two working shift right instructions (SRA and SRL). The "bug" you are referring to is an undocumented opcode which, by chance, behaved like you describe.

C considered dangerous: function pointers

DHR — Sat, 08 Sep 2018 17:02:23 +0000

"In C, a function pointer can be called even if the type of the pointer does not match the type of the function being called—assembly doesn't care, it just jumps to a location, he said. "

This hasn't been true for a long long time. ANSI/ISO C copied function prototypes from C++ for their first standard. This made parameter types part of the type of a function type (and hence of a pointer to a function type). You could willfully program in K&R style function types but generally not by accident (or use casts).

The one exception: it seems easy to declare a function with no parameters using an empty parameter list. That is a K&R declaration. You are supposed to use "void" as the parameter list in the prototype. Somewhat awkward.

I haven't looked at the slides to see if this serious error is in them.

Undefined behaviour

Wol — Fri, 07 Sep 2018 18:41:02 +0000

> At the extreme, C compilers, when encountering implementation defined behavior, have stopped compiling and ran a game.

Given that C has always been considered a "super-assembly" language, and most of the undefined behaviours were, I believe, down to the fact that the original writers of C knew that different hardware behaves in different ways, I would have thought that the C spec should have said "When behaviours are undefined because different hardware behaves differently, it is down to the compiler writers to provide a definition for their implementation".

Bit like the Z80 scenario where the right shift instruction behaved as expected (integer divide by two) but thanks to a bug the left shift did a "double and increment" operation. If the C language didn't specify the behaviour of a left shift thanks to this, then it should have placed the onus on the compiler writers to specify it, not just leaving the behaviour in doubt.

(And this silicon bug really did exist, and quite a few assembler programs took advantage of it, I believe ... left-shift-and-increment it got known as :-)

Cheers,
Wol

Searching LKML?

nix — Fri, 07 Sep 2018 10:58:56 +0000

This is the downside :( I wish Mailman next/prev links weren't constrained to within a single month/week like that.

Finding the declaration of a variable in Emacs

nix — Fri, 07 Sep 2018 10:57:48 +0000

You can have Semantic / CEDET show you the definition of the symbol under point after a short delay in the echo area via semantic-idle-local-symbol-highlight-mode, also enableable globally for all buffers, and automatically via adding global-semantic-idle-local-symbol-highlight-mode to semantic-default-submodes.

Doing this for non-local symbols requires CEDET to know how to locate things in your project, which usually happens more or less automatically, particularly if you use etags or GNU GLOBAL.

Undefined behaviour

dvdeug — Thu, 06 Sep 2018 22:45:45 +0000

In theory, there is nothing but ignorance. September 7th, 2018 has never happened before, so we don't know what will happen on September 7th, 2018. In practice, we can usually use what has gone on in the past to get a good guess at what will happen in the future.

At the extreme, C compilers, when encountering implementation defined behavior, have stopped compiling and ran a game. (Early versions of GCC did so upon encountering a #pragma in C code.) Current versions of GCC regularly assume that undefined behavior can't happen, meaning that conditional checks can be deleted; branches of "if" statements disappear and "while" loops are compiled into infinite loops. We have heard in practice lots of complaints about C compilers doing unexpected things in the presence of implementation defined behavior.

Just as much to the point, ignorance is information. If we have no idea what's going to happen, then we should generally expect the unexpected; lacking knowledge, our presumptions about what is going to happen are just wrong.

Undefined behaviour

Wol — Thu, 06 Sep 2018 10:17:27 +0000

In other words, the well-known proof "reductio ad absurdam".

Cheers,
Wol

memmove faster than memcpy for small N

quotemstr — Thu, 06 Sep 2018 07:14:22 +0000

No, you write the code. I'm not the one making the absurd claim. A program is never made slower by converting always-taken conditionals into jumps.

memmove faster than memcpy for small N

liw — Thu, 06 Sep 2018 06:55:09 +0000

I think both sides need to provide code and benchmarks to settle this question. Hypotheticals and claims based on mental models only go far. Even arguing based on code is insuffident, given how complex an issue performance analysis can be.

I tried writing a benchmark. GCC optimised it away.

memmove faster than memcpy for small N

jreiser — Thu, 06 Sep 2018 03:42:23 +0000

Code, please. I'll bet a beer that your code will be slower for small N (or incorrect.)

memmove faster than memcpy for small N

quotemstr — Wed, 05 Sep 2018 23:45:29 +0000

That makes no sense. A memcpy could turn those branches into unconditional jumps and avoid any mispredict penalties.

memmove faster than memcpy for small N

jreiser — Wed, 05 Sep 2018 23:03:06 +0000

The key is "for small N". The glibc implementation of memcpy favors speed for non-small N, and has several checks and conditional branches before doing much "useful work". The penalty for a mis-predicted conditional branch is often about 12 cycles. The implementation of memmove has fewer initial checks (mostly for overlap or not), and starts doing "useful work" sooner. For small N, the fewer mis-predicted conditional branches enable faster memmove.

C considered dangerous

rweikusat2 — Wed, 05 Sep 2018 20:12:33 +0000

Well, for the sake of the contrived example, let's assume that "during maintenance",

memcpy_s(buf, BUFLEN, src, len)

is changed to

memcpy_s(src, len, buf, BUFLEN)

now, there's no buffer overflow but likely a very serious memory corruption problem.

There's exactly no reason for this assumption except that's it's another way to break the working code but there wasn't any reason for your assumption, either.

C considered dangerous

pizza — Wed, 05 Sep 2018 10:21:39 +0000

Unless, of course, 'buf' is just a raw pointer -- then sizeof(buf) won't give you what you want.

(Personally, most uses of 'memcpy' in my code involve assembling chunks of data into a buffer. Nearly all of the memcpy()s' destinations are at nonzero offsets of the original buffer, so memcpy_s() really doesn't do me any good...)

C considered dangerous

excors — Wed, 05 Sep 2018 10:12:08 +0000

Just be careful if someone might move that into a separate function like "void do_copy(char buf[BUFLEN], ...) { memcpy_s(buf, sizeof(buf), ...); }", and either ignores compiler warnings or is using an older compiler, since sizeof(buf) is now 8 regardless of BUFLEN. C is fun. Better to use a decent ARRAY_SIZE macro (like the one in the Linux kernel) that fails to compile if the argument is not an array.

C considered dangerous

anton — Wed, 05 Sep 2018 09:23:40 +0000

Yes, that's even better. I was actually thinking about the maintenance programmer changing the len parameter of memcpy (to, say, len+a-b), and forgetting to change the assertion; there are often stale comments, so I would not be surprised about stale assertions (unless the tests cause them to trigger).

C considered dangerous

johill — Wed, 05 Sep 2018 09:08:09 +0000

To be fair, in your example the size of the buffer could also be changed without changing BUFLEN (which is your objection to the assert), so you should write

memcpy_s(buf,sizeof(buf),src,len);

in this case, I guess.

C considered dangerous

anton — Wed, 05 Sep 2018 08:56:06 +0000

Let's take a look at an example:

char buf[BUFLEN];

And we want to copy len bytes from src to buf, and we know that len<=BUFLEN. Doing the copy with

memcpy(buf,src,len);

would be correct, as would be

memcpy_s(buf,BUFLEN,src,len);

Now, during maintenance, there are changes that can cause len>BUFLEN. Now the first variant has a buffer overflow, while the second does not (but, as written, does not report the error). Tracking BUFLEN and len is not the problem here; it's tracking whether the assumption implied in the memcpy-using code is true.

The tangible benefit that one gets with the memcpy_s variant is that we do not get a buffer overflow even if there is a mistake in tracking whether the assumption is true. True, we do not get that benefit if we write the wrong value as the second parameter of memcpy_s, but that's a less likely mistake in this case, and probably in many others.

You can make the assumption explicit by writing

assert(len<BUFLEN);

before the memcpy, but then you have to "track two lengths" again, and in addition, assertion checking can be disabled, or the length in the memcpy can be changed without changing the assertion.

Undefined behaviour

farnz — Wed, 05 Sep 2018 08:45:46 +0000

Exactly, but that sort of reasoning about a standard is common - e.g. "The POSIX standard says that int must be at least 32 bits in size, therefore I can assume that long is 64 bits in size". The point of "nasal demons" is that it is equally correct to say "The POSIX standard says that int must be at least 32 bits in siz, therefore assuming long is 64 bits in size will make demons fly out of your nose" - both are logically invalid statements, but while the first sounds plausible, the second does not.

In other words, it's a way of pointing out that your conclusion does not follow from your antecedents in a comical fashion, and thus that you need to go back and fill in your chain of reasoning.

Undefined behaviour

rweikusat2 — Tue, 04 Sep 2018 21:17:20 +0000

> "Nasal demon" is just a short-cut to saying that "the behaviour of a program is not defined by the standards you are relying upon to > define the progam's meaning, and thus anything could go".

As I already wrote: "It is unknown, hence ..." is logically invalid.

Undefined behaviour

farnz — Tue, 04 Sep 2018 20:38:17 +0000

"Nasal demon" is just a short-cut to saying that "the behaviour of a program is not defined by the standards you are relying upon to define the progam's meaning, and thus anything could go". This isn't just the ISO C standard, but also POSIX, and even implementation defined standards; in the cases you're talking about, "nasal demons" is a distraction, as there are standards that define the behaviour in question, even if ISO C permits a wider variety of standards-compliant behaviour.

Further, "undefined" behaviour in the sense that "nasal demons" refers to is more than just behaviour where the implementation defines it - it's behaviour where the implementation is allowed to refuse to define what it means. E.g. int i = 0; ++i++ += i++ + ++i; is a common example of undefined behaviour - because it breaks the rules on sequence points, the implementation does not have to give it any meaning at all.

Undefined behaviour

rweikusat2 — Tue, 04 Sep 2018 20:08:09 +0000

This seems to be an entirely generic text which isn't related to the one I wrote, especially considering that I gave a specific example of documented gcc behaviour for a construct with undefined behaviour insofar the C standard is concerned.

"Nasal daemon" and other such nonsense is just the appeal to ignorance I wrote about, though: Behaviour which isn't defined by ISO-C is not a defined phenomenon properties like "you don't want to depend on it" can be assigned to for some rational reason. Eg, the only I/O facilities defined by ISO-C are stdio-streams and no behaviour is defined for the case when data written by the stdio of one C implementation is later read via the stdio of a different implemention. It follows that the behaviour of a program compiled with clang is undefined whenever it tries to read data written by a program compiled with gcc. Surely, clang is something nobody would ever want to depend on!

Searching LKML?

peda — Tue, 04 Sep 2018 17:02:08 +0000

While I don't like every aspect of lkml.org, one thing it does is link things together even if time has passed.

E.g. https://lkml.org/lkml/2018/8/30/877 has a link to https://lkml.org/lkml/2018/9/1/41 (and back).

Now, that's only 2 days, but its different months/weeks/whatwever, and that tends to work very badly with archives of the style you prefer, as all threading is broken every so often.

The above two messages in your preferred archive are
http://lkml.iu.edu/hypermail/linux/kernel/1808.3/04944.html
http://lkml.iu.edu/hypermail/linux/kernel/1809.0/00008.html

Please tell me where the thread-next/thread-prev links for these two messages can be found in that archive, because I can't find them, and it really make the whole thing less appealing.

C considered dangerous

ndesaulniers — Tue, 04 Sep 2018 16:27:56 +0000

I've fixed this in: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/...

Clang can compile a working kernel for x86_64 if the asm goto #error is removed; asm-goto is under development in llvm currently.

Clang can compile arm64 without CONFIG_LSE_ATOMICS. -ffixed and friends need to be implemented.

Other architectures aren't on my radar.

Searching LKML?

cortana — Tue, 04 Sep 2018 15:00:59 +0000

Where has this been all my life!

Undefined behaviour

farnz — Tue, 04 Sep 2018 12:35:06 +0000

The problem with defining behaviour in terms of "this specific implementation does this" is that absent some form of promise from the implementers (either because they promise to comply with a standard that does what you want, or because they document the behaviour you want as deliberate), you can't distinguish "this behaviour is a consequence of a compiler bug that will be fixed removing the desired behaviour" from "this is desirable behaviour that the implementers will try to keep" from "the implementers will treat the absence of this behaviour as a bug and fix it".

This is especially true of behaviour that comes from interacting bugs - it may be true that with GCC 6.7.9.1 on an Intel Xeon E3-1285 v3 with a fixed microcode version and C code compiled with -march=haswell -O3 you get a behaviour you want, but if that's not promised and a microcode update breaks things, you're stuffed.

Hence the interest in what is defined in documentation - yes, I can validate a given implementation's behaviour for a given machine if I so desire, but what I'm normally interested in is "if I upgrade my machine and/or compiler, will my code continue to do what I want?". Nasal demons is simply a short cut for "even though you could validate behaviour, because no-one is trying to maintain this behaviour in future, you don't want to depend on it."

Searching LKML?

nix — Tue, 04 Sep 2018 11:41:30 +0000

I have always used http://lkml.iu.edu/hypermail/linux/kernel/ which is a normal readable mailman-style archive rather than one of these horrible modern ones which makes conversations almost impossible to follow. (Seriously, lkml.org throws away branches of the thread tree that you're just about to read so it can show the tree to you in a 5cm box, and then doesn't visibly offer a 'next in thread' button. What on earth were they thinking?!)

Undefined behaviour

excors — Tue, 04 Sep 2018 11:14:44 +0000

Even if the behaviour is not defined by the standard, or by the compiler's documentation, it is implicitly defined by the compiler's implementation. The standard doesn't prohibit nasal demons, but you can look at GCC's code and see there is no possible way they can occur. Or you can take a more scientific approach, and make a hypothesis (aka speculation) and try to find evidence to disprove it, in the context of a specific compiler. None of that tells you definitively what another standards-conforming compiler might do, but in practice people only really care about a small group of compilers and compiler versions, and most them behave very similarly to each other. I don't think it's necessary or helpful to simply give up and say we can't even attempt to understand that behaviour.

(Incidentally, I've found void pointer arithmetic convenient for things like "void create_packet(void *buf, Header *hdr, void *payload, size_t len) { memcpy(buf, hdr, sizeof(Header)); memcpy(buf + sizeof(Header), payload, len); }", since void* seems the appropriate type for buffers that don't correspond to a specific C type, and sometimes you want to refer to byte offsets within those buffers. Explicitly casting to char* every time you do arithmetic is ugly and semantically inappropriate.)

Finding the declaration of a variable in Emacs

cpitrat — Tue, 04 Sep 2018 09:34:44 +0000

You can use exuberant-ctags. By default, it doesn't include local variables but you can have them with --c-kinds=+l.
To produce file for emacs you also need -e.
So something like:
ctags -e --c-kinds=+l .

Then you can jump to local variable declarations using M-.

Undefined behaviour

rweikusat2 — Tue, 04 Sep 2018 09:13:27 +0000

A thought which occurred to me only recently, despite spending years thinking about this: All this harping on on what "undefined behaviour" could or might be is nothing but a logic error commonly known as "appeal to ignorance".

Behaviour the C standard doesn't define is just that: Behaviour the C standard doesn't define, hence, no information about it can be derived from this text. Other applicable documentation might, eg, according to ISO-C, arithmetic on void-pointers has undefined behaviour, yet, there's a longstanding gcc-extension to treat void-pointers like char-pointers in this respect, despite it's entirely unclear what - besides causing silent memory corruption - this is actually supposed to be good for (one can guess that "legacy GNU C" is "just more equal than other C", obviously, but that's just a guess some people might consider probable).

In absence of such other documentation or other information on real world behaviour, there's not only no point in speculating wildy about what "undefined behaviour" could mean, it's also either fallacious or intellecutally dishonest: A claim of the form "We don't know that A is false, hence it could be true" or "We don't know that A is true, hence, it could be false" is a non-sequitur: Nothing follows from ignorance.