LWN.net Logo

undefined behaviour

undefined behaviour

Posted Jul 17, 2009 19:18 UTC (Fri) by ehabkost (guest, #46058)
In reply to: undefined behaviour by bluebirch
Parent article: Linux 2.6.30 exploit posted

The point is that gcc itself is relying on undefined behavior, not the program. If dereferencing a NULL pointer is undefined, gcc must NOT assume that the program will crash anyway and the check for NULL can be optimized out.


(Log in to post comments)

undefined behaviour

Posted Jul 17, 2009 19:38 UTC (Fri) by foom (subscriber, #14868) [Link]

Well, that's a wrong point. When the C standard says something is undefined, that means the *compiler* can do whatever it wants. GCC would be within its rights to automatically exec NetHack whenever you dereference a null pointer. :) But instead it chooses to assume that the program will crash, and optimizes the rest of the program accordingly. Which is also a perfectly valid option. "Undefined" is for the benefit of the compiler, not a constraint upon the compiler.

Actually no, the idea is slightly different...

Posted Jul 18, 2009 0:57 UTC (Sat) by khim (subscriber, #9252) [Link]

GCC would be within its rights to automatically exec NetHack whenever you dereference a null pointer. :) But instead it chooses to assume that the program will crash, and optimizes the rest of the program accordingly.

No, no, no. Nothing of the sort. Idea is different and it's somewhat simpler:
1. Behavior is undefined so program can do anything it wants. It can destroy the world for god's sake!
2. Program which can destroy the world is pretty useless so obviously people will not write such program.
3. Ergo program is withing defined behavior. Somehow. Compiler does not need to guarantee that - it's programmer's responsibility.
4. This means some other part of program checks the pointer for NULL (even if compiler has no idea which one).
5. And that means the next check is redundant and can be removed.

That's why it's so hard to undertand for the outsider the discussion which goes in cyrcles when GCC developers talk with normal users:
A. This result is totally ridiculous - fix it!
B. This is undefined behavior - fix your program. WONTFIX.
A. What do you mean "undefined behavior"? It introduces security bugs.
B. This is undefined behavior - fix your program. WONTFIX.
A. Argh. This is all just stupid: how can you even imagine such behavior? .
B. This is UNDEFINED behavior - fix your program. WONTFIX.
Ad infinitum...

GCC developer really don't care what happens to the program with undefined behavior. Not one jot. What happens - will happens. ICE, crash, whatever. The programmer must ensue his (or her) program does not contain undefined constructs - then and only then it's time to complain.

Note: not all behaviors come from C standard. Some come from other standards, some come from descussions from in mailing lists (for example if you go with C standard it becomes impossible to write multithreaded programs so there are some additiona guarantees invented by GCC developers). But if you agree that something is "undefined behavior" then the resolution WONTFIX comes automatically.

Actually no, the idea is slightly different...

Posted Jul 18, 2009 7:03 UTC (Sat) by ABCD (subscriber, #53650) [Link]

I agree with most of your post, except that I think the GCC devs do care if you get an ICE: in that case, there should have been either an informative error message, or some kind of output. As I understand it, in an ideal world (where GCC doesn't have any bugs, per the GCC devs' definition of a GCC bug) an ICE is never the proper response to any input, no matter how undefined.

Actually no, the idea is slightly different...

Posted Jul 18, 2009 10:58 UTC (Sat) by nix (subscriber, #2304) [Link]

Undefined behaviour only and precisely means 'behaviour which is not
defined by the language standard'. It is a matter of QoI, tastefulness and
portability matter whether the GCC devs choose to define this behaviour.
There are many things that ISO C considers undefined that GCC has defined:
we call them language extensions. But if GCC also doesn't define what
happens, well, it's not tested, anything goes.

(Undefined behaviour isn't a magic word. It means *exactly what it says*.)

Actually no, the idea is slightly different...

Posted Jul 19, 2009 0:12 UTC (Sun) by xilun (subscriber, #50638) [Link]

The compiler does not introduce security bugs. The program simply contains bugs, that triggers undefined behaviors, and undefined behaviors can very often be exploited. This is as simple as this.

Do you run all of your programs in production continuously under Valgrind? Because the way you (incorrectly) intrepret the spirit of the standard, I think you should, for your security. Memory is cheap and CPU are fast anyway.

Actually no, the idea is slightly different...

Posted Jul 19, 2009 0:36 UTC (Sun) by dlang (✭ supporter ✭, #313) [Link]

what I would consider reasonable 'undefined' behavior for a compiler to do when a null pointer is used would be to put whatever garbage that it wants in the resulting variable. making _use_ of the resulting data is where you would run into grief (because the data is essentially random). another reasonable thing to do would be to do whatever you would do if the pointer was pointing at an address that didn't exist in the system.

deciding to overwrite the hard drive, run nethack, etc may technically qualifies as undefined, but is defiantly not reasonable.

deciding to 'optimize away' a check immediately afterwards that checks if the pointer is null (prior to the resulting variable being used) is not reasonable.

Actually no, the idea is slightly different...

Posted Jul 19, 2009 1:22 UTC (Sun) by xilun (subscriber, #50638) [Link]

I'm NOT calling for the compiler to intentionally generate malicious code (that would be stupid, even if still conforming). I'm saying that it does NOT have to generate extra checks by default, nor refrain from optimising out useless ones, because this would not be in the spirit of C.

"deciding to 'optimize away' a check immediately afterwards that checks if the pointer is null (prior to the resulting variable being used) is not reasonable"

In the name of what?

You could abritrarily disallow a lot of optimisations with such edict.

Everybody that actually have read the C standard know that it perfectly allow such optimisation. See 6.3.2.3. See 6.5.3.2 (note)

Refraining from making optimisation just because it could make buggy program buggier is not reasonable. Just fix the buggy program. Or in some limited cases, explicitely use the flag that the GCC maintainers kindly provide you, so that even if your program is not strictly conforming, it remains conforming given this particular compiler and compilation option.

undefined behaviour

Posted Jul 17, 2009 19:43 UTC (Fri) by bluebirch (guest, #58264) [Link]

Gcc isn't assuming the program will crash with (this) undefined behaviour. Gcc is assuming that the program doesn't get into undefined behaviour. And if it does, that is a bug in the program.

The correct behaviour isÂ…, well, undefined. So opening a security hole is no less correct (by definition) than crashing or causing "demons to fly out of your nose".

undefined behaviour

Posted Jul 17, 2009 20:58 UTC (Fri) by stevenb (guest, #11536) [Link]

That is what you say.

When GCC makes that the default behavior, there will be others bashing GCC for not optimizing away an obvious unnecessary null-pointer check.

Whatever GCC does, there will always be folks around here and everywhere else who disagree with it. GCC bashing is just the favorite hobby of the entire FOSS developer community, it seems. Just sad...

undefined behaviour

Posted Jul 18, 2009 9:04 UTC (Sat) by Ross (subscriber, #4065) [Link]

That doesn't quite make sense. First, compilers are able to do whatever they like when a program does something undefined -- so in that sense they certainly rely on the ability to make the program do whatever they like when the behavior is undefined (otherwise the behavior would be specified in the standard and it would no longer be undefined). But where the logic in your statement fails is that it is presumably the program which has a need to stick to defined behavior, to well, act in a useful and predictable manner. Any program relying on some specific thing to happen or not happen when it triggers undefined behavior is going to be unhappy with the results. Even if the code is tested and the programmer can see that the compiler does "the right thing", there is no guarantee it will happen that way every time, or that other compilers (or updates to the same compiler) will produce the same effect.

So your other question is who defines what is defined. The C standard defines what things trigger undefined behavior and it is the obligation of the program to avoid them. It's like a contract between programs and the compiler. In a few places specific compilers may define behavior in some of those cases as extensions, but there is no such extension here. Any program which fails to avoid dereferencing of NULL pointers and cares about the subsequent execution of the program is buggy, and blaming the compiler for the problem is not reasonable.

Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds