User: Password:
Subscribe / Log in / New account

GCC and pointer overflows

GCC and pointer overflows

Posted Apr 16, 2008 22:49 UTC (Wed) by aegl (subscriber, #37581)
Parent article: GCC and pointer overflows

I'm having some dificulty parsing the rationale by which this optimization is allowed.

"in a correct program, pointer addition will not yield a pointer value outside of the same object"

Is gcc deciding that "buffer + len < buffer" makes this an "incorrect" program, so it can generate any random code (in this case no code at all) because the program is "incorrect" and so deserves to die?

Could gcc get more radical and apply the same logic to the first clause of this test? The declaration of "buffer" is in scope and "buffer_end" is statically initialized and never changed (OK ... it would have to be declared "static" for gcc to be really sure about this, so lets pretend that it was). So a test for "buffer + len >= buffer_end" appears to be just as "incorrect" if you believe that pointer addition will not yield a pointer "outside of the same object".

I'm sure the intent of the "same object" rule is to stop programmers from making assumptions about the proximity of different objects. E.g.

int a[5], b[5], *p = a;
p += 5;
/* should not assume that p now points to "b" */

It seems a bit of a stretch to get from this to outlawing a test like "buffer + len < buffer".

(Log in to post comments)

GCC and pointer overflows

Posted Apr 16, 2008 23:20 UTC (Wed) by wahern (subscriber, #37304) [Link]

No, the intent [of the standard] is to support segmented memory architectures. The "one after"
rule is a concession for prototypical looping patterns.

When you loop in reverse you have to be careful, because there's no "one before" rule.
Compilers like TinyCC, in debug mode, will insert instrumentation code which relies on these
rules to check buffer under and overflows (not just accesses, but actual pointer
arithmetic--it can detect invalid pointer values). So it's smart to stick to the ISO standard,
because these rules can cut both ways; that is, they can help not only optimization, but
debugging/security, too.

GCC and pointer overflows

Posted Apr 16, 2008 23:31 UTC (Wed) by wahern (subscriber, #37304) [Link]

I should be more clear. Because there's no "one before" rule, then "buffer + len < buffer"
will always be false in a correct program. A correct program would not attempt to derive an
invalid pointer (let alone access an object through such a pointer).

It's worth noting that "buffer" is the actual object, not merely a pointer. This is an
instance where we can't forget that arrays and pointers in C really are distinct, even though
we can so often treat an array like a pointer. And the compiler in this instance has
sufficient information about the object to make the optimization, whereas in many other
instances it would not be able to do this (and so people get the wrong impression). This is a
good thing.

GCC and pointer overflows

Posted Apr 17, 2008 9:39 UTC (Thu) by kleptog (subscriber, #1183) [Link]

There's a terminology problem here: the program is not incorrect, the programmer has made an
incorrect assumption. The assumption is that (buf + len < buf) will be true if len is very

Besides the fact that the assumption is false if sizeof(*buf) != 1, the GCC team (and other
compilers) point out that this assumption is not warrented by the C spec. Stronger still, the
C spec allows you to *assume* the test is false, no matter the value of len (assuming len is
unsigned btw).

That said, I'd love a way to say:

if( __wraps( buf + len ) )

GCC and pointer overflows

Posted Apr 17, 2008 16:01 UTC (Thu) by wahern (subscriber, #37304) [Link]

if (~sizeof buf < len) {

This only works with unsigned values, and there are probably some caveats with width and
promotion rules (portable, nonetheless).

Also, assuming your environment uses linear addressing, and there's no other funny stuff going
on with pointer bits (like the effectively 16 free bits on AMD64--using 48-bit addressing).

if (~(uintptr_t)buf < len)  {

I believe this should work on Windows and all Unix systems (guaranteed by additional SUSv3
constraints), but I'm not positive.

GCC and pointer overflows

Posted Apr 17, 2008 22:03 UTC (Thu) by jzbiciak (subscriber, #5246) [Link]

Of course, it fails for dynamically allocated and grown buffers since sizeof() can't tell you the length.

Also, you failed to account for element size. The following should work, though, for arrays of static size:

    if (len > (sizeof(buf) / sizeof(buf[0]))

I don't understand why you have the bitwise negation operator in there. Also, len is a length, not a pointer type, so pointer format doesn't matter.

GCC and pointer overflows

Posted Apr 19, 2008 5:51 UTC (Sat) by wahern (subscriber, #37304) [Link]

The question was how to check if arithmetic overflowed/wrapped, not whether an index or length
is valid.

Copyright © 2018, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds