Glibc change exposing bugs

Posted Oct 21, 2013 20:37 UTC (Mon) by nix (subscriber, #2304)
In reply to: Glibc change exposing bugs by jzbiciak
Parent article: Glibc change exposing bugs

Your comment was interesting anyway. This is the relevant guarantee from C89 (C99 and C11 have similar wording):

If two pointers to object or incomplete types compare equal, they point to the same object. If two pointers to functions compare equal, they point to the same function. If two pointers point to the same object or function, they compare equal. If one of the operands is a pointer to an object or incomplete type and the other has type pointer to a qualified or unqualified version of void , the pointer to an object or incomplete type is converted to the type of the other operand.

The problem here is that this does not guarantee that two pointers to the same object always compare equal, but rather that if they compare equal, they are pointers to the same object (and similarly for comparison operators). We can tell if two pointers definitely are pointers within the same object, but if the comparison fails we cannot conclude anything. This is unfortunately the opposite of the guarantee that memmove() needs if it is to transform itself into a memmove() when needed, so (in the absence of a Standard-blessed way to normalize pointers) you are indeed forced to do a double-copy at all times when writing memmove() in Standard C.

Glibc change exposing bugs

Posted Oct 21, 2013 20:49 UTC (Mon) by khim (subscriber, #9252) [Link] (4 responses)

This is unfortunately the opposite of the guarantee that memmove() needs if it is to transform itself into a memmove() when needed, so (in the absence of a Standard-blessed way to normalize pointers) you are indeed forced to do a double-copy at all times when writing memmove() in Standard C.

Note that in real world there are no such guarantee (hint, hint) thus GLibC's memmove sometimes works and sometimes does not work.

Glibc change exposing bugs

Posted Oct 23, 2013 14:23 UTC (Wed) by nix (subscriber, #2304) [Link] (3 responses)

You appear to have read what I said exactly backwards. Of *course* C on Unix conforms to the guarantee that pointers that compare equal will point to the same object! What you can do with mmap() is produce two pointers that do *not* compare equal but which nevertheless point to the same object. This is exactly what torpedoes a fast auto-reducing-to-memcpy() memmove() implementation, since there is no way to efficiently tell if two pointers point into the same aliased region: even modifying the region via one pointer and probing via the other won't work because they could be pointing at different parts of the aliased region rather than e.g. at the start of it (you are not restricted to call memcpy()/memmove() on pointers returned from malloc(): you can copy parts of objects, and the like).

This behaviour is explicitly permitted by the Standard: segmented architectures like MS-DOS were like this decades ago. The guarantee that a == b returns nonzero only when a and b are pointers to the same object holds nonetheless. It's just a less useful guarantee than we might like.

Glibc change exposing bugs

Posted Oct 23, 2013 15:47 UTC (Wed) by khim (subscriber, #9252) [Link] (2 responses)

My point was that real-world GLibC-implemented memmove does not actually work when used on POSIX system. It compares pointers and assumes that if they are different then underlying memory is also different!

Which means, strictly speaking, that memmove in GLibC is not standards-compliant :-)

Glibc change exposing bugs

Posted Oct 23, 2013 16:47 UTC (Wed) by jzbiciak (guest, #5246) [Link]

From my perspective, the two of you are in violent agreement. :-)

Glibc change exposing bugs

Posted Oct 23, 2013 18:09 UTC (Wed) by nix (subscriber, #2304) [Link]

What? Surely not...

... bloody hell, it does. Or many of the assembler versions do anyway. Or, rather, it assumes that distinct addresses cannot alias.

I suppose this is probably safe in practice, because if you *do* use mmap() to set up aliased regions at distinct addresses you are suddenly in hardware-dependent land (due to machines with VIPT caches such as, IIRC, MIPS, not being able to detect such aliasing at the caching level, so you suddenly need to introduce -- necessarily hardware-dependent -- cache flushes and memory barriers) so you have to know what you're doing anyway, and little things like memmove() falling back to memcpy() at unexpected times are things you're supposed to know about.

I hope.