Not logged in
Log in now
Create an account
Subscribe to LWN
LWN.net Weekly Edition for December 5, 2013
Deadline scheduling: coming soon?
LWN.net Weekly Edition for November 27, 2013
ACPI for ARM?
LWN.net Weekly Edition for November 21, 2013
Glibc change exposing bugs
Posted Nov 10, 2010 20:38 UTC (Wed) by neilbrown (subscriber, #359)
So implementing memcpy as memmove - which Linus says in the bugzilla threads is largely what the kernel does - sounds very sensible. memmove is much harder to misuse.
Posted Nov 13, 2010 1:07 UTC (Sat) by rriggs (subscriber, #11598)
Which one do you think your average C programmer will choose?
Which one do you think new programmers are taught to use (in schools that still teach C programming)?
Posted Nov 13, 2010 2:52 UTC (Sat) by neilbrown (subscriber, #359)
Posted Nov 15, 2010 16:43 UTC (Mon) by renox (subscriber, #23785)
And memcpy should also be named as mem_unsafe_copy, but yes if you tell developers to use safe function by default and to optimize only when they can show benchmarks that the optimisation will make a difference, then yes, you'd get probably better software (if a bit slower).
Posted Oct 17, 2013 12:49 UTC (Thu) by jzbiciak (✭ supporter ✭, #5246)
You're calling memmove verbose as compared to memcpy? Even Ken Thompson said if he had it to do over, he'd spell creat() with the final 'e'.
Posted Nov 25, 2010 15:13 UTC (Thu) by Spudd86 (guest, #51683)
The problem is that most apps don't actually need the those bits, so they just needlessly break software like pulseaudio (and also break on bluetooth audio too).
Pulseaudio does use those unemulatable APIs, but it also falls back if they don't work, and it has good reasons to use those APIs (so it can hand over large chunks of audio data, but still be able to decide it wants to change that same data later (if for example something else starts playing audio), this saves you power because pulse won't wake your CPU as much, but it also uses APIs that don't emulate well AND until pulse came along nobody ever tried to do that sort of thing so it broke)
Posted Nov 25, 2010 16:07 UTC (Thu) by foom (subscriber, #14868)
Posted Nov 25, 2010 16:39 UTC (Thu) by Spudd86 (guest, #51683)
Posted Oct 17, 2013 12:42 UTC (Thu) by jzbiciak (✭ supporter ✭, #5246)
One major reason the remaining distinction between memcpy and memmove exists in the standard seems to be this:
To write memmove completely within conformant C, you need a malloc and a double-copy. That's because in that mythical Platonic ideal of a language, you cannot compare two pointers that do not point into the same object, and you are not guaranteed that the arguments to memmove point within the same object. That is, a fully compliant memmove would look something like this:
void *memmove(void *dst, const void *src, size_t len)
char *srcc = (char *)src;
char *dstc = (char *)dst;
char *temp = malloc(len);
/* What if 'malloc' fails? call abort()? Unspecified! */
for (i = 0; i != len; i++)
temp[i] = srcc[i];
for (i = 0; i != len; i++)
dstc[i] = temp[i];
And, on 16-bit segmented computers or other computers lacking flat memory spaces, both of which are rather from a Platonic ideal, comparing two pointers isn't always as straightforward as you might like. So practically, memcpy offers some noticeable performance benefits on those machines.
Yes, I'm aware that the actual language in the standard says 'as if' the source was first copied to a temporary array. But, as I recall, a fully conformant C program has no other option. The 'as-if' clause allows library writers to avoid such shenanigans, without requiring them to do so. So much hair-splitting...
If it weren't for that, you could make the argument that separate memcpy and memmove were historical accidents, and change the C standard at some point to remove the restrictions on memcpy to make them both equivalent. That new memcpy would then adhere to Rusty's Maxim, or at least come much closer. And, from the thread linked above, that's pretty much what BSD did, it sounds like.
As a half step, you could define memcpy as always copying forward, to make "sliding down" safe, but that just seems a little goofy for a number of reasons.
I'm personally with Linus that the glibc breakage seems gratuitous. I'd lean towards making memcpy and memmove equivalent if their performance is largely indistinguishable. Arguing that the software is broken when it worked for year with the old library reminds me of this silly meme. It's the kind of hair-splitting that only a bureaucrat or chapter-verser could love.
Posted Oct 17, 2013 12:44 UTC (Thu) by jzbiciak (✭ supporter ✭, #5246)
...rather far from a Platonic ideal...
Need. More. Coffee.
Posted Oct 18, 2013 13:31 UTC (Fri) by meuh (subscriber, #22042)
If we were on "stackoverflow", you would have earned the "Necromancer" badge ;)
Posted Oct 18, 2013 14:08 UTC (Fri) by jzbiciak (✭ supporter ✭, #5246)
Posted Oct 21, 2013 20:37 UTC (Mon) by nix (subscriber, #2304)
If two pointers to object or incomplete types compare equal, they point to the same object. If two pointers to functions compare equal, they point to the same function. If two pointers point to the same object or function, they compare equal. If one of the operands is a pointer to an object or incomplete type and the other has type pointer to a qualified or unqualified version of void , the pointer to an object or incomplete type is converted to the type of the other operand.
Posted Oct 21, 2013 20:49 UTC (Mon) by khim (subscriber, #9252)
This is unfortunately the opposite of the guarantee that memmove() needs if it is to transform itself into a memmove() when needed, so (in the absence of a Standard-blessed way to normalize pointers) you are indeed forced to do a double-copy at all times when writing memmove() in Standard C.
Note that in real world there are no such guarantee (hint, hint) thus GLibC's memmove sometimes works and sometimes does not work.
Posted Oct 23, 2013 14:23 UTC (Wed) by nix (subscriber, #2304)
This behaviour is explicitly permitted by the Standard: segmented architectures like MS-DOS were like this decades ago. The guarantee that a == b returns nonzero only when a and b are pointers to the same object holds nonetheless. It's just a less useful guarantee than we might like.
Posted Oct 23, 2013 15:47 UTC (Wed) by khim (subscriber, #9252)
My point was that real-world GLibC-implemented memmove does not actually work when used on POSIX system. It compares pointers and assumes that if they are different then underlying memory is also different!
Which means, strictly speaking, that memmove in GLibC is not standards-compliant :-)
Posted Oct 23, 2013 16:47 UTC (Wed) by jzbiciak (✭ supporter ✭, #5246)
Posted Oct 23, 2013 18:09 UTC (Wed) by nix (subscriber, #2304)
... bloody hell, it does. Or many of the assembler versions do anyway. Or, rather, it assumes that distinct addresses cannot alias.
I suppose this is probably safe in practice, because if you *do* use mmap() to set up aliased regions at distinct addresses you are suddenly in hardware-dependent land (due to machines with VIPT caches such as, IIRC, MIPS, not being able to detect such aliasing at the caching level, so you suddenly need to introduce -- necessarily hardware-dependent -- cache flushes and memory barriers) so you have to know what you're doing anyway, and little things like memmove() falling back to memcpy() at unexpected times are things you're supposed to know about.
Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds