Betrayed by a bitfield

Posted Feb 6, 2012 18:38 UTC (Mon) by chrisV (guest, #43417)
In reply to: Betrayed by a bitfield by daglwn
Parent article: Betrayed by a bitfield

Yours cannot be the correct reading. The basic C11 requirement for multi-threaded programs is in §5.1.2.4/4: "Two expression evaluations conflict if one of them modifies a memory location and the other one reads or modifies the same memory location." In this normative provision, "memory location" cannot possibly mean anything other than the internal representation of a built-in type (char, int, or other integer, or a pointer), otherwise writing multi-threaded programs would be impossible. The POSIX standard for pthreads imposes a similar requirement on the implementation.

Note that marking a struct or a field of a struct volatile is irrelevant to all this. This (§5.1.2.4) is a part of the C specification relating to access to an object of a given built-in type by multiple threads within a single process. Volatile is concerned only with asynchronous access to a given memory location by a single thread in a single process (for example, by an interrupt). Of course, if volatile were to work at all in the usage in the kernel, some prior cache synchronization would be required so approximating to what is wanted here. But even so, the compiler is not obliged to provide the behavior demanded by the kernel code in question.

Betrayed by a bitfield

Posted Feb 6, 2012 20:02 UTC (Mon) by daglwn (guest, #65432) [Link] (10 responses)

Thanks for explaining things.

> But even so, the compiler is not obliged to provide the behavior demanded
> by the kernel code in question.

Again, not being familiar with C11, what does the memory model give that can help multithreaded programming, then? How does §5.1.2.4/4 work in the case where the machine access granularity is larger than some primitive types and a RMW must happen?

Betrayed by a bitfield

Posted Feb 6, 2012 21:04 UTC (Mon) by chrisV (guest, #43417) [Link] (9 responses)

"Again, not being familiar with C11, what does the memory model give that can help multithreaded programming, then? How does §5.1.2.4/4 work in the case where the machine access granularity is larger than some primitive types and a RMW must happen?"

Discarding volatile, which is irrelevant, it requires the faulty behavior observed in the kernel not to occur. On 64 bit architectures it can do this by various pessimizations. It could use slower 32 bit accesses where available. If that is not available it may have to pad so that any 32-bit scalar occupies 64 bits of space, where necessary to conform to the standard.

None of this is new. gcc's pthreads implementations have been doing this for years. It might be instructive to look at the assembly output of the same test cases compiled with the -pthread switch.

Betrayed by a bitfield

Posted Feb 6, 2012 21:11 UTC (Mon) by dlang (guest, #313) [Link] (2 responses)

32 bit operations are not slower on all 64 bit architectures, including specifically the ia64 architecture where this problem was discovered.

Betrayed by a bitfield

Posted Feb 6, 2012 21:32 UTC (Mon) by chrisV (guest, #43417) [Link] (1 responses)

Well that should make it easier to get the gcc developers to deal with the ia64 at least, although I would take a little persuading that moving memory from 64 bit registers into non-64-bit aligned memory locations didn't cause some slow down in code operating in 64 bit mode. Do you have any citations for that?

Betrayed by a bitfield

Posted Feb 6, 2012 21:39 UTC (Mon) by dlang (guest, #313) [Link]

Linus and several others in the kernel thread on the subject have said this.

Betrayed by a bitfield

Posted Feb 7, 2012 0:43 UTC (Tue) by daglwn (guest, #65432) [Link] (5 responses)

> If that is not available it may have to pad so that any 32-bit scalar
> occupies 64 bits of space, where necessary to conform to the standard.

Or pad 8 bits to 32, etc. This is what I mean by breaking ABI compatibility. If interfaces include struct values this could be a real nightmare.

Personally, I think this requirement is ridiculous absent some special attribute to enforce it. If there was a qualifier on a type to indicate "shared," _a_la_ UPC, that would be one thing. But to have this requirement for every single data item in the program is a very bad idea.

Betrayed by a bitfield

Posted Feb 7, 2012 8:43 UTC (Tue) by khim (subscriber, #9252) [Link] (4 responses)

Since C11 includes _Alignas and you can specify you own alignment it's not a disaster, albeit it is inconvenience.

Perfectly working C++ program can already be broken by recompilation in C++11 mode, so it's not the first time upgrade broke things.

Of course is only possible if original program violated specs and worked by accident (if you can show me genuinely different behavior in standards-compliant program it'll be interesting to know, too, but so far all examples I've seen contained subtle violations of one form or another).

Betrayed by a bitfield

Posted Feb 7, 2012 18:35 UTC (Tue) by daglwn (guest, #65432) [Link] (3 responses)

Ah, cool, didn't know about _Alignas. I'm a codegen guy so I'm not playing around in the C frontend very often. I only look at the standard when I really have to. :)

There's still an ABI problem if the compiler always has to align members to uphold the requirement.

> Perfectly working C++ program can already be broken by recompilation in
> C++11 mode

But as we all know, C++ is not C. :) I don't see this as a problem.

Betrayed by a bitfield

Posted Feb 7, 2012 20:53 UTC (Tue) by khim (subscriber, #9252) [Link] (2 responses)

> Perfectly working C++ program can already be broken by recompilation in
> C++11 mode

But as we all know, C++ is not C. :) I don't see this as a problem.

C++ is quite explicitly not C, but C++11 pretends that it's still C++.

Betrayed by a bitfield

Posted Feb 7, 2012 23:29 UTC (Tue) by daglwn (guest, #65432) [Link] (1 responses)

Ah, I read "C11."

Yes, you are correct, but I don't think there's an ABI issue. An ABI issue is much harder to deal with than a semantic change.

With an ABI issue you've got to recompile the world (your project, libraries it links to, etc.) to get a working application. With a semantic change you only recompile the bits that had to recoded to account for the change.

Betrayed by a bitfield

Posted Feb 8, 2012 8:54 UTC (Wed) by khim (subscriber, #9252) [Link]

YMMV, as usual.

We recompile the world anyway, so ABI change is less of a problem, but the fact that just a recompilation does not fix the issue and you need to do a lot of investigations is a problem.