User: Password:
|
|
Subscribe / Log in / New account

Betrayed by a bitfield

Betrayed by a bitfield

Posted Feb 4, 2012 17:56 UTC (Sat) by daglwn (guest, #65432)
In reply to: Betrayed by a bitfield by nix
Parent article: Betrayed by a bitfield

> This includes implementations of data member assignment that overwrite
> adjacent members in separate memory locations.

That would seem to be the "out" that gets around the problem of the target machine not having an appropriately-sized load/store instruction to avoid false sharing.

I've not studied the C11 standard yet. Is that the intent of this statement? Otherwise C11 structs with volatile members could be ABI-incompatible with source-equivalent C99 structs due to forced alignment to avoid placing volatile members next to other members in a single load/store access unit.

In other words, what happens if I have a volatile char next to a non-volatile char and my target only has 32-bit loads and stores?


(Log in to post comments)

Betrayed by a bitfield

Posted Feb 4, 2012 21:27 UTC (Sat) by nix (subscriber, #2304) [Link]

Er, that footnote says that those are *precluded*, not permitted (though of course as a footnote it is not normative).

I'm not sure how you can implement these requirements without either ABI changes or an efficiency reduction (on platforms where sub-maximum-sized writes are inefficient). We might have to eat the efficiency loss: we surely cannot fix this by aligning every member of every structure to cacheline boundaries or something like that, that's ridiculous.

(But I am a neophyte in this area. Someone, anyone, who knows more than me, please chip in!)

Betrayed by a bitfield

Posted Feb 6, 2012 18:06 UTC (Mon) by daglwn (guest, #65432) [Link]

> Er, that footnote says that those are *precluded*, not permitted (though
> of course as a footnote it is not normative).

I think it's an "out" because the footnote only covers data in "separate memory locations." Two chars that don't cross an alignment boundary would be in the same "memory location" in some sense. They're in the same atomically-readable space.

But I don't know if that's the intent. If not, I think C11 has a serious issue. It will require ABI changes and that's not going to make people happy.

Betrayed by a bitfield

Posted Feb 6, 2012 18:38 UTC (Mon) by chrisV (subscriber, #43417) [Link]

Yours cannot be the correct reading. The basic C11 requirement for multi-threaded programs is in §5.1.2.4/4: "Two expression evaluations conflict if one of them modifies a memory location and the other one reads or modifies the same memory location." In this normative provision, "memory location" cannot possibly mean anything other than the internal representation of a built-in type (char, int, or other integer, or a pointer), otherwise writing multi-threaded programs would be impossible. The POSIX standard for pthreads imposes a similar requirement on the implementation.

Note that marking a struct or a field of a struct volatile is irrelevant to all this. This (§5.1.2.4) is a part of the C specification relating to access to an object of a given built-in type by multiple threads within a single process. Volatile is concerned only with asynchronous access to a given memory location by a single thread in a single process (for example, by an interrupt). Of course, if volatile were to work at all in the usage in the kernel, some prior cache synchronization would be required so approximating to what is wanted here. But even so, the compiler is not obliged to provide the behavior demanded by the kernel code in question.

Betrayed by a bitfield

Posted Feb 6, 2012 20:02 UTC (Mon) by daglwn (guest, #65432) [Link]

Thanks for explaining things.

> But even so, the compiler is not obliged to provide the behavior demanded
> by the kernel code in question.

Again, not being familiar with C11, what does the memory model give that can help multithreaded programming, then? How does §5.1.2.4/4 work in the case where the machine access granularity is larger than some primitive types and a RMW must happen?

Betrayed by a bitfield

Posted Feb 6, 2012 21:04 UTC (Mon) by chrisV (subscriber, #43417) [Link]

"Again, not being familiar with C11, what does the memory model give that can help multithreaded programming, then? How does §5.1.2.4/4 work in the case where the machine access granularity is larger than some primitive types and a RMW must happen?"

Discarding volatile, which is irrelevant, it requires the faulty behavior observed in the kernel not to occur. On 64 bit architectures it can do this by various pessimizations. It could use slower 32 bit accesses where available. If that is not available it may have to pad so that any 32-bit scalar occupies 64 bits of space, where necessary to conform to the standard.

None of this is new. gcc's pthreads implementations have been doing this for years. It might be instructive to look at the assembly output of the same test cases compiled with the -pthread switch.

Betrayed by a bitfield

Posted Feb 6, 2012 21:11 UTC (Mon) by dlang (subscriber, #313) [Link]

32 bit operations are not slower on all 64 bit architectures, including specifically the ia64 architecture where this problem was discovered.

Betrayed by a bitfield

Posted Feb 6, 2012 21:32 UTC (Mon) by chrisV (subscriber, #43417) [Link]

Well that should make it easier to get the gcc developers to deal with the ia64 at least, although I would take a little persuading that moving memory from 64 bit registers into non-64-bit aligned memory locations didn't cause some slow down in code operating in 64 bit mode. Do you have any citations for that?

Betrayed by a bitfield

Posted Feb 6, 2012 21:39 UTC (Mon) by dlang (subscriber, #313) [Link]

Linus and several others in the kernel thread on the subject have said this.

Betrayed by a bitfield

Posted Feb 7, 2012 0:43 UTC (Tue) by daglwn (guest, #65432) [Link]

> If that is not available it may have to pad so that any 32-bit scalar
> occupies 64 bits of space, where necessary to conform to the standard.

Or pad 8 bits to 32, etc. This is what I mean by breaking ABI compatibility. If interfaces include struct values this could be a real nightmare.

Personally, I think this requirement is ridiculous absent some special attribute to enforce it. If there was a qualifier on a type to indicate "shared," _a_la_ UPC, that would be one thing. But to have this requirement for every single data item in the program is a very bad idea.

Betrayed by a bitfield

Posted Feb 7, 2012 8:43 UTC (Tue) by khim (subscriber, #9252) [Link]

Since C11 includes _Alignas and you can specify you own alignment it's not a disaster, albeit it is inconvenience.

Perfectly working C++ program can already be broken by recompilation in C++11 mode, so it's not the first time upgrade broke things.

Of course is only possible if original program violated specs and worked by accident (if you can show me genuinely different behavior in standards-compliant program it'll be interesting to know, too, but so far all examples I've seen contained subtle violations of one form or another).

Betrayed by a bitfield

Posted Feb 7, 2012 18:35 UTC (Tue) by daglwn (guest, #65432) [Link]

Ah, cool, didn't know about _Alignas. I'm a codegen guy so I'm not playing around in the C frontend very often. I only look at the standard when I really have to. :)

There's still an ABI problem if the compiler always has to align members to uphold the requirement.

> Perfectly working C++ program can already be broken by recompilation in
> C++11 mode

But as we all know, C++ is not C. :) I don't see this as a problem.

Betrayed by a bitfield

Posted Feb 7, 2012 20:53 UTC (Tue) by khim (subscriber, #9252) [Link]

> Perfectly working C++ program can already be broken by recompilation in
> C++11 mode

But as we all know, C++ is not C. :) I don't see this as a problem.

C++ is quite explicitly not C, but C++11 pretends that it's still C++.

Betrayed by a bitfield

Posted Feb 7, 2012 23:29 UTC (Tue) by daglwn (guest, #65432) [Link]

Ah, I read "C11."

Yes, you are correct, but I don't think there's an ABI issue. An ABI issue is much harder to deal with than a semantic change.

With an ABI issue you've got to recompile the world (your project, libraries it links to, etc.) to get a working application. With a semantic change you only recompile the bits that had to recoded to account for the change.

Betrayed by a bitfield

Posted Feb 8, 2012 8:54 UTC (Wed) by khim (subscriber, #9252) [Link]

YMMV, as usual.

We recompile the world anyway, so ABI change is less of a problem, but the fact that just a recompilation does not fix the issue and you need to do a lot of investigations is a problem.

Betrayed by a bitfield

Posted Feb 6, 2012 18:40 UTC (Mon) by nix (subscriber, #2304) [Link]

Not quite. s3.14's definition of 'memory location' is
either an object of scalar type, or a maximal sequence of adjacent bit-fields all having nonzero width
Therefore, even a 'long long' variable occupies a single 'memory location' by that defintion. However, a two-byte array of two chars occupies two memory locations. A location is not the same as an address.


Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds