Betrayed by a bitfield

Posted Feb 6, 2012 12:41 UTC (Mon) by chrisV (guest, #43417)
Parent article: Betrayed by a bitfield

This is not a gcc bug even in the case of struct fields marked volatile, if the whole struct is not marked volatile.

In a user-space program all this would be irrelevant. In the case of different threads accessing different 32 bit integers within an aligned 64-bit boundary, POSIX would prohibit gcc making a 64-bit read/write if the program is compiled with the -pthread compiler option and memory corruption would otherwise arise (this is implicit in Base Definitions, General Concepts, section 4.11, Memory Synchronization, of the SUS). In the case of different processes accessing shared memory between processes (about which the C standard says nothing) you are likewise relying on the guarantees of the shared memory implementation, if any.

The kernel is expecting POSIX-like behavior from a C compiler. gcc is not obliged to provide this if that has not been required of it, either by means of the -pthread switch or by gcc being required (once it has been implemented) to use the similar C11 memory model for multi-threaded programs. Those writing kernels have to force the issue in some other way, such as by writing machine-word sized fields in such cases or by writing processor specific assembly language to force the correct synchronized behavior.

Betrayed by a bitfield

Posted Feb 7, 2012 17:12 UTC (Tue) by BenHutchings (subscriber, #37955) [Link] (6 responses)

None of the options you are suggesting makes a slight difference to gcc behaviour in this case. In any case, the C11 memory model is to a large extent a codification of what is already understood in the industry to be necessary to support multithreaded C programs.

Your unhelpful standards-lawyering attitude is exactly what many programmers have come to hate about certain compiler developers.

Betrayed by a bitfield

Posted Feb 7, 2012 19:21 UTC (Tue) by chrisV (guest, #43417) [Link] (3 responses)

Are you telling me that you have actually run a test case compiled with the -pthread option which exhibits this problem? If not, I just don't believe you because I have written shed loads of multi-threaded code using POSIX threads which has never encountered a false sharing problem of this kind.

If you run a test case compiled with the -pthread flag which exhibits this problem then this is an egregious failure on the part of the threading implementation of gcc and you should post a bug. (It is far more serious than the non-standards complying kernel code.)

Betrayed by a bitfield

Posted Feb 7, 2012 19:36 UTC (Tue) by chrisV (guest, #43417) [Link]

And since you refer to "none of the options you are suggesting makes a slight difference", can you also clarify whether you mean you have run a test case demonstrating false sharing even in the case of 64-bit scalars, and/or even in the case of assembly instructions which should have forced synchronization?

Betrayed by a bitfield

Posted Feb 15, 2012 17:01 UTC (Wed) by cbf123 (guest, #74020) [Link] (1 responses)

I just tested with gcc version 4.3.2 (Wind River Linux Sourcery G++ 4.3-85) for powerpc. Building the test app (the volatile version) as "gcc x.c -c -m64 -pthread" resulted in the following code, which clearly shows a 64-bit read/write cycle.

0000000000000000 <.wrong>:
   0:   fb e1 ff f8     std     r31,-8(r1)
   4:   f8 21 ff c1     stdu    r1,-64(r1)
   8:   7c 3f 0b 78     mr      r31,r1
   c:   f8 7f 00 70     std     r3,112(r31)
  10:   e9 3f 00 70     ld      r9,112(r31)
  14:   e8 09 00 08     ld      r0,8(r9)
  18:   39 60 00 01     li      r11,1
  1c:   79 60 f8 2c     rldimi  r0,r11,31,32
  20:   f8 09 00 08     std     r0,8(r9)
  24:   e8 21 00 00     ld      r1,0(r1)
  28:   eb e1 ff f8     ld      r31,-8(r1)
  2c:   4e 80 00 20     blr

Betrayed by a bitfield

Posted Feb 16, 2012 20:21 UTC (Thu) by chrisV (guest, #43417) [Link]

Any chance you could file a bug on this one? It is not so bad if the non-standard use of volatile has this effect (although no doubt still very annoying to the kernel developers), but it seems to me to be something else when the code happens to be POSIX-conforming code and corrupts memory locations.

(Yes I have seen the argument in papers in the early proposals for the C/C++ threading model that "memory location" is ambiguous in Base Definitions, section 4.11, of the SUS, and could refer to a machine's natural word size rather than individual scalars, and so C11/C++11 now explicitly states that "memory location" in its equivalent wording means any scalar or bitfield, but that is a completely perverse construction of the SUS as it makes POSIX threads completely useless as a standard, bringing threading back to individual non-standard ABIs.)

Betrayed by a bitfield

Posted Feb 7, 2012 20:18 UTC (Tue) by chrisV (guest, #43417) [Link] (1 responses)

I want to keep your issues separate and I have asked you under a separate posting about your tests concerned. On the POSIX point, setting out the assembly output of test cases for Itanium, first without -pthread and then with -pthread, and showing the false sharing would be sufficient, as -pthread is supposed to switch on the POSIX memory model in gcc and it will quickly become apparent if it doesn't.

This posting is just to comment on your observations on the C11 memory model. You are quite right in saying it represents accumulated wisdom, because its starting point was the (comparatively under-specified) POSIX memory model for multi-threaded programs. However, if you read the lkml postings in questions, you will see that the kernel community do not want the generalized C11 memory model for the kernel. They want the preclusion of false sharing; they don't want full sequential consistency in all cases because of its efficiency implications.

Betrayed by a bitfield

Posted Feb 17, 2012 16:20 UTC (Fri) by jwakely (subscriber, #60262) [Link]

> as -pthread is supposed to switch on the POSIX memory model in gcc

Says who?

On ia64-linux it appears to be equivalent to just -D_REENTRANT, on x86-linux it also passes -lpthread to the linker, but it has no effect on code-generation.

> they don't want full sequential consistency in all cases because of its efficiency implications.

Which part of the C11 memory model requires sequential consistency?