It's different viewpoints... [LWN.net]

C as portable assembler

Posted May 30, 2011 18:56 UTC (Mon) by jrn (subscriber, #64214) [Link]

> Sure, some platforms were perfectly happy with such code. But then if you want low-level non-portable language... asm is always there.

And so is C. :) After all, what language is the Linux kernel written in?

> Or, alternatively, you can actually read specifications and see what the language actually supports.

I don't think the case of signed overflow is one of trial and error versus reading specifications. It seems more like one of folk knowledge versus new optimizations --- old gcc on x86 and many similar platforms would use instructions that wrap around for signed overflow, so when compiling old code that targeted such platforms, it seems wise to use -fwrapv, and when writing new code it seems wise to add assertions to document why you do not expect overflow to occur.

Of course, reading the spec can be a pleasant experience independently from that.

C as portable assembler

Posted May 31, 2011 7:17 UTC (Tue) by khim (subscriber, #9252) [Link]

> Sure, some platforms were perfectly happy with such code. But then if you want low-level non-portable language... asm is always there.

And so is C. :) After all, what language is the Linux kernel written in?

Linux kernel is written in C, quite portable and people fight constantly to fix hardware and software compatibility problems. Note that while GCC improvements are source of a few errors they are dwarfed by number of hardware compatibility errors. Most of the compiler problems happen when people forget to use appropriate constructs defined to make hardware happy: by some reason macroconstructs designed to fight hardware make code sidestep a wide range of undefined C behaviors. Think about it.

I don't think the case of signed overflow is one of trial and error versus reading specifications.

It is, as was explained before. There are other similar cases. For example standard gives you ability to convert pointer to int in some cases, but even then you can not convert int to pointer because on some platforms pointer is not just a number - yet people who don't know better often do that. Will you object if gcc and/or clang will start to miscompile such programs tomorrow?

It seems more like one of folk knowledge versus new optimizations --- old gcc on x86 and many similar platforms would use instructions that wrap around for signed overflow, so when compiling old code that targeted such platforms, it seems wise to use -fwrapv, and when writing new code it seems wise to add assertions to document why you do not expect overflow to occur.

Note that all these new optimizations are perfectly valid for the portable code. Surprisingly enough -fwrapv exist not to make broken programs valid but to make sure Java overflow semantic is implementable in GCC. Sure, you can use it in C, but it does not mean your code is suddenly correct after that.

Of course, reading the spec can be a pleasant experience independently from that.

Actually it's kind of sad that the only guide we have here is the standard... Given how often undefined behavior bites us you'd think we'll have books which explain where and how they can be triggered in "normal" code. Why people accept that i = i++ + ++i; is unsafe and unpredictable code but lots of other cases which trigger undefined behavior are perceived as safe? It's matter of education...

C as portable assembler

Posted May 31, 2011 17:56 UTC (Tue) by anton (guest, #25547) [Link]

Why people accept that i = i++ + ++i; is unsafe and unpredictable code

Who would write "i = i++ + ++i;" anyway?
It is easy to write what you intended here (whatever that was) in a way that's similarly short and fast and generates a similar amount of code.

but lots of other cases which trigger undefined behavior are perceived as safe?

Because they were safe, until the gcc maintainers decided to break them (and the LLVM maintainers follow them like lemmings).

That's the point...

Posted Jun 1, 2011 9:16 UTC (Wed) by khim (subscriber, #9252) [Link]

It is easy to write what you intended here (whatever that was) in a way that's similarly short and fast and generates a similar amount of code.

It's easy to do in other cases, too. You can always use memcpy to copy from float to int. GCC will eliminate memcpy and unneeded variables.

$ cat test.c
#include <string.h>

int convert_float_to_int(float f) {
  int i;
  memcpy(&i, &f, sizeof(float));
  return i;
}
$ gcc -O2 -S test.c
$ cat test.s
        .file "test.c"
        .text
        .p2align 4,,15
.globl convert_float_to_int
        .type convert_float_to_int, @function
convert_float_to_int:
.LFB22:
        .cfi_startproc
        movss %xmm0, -4(%rsp)
        movl -4(%rsp), %eax
        ret
        .cfi_endproc
.LFE22:
        .size convert_float_to_int, .-convert_float_to_int
        .ident "GCC: (Ubuntu 4.4.3-4ubuntu5) 4.4.3"
        .section
    .note.GNU-stack,"",@progbits

Because they were safe, until the gcc maintainers decided to break them (and the LLVM maintainers follow them like lemmings).

They were never completely safe albeit cases where they break were rare. Today it happens more often. This is not the end of the world, but this is what you must know and accept.

C as portable assembler

Posted May 31, 2011 18:05 UTC (Tue) by anton (guest, #25547) [Link]

The funny thing is that new gcc (at least up to 4.4) still generates code for signed addition that wraps around instead of code that traps on overflow. It's as if the aim of the gcc maintainers was to be least helpful to everyone: for the low-level coders, miscompile their code silently; for the specification pedants, avoid giving them ways to detect when they have violated the spec.

C as portable assembler

Posted May 31, 2011 21:00 UTC (Tue) by jrn (subscriber, #64214) [Link]

There is -fwrapv and -ftrapv. If you think one of those should be the default, no doubt there are some other optimization tweaks (-fno-strict-aliasing?) that you also like; so I encourage you to work on an -Oanton switch in either a wrapper for gcc or gcc itself, for the sake of sharing.

I am pretty happy with -O2 for my own needs, since it generates fast code for loops, but I understand that different situations may involve different requirements and would be happy to live in a world with more people helping make the gcc UI more intuitive.

Sorry, but this is just wrong...

Posted May 31, 2011 17:39 UTC (Tue) by anton (guest, #25547) [Link]

Lots of platforms it was flaky because FPU was physically separate.

My code ran fine on systems with physically separate FPU (e.g., MIPS R2000+R2010). Also, why should the separate FPU affect the types foo and bar?

Anyway, if there is a hardware issue that we have to deal with, that's fine with me, and I will deal with it. But a compiler that miscompiles on hardware that's perfectly capable of doing what I intend (as evidenced by the fact that gcc-2.x -O and gcc-4.x -O0 achieve what I intend) is a totally different issue.

But then if you want low-level non-portable language... asm is always there.

I want a relatively portable low-level language, and gcc-2.x -O and (for now) gcc-4.x -O0 provide that, and my code ports nicely to all the hardware I can get my hands on (and to some more that I cannot); that's definitely not the case for asm, and it's not quite the case with gcc-4.x -O. For now we work around the breakage of gcc-4.x, but the resulting code is not as fast and small as it could be; and we did not have to endure such pain with gcc-2.x.

And new gcc releases are the biggest source of problems for my code. New hardware is much easier.

Or, alternatively, you can actually read specifications and see what the language actually supports.

The C standard specification is very weak, and has more holes than content (was it 190 undefined behaviours? Plus implementation-defined behaviours). Supposedly the holes are there to support some exotic platforms (such as ones-complement machines where signed addition traps on overflow). Sure, people who want to port to such platforms will have to avoid some low-level-coding practices, and will have to suffer the pain that you want to inflict on all of us.

But most of us and our code will never encounter such platforms (these are deservedly niche platforms, and many of these niches become smaller and smaller over time) and we can make our code faster and smaller with these practices, at least if we have a language that supports them. The language implemented by gcc-2.x does support these practices. We just need a compiler for this language that targets modern hardware.

So, the ANSI C specification and the language it specifies is pretty useless, because of these holes. Even Chris Lattner admits that "There is No Reliable Way to Determine if a Large Codebase Contains Undefined Behavior". So what you suggest is just impractical. Not just would it mean giving up on low-level code, the compiler could still decide to format the hard disk if it likes to, because most likely the code still contains some undefined behaviour.

It's kind of funny, but real low-level stuff (like OS kernels or portable on-bare-metal programs) usually survive "evil compilers" just fine.

I have seen enough complaints from kernel developers about breakage from new gcc versions, and that despite that fact that Linux is probably one of the few programs (apart from SPEC CPU) that the gcc maintainers care for. The kernel developers do what we do, they try to work around the gcc breakage, but I doubt that that's the situation they wish for.

Well, if you want some different language you can create it...

Posted Jun 1, 2011 9:14 UTC (Wed) by khim (subscriber, #9252) [Link]

Also, why should the separate FPU affect the types foo and bar?

If FPU module is weakly tied to CPU module then you must explicitly synchronize them. Functions like mamcpy did that so they were safe, regular operations didn't. That's why standard only supports one way to do what you want and it's memcpy. Which is eliminated completely by modern compilers like or clang where it's not needed.

My code ran fine on systems with physically separate FPU (e.g., MIPS R2000+R2010).

Well, sure. Not all combinations were flaky. It does not change the fact that the only way to convert float to int safely was, is and will be memcpy (till the next redaction of C standard, at least).

But a compiler that miscompiles on hardware that's perfectly capable of doing what I intend (as evidenced by the fact that gcc-2.x -O and gcc-4.x -O0 achieve what I intend) is a totally different issue.

Why? Compiler just makes your hardware less predictable - but only till the boundaries outlined in the standard. That's what the compiler is supposed to do! These boundaries were chosen to support wide range of architectures and optimizations, but if you want different boundaries - feel free to create new language.

For now we work around the breakage of gcc-4.x, but the resulting code is not as fast and small as it could be; and we did not have to endure such pain with gcc-2.x.

This is possible but it just shows you need some different capabilities from your language. You can propose some extensions and/or optimizations to gcc and clang developers to make your code faster. Complains that your code does not work when it clearly violates the spec will lead you nowhere.

GCC and CLANG developers are not stuck up snobs, they are ready to add new extensions when it's needed (this changes the language spec and makes some previously undefined constructs defined), but they clearly are reluctant to try to guess what your code is doing without your help - if they have no clear guidance they use the spec.

The language implemented by gcc-2.x does support these practices.

Ok, if you like it, then use it.

We just need a compiler for this language that targets modern hardware.

No problem, it's there. It may look like a stupid excuse, but it's not. The only way to support something which exist only as implementation on newer platform is emulation. That's why there are all these PDP-11 emulators, NES emulators, etc.

If you want something different then first you must document your assumptions which don't adhere to the C99 spec and then talk with compiler developers.

So, the ANSI C specification and the language it specifies is pretty useless, because of these holes.

Huh? Where THIS comes from? Compiler does not know what the program actually does, but surely the programmer which wrote it does! S/he can avoid undefined behaviors - even if it's not always simple and/or easy. If the programmer likes to play Russian Roulette with the language - it's his/her choice, but then he should expect to be blown to bits from time to time.

Not just would it mean giving up on low-level code, the compiler could still decide to format the hard disk if it likes to, because most likely the code still contains some undefined behaviour.

Compiler can only do that if you trigger undefined behavior. Most undefined behaviors are pretty easy to spot and avoid but some of them are not. Instead of whining here and asking for the pie in the sky which will never materialize you can offer changes to the specifications which will simplify life of the programmer. Then they may be adopted either as extensions or as new version of C. It's supposed to be revised every 10 years, you know.

The kernel developers do what we do, they try to work around the gcc breakage, but I doubt that that's the situation they wish for.

Sure, but they use new GCC capabilities when they become available too and they accepted that they come as package deal. Note that kernel no longer support GCC 2.95 at all.

There was one time when kernel developers said what you are saying and refused to support newer versions of GCC but this lead them nowehere. Today linux kernel can be compiler with GCC 4.6 just fine.