"gold standard"? [LWN.net]

"gold standard"?

Posted Apr 26, 2012 3:25 UTC (Thu) by cpeterso (guest, #305)
Parent article: LFCS 2012: LLVM and Linux

Is GCC really the "gold standard" of optimized code performance? GCC's optimizer used to have a bad reputation compared to Intel's ICC compiler and ARM's ARMCC compiler.

Have Intel or ARM contributed to Clang/LLVM? I've always been surprised that they maintain their own proprietary, closed-source compilers rather than contributing more resources to GCC (or Clang/LLVM).

to post comments

"gold standard"?

Posted Apr 26, 2012 15:48 UTC (Thu) by davide.del.vento (guest, #59196) [Link] (13 responses)

No, gcc is the slowest compiler, AFAIK. Its advantage is being very portable across architectures, which isn't true for Intel's or ARM's.

And what's about the (recently re-opensources) Pathsscale? It is on par with Intel, in my experience. And then there is the PGI.... The compiler's game is a large race and GCC and LLVM are not the only two horses in it.

I believe Intel does its own closed source compiler to show the best it can on its hardware. They don't want that "best" being available on other vendor's hardware.

"gold standard"?

Posted Apr 26, 2012 21:12 UTC (Thu) by robert_s (subscriber, #42402) [Link] (12 responses)

"No, gcc is the slowest compiler, AFAIK."

Believe me, there are plenty of much slower compilers.

As for hardware vendors' compilers, it would be pretty embarrassing for them (not to mention pointless) if they _couldn't_ outperform competitors on their own hardware.

I'm also not so convinced by any "4% off gcc" claims about LLVM. Phoronix (yes, caveats aside) routinely test LLVM releases and it's common for LLVM to be a good 50% behind gcc. A lot of these claims you hear about LLVM's speed often turn out to be "when compared to a 3 year old gcc".

Not that I'm not an LLVM fan though. It's just important to be realistic.

"gold standard"?

Posted Apr 26, 2012 21:40 UTC (Thu) by daglwn (guest, #65432) [Link] (3 responses)

> Believe me, there are plenty of much slower compilers.

Yes there are but I believe the statement is true if one considers "regularly used" compilers. gcc's generated code is notoriously slow. Still, it will always win on some codes simply because no one compiler can completely dominate every other.

gcc's great strength is in its portability and cross-compile capabilities, which no other compiler can touch and won't any time soon.

> As for hardware vendors' compilers, it would be pretty embarrassing for
> them (not to mention pointless) if they _couldn't_ outperform
> competitors on their own hardware.

They will not outperform competitors on every possible code. I know of compilers that handily beat icc for x86 on some codes but also get trounced by icc on other codes. And it is not terribly difficult to examine the generated code of a compiler and figure out a bit of what it's doing to get performance. Reproducing it is sometimes tricky but in most cases it isn't. The techniques are known. The devil's in the tuning.

The problem is mostly one of time and compiler developer resources. By focusing/specializing on a key set of codes or operating arenas (embedded, HPC, etc.) a compiler team can create something that will beat a vendor compiler on those codes or in that particular arena.

"gold standard"?

Posted Apr 26, 2012 22:57 UTC (Thu) by stevenb (guest, #11536) [Link] (2 responses)

"gcc's generated code is notoriously slow."

There are plenty benchmarks and user stories that say otherwise. IMHO there is a perception problem that is being perpetuated because complaints about GCC (or any other compiler) are always shouted out louder than compliments. And saying anything positive about GCC seems to be salon unfaehig anyway in many Linux discussion forums (especially LKML)...

"gold standard"?

Posted Apr 27, 2012 1:07 UTC (Fri) by daglwn (guest, #65432) [Link] (1 responses)

As I said, one can show gcc generating faster executables than other compilers on some codes. This will always be so. But the gcc team does not focus on performance exclusively, nor should it.

For all-out performance over a wide range of codes, it's hard to beat compilers that focus on it, such as icc, pathscale and pgi.

"gold standard"?

Posted Apr 29, 2012 8:24 UTC (Sun) by ebiederm (subscriber, #35028) [Link]

A lot depends on what you mean by performance.

Compiling your kernel of numerical code fast is one thing, and there are certainly compilers that specialize in that and all kinds of cpu architecture extensions to take advantage of it.

Where I have seen gcc excel is in making the work-a-day integer code lean and tight and cache friendly.

Most codes are integer codes and we need good work-a-day compilers not the hotrod numerical code wonders that are fast when the compiler does not ICE or generate incorrect code.

With luck clang can catch up to gcc and be a good work-a-day compiler but it seems clear that clang is not there yet.

"gold standard"?

Posted Apr 27, 2012 3:02 UTC (Fri) by davide.del.vento (guest, #59196) [Link] (6 responses)

> Believe me, there are plenty of much slower compilers.

A couple of names?

"gold standard"?

Posted Apr 27, 2012 10:02 UTC (Fri) by robert_s (subscriber, #42402) [Link] (5 responses)

Phew ok, PCC, digital mars, from what I've seen MSVC, Open64 probably more than half the time (though the situations where it wins can be impressive...), I don't think OpenWatcom can really keep up these days...

...and now LLVM.

"gold standard"?

Posted Apr 27, 2012 10:25 UTC (Fri) by daglwn (guest, #65432) [Link]

LLVM is not a compiler.

"gold standard"?

Posted Apr 27, 2012 11:49 UTC (Fri) by dgm (subscriber, #49227) [Link] (3 responses)

> from what I've seen MSVC

It certainly wasn't that way back when gcc 3.4 was the current release. I know because I benchmarked it for my university. MSCC both compiled faster and produced faster binaries, and by a good margin.

I think that it's time to redo those benchmarks...

"gold standard"?

Posted Apr 30, 2012 18:06 UTC (Mon) by robert_s (subscriber, #42402) [Link] (2 responses)

*shrug*

http://www.willus.com/ccomp_benchmark2.shtml

"gold standard"?

Posted May 2, 2012 11:37 UTC (Wed) by dgm (subscriber, #49227) [Link] (1 responses)

Uh, OK. My wife wants to thank you for saving me a weekend. Hummmm.

AFAICS, MSVC is still *much* faster compiling than GCC (between two and four times faster). Also, MSVC generated code is smaller and runs faster, but the difference is not as big as it used to be.

"gold standard"?

Posted May 13, 2012 10:13 UTC (Sun) by robert_s (subscriber, #42402) [Link]

"Also, MSVC generated code is smaller and runs faster, but the difference is not as big as it used to be."

Well no, according the the "average" page gcc 4.6.3 maxes out at "1.18" for runtime while msvc 2010 maxes out at "1.24".

But, y'know, benchmarks and all that...

"gold standard"?

Posted May 2, 2012 8:29 UTC (Wed) by nwmcsween (guest, #62367) [Link]

> I'm also not so convinced by any "4% off gcc" claims about LLVM. Phoronix (yes, caveats aside) routinely test LLVM releases and it's common for LLVM to be a good 50% behind gcc

50% on very biased benchmarks that utilize openmp, which isn't supported as of now in clang/llvm.