LWN: Comments on "LLVM 3.3 released"

LLVM 3.3 released

JoeBuck — Wed, 19 Jun 2013 22:37:07 +0000

Phoronix has done a lot of gcc vs. llvm benchmarking. Certainly you could raise some issues about parts of their methodology, but their general approach seems fair and they provide enough information to duplicate or critique their work. They generally show that the two compilers are quite competitive other than llvm's lack of OpenMP support which affects some benchmarks severely.

However, there's a risk in focusing on well-known benchmarks because competitive compiler projects do so. It's better to benchmark your own applications, code that the compiler developers haven't seen before, to really understand who's better, and I would expect on average that llvm will compile a bit faster while g++ will produce slightly better code, but the variance will be high enough that it might be the other way around on your code. Both compilers have gotten a lot better recently thanks to the competition.

It certainly appears that llvm has caught up to gcc for the most part. At this point, llvm has regex in C++11 and g++ lacks it; g++ has OpenMP which llvm lacks. Whether this matters depends on whether you need the missing feature.

LLVM 3.3 released

hummassa — Wed, 19 Jun 2013 20:43:20 +0000

That was *really* useful. Many thanks!

LLVM 3.3 released

kreijack — Wed, 19 Jun 2013 18:00:35 +0000

I am quite sure that there are a lot of benchmark where LLVM beat GCC and/or the opposite. And sure, the regression reported should be investigated by the CLANG team.

However, what I stated is that the "Quality of clang generated code" is different from a low performance in a single case.

Does anyone know if there is an organic performances comparative between the two compilers ? Something with a rationale behind ?

LLVM 3.3 released

cmrx64 — Wed, 19 Jun 2013 13:23:56 +0000

I built it myself, along with libcxxrt

LLVM 3.3 released

Jonno — Wed, 19 Jun 2013 12:21:05 +0000

libc++ has been included in Debian experimental for almost a year, and in unstable/testing since May.

Debs can be found at: http://ftp.debian.org/debian/pool/main/libc/libc++/

LLVM 3.3 released

hummassa — Wed, 19 Jun 2013 02:36:21 +0000

There is no .deb at http://llvm.org/apt/, where did you get libc++ from?

LLVM 3.3 released

farsley72 — Wed, 19 Jun 2013 01:14:21 +0000

Disregarding an inconvenient data point doesn't make it any less valid.

It has taken gcc a long while to evolve a good optimizer and code generator. llvm is still a "young" project in comparison, so when llvm/clang folks already start claiming "independent evaluations show that LLVM 3.3's performance exceeds that of (...) its primary competition on many benchmarks", we have a right to call them on their claims.

The bug shows that clang has difficulties optimizing (vectorizing) loops that are slightly more involved than garden variety, resulting in ~50% slower code than gcc.

LLVM 3.3 released

cmrx64 — Tue, 18 Jun 2013 22:27:50 +0000

libcxx runs quite well on Linux, when I tested it ~2 months ago.

LLVM 3.3 released

rriggs — Tue, 18 Jun 2013 21:28:02 +0000

LLVM's "complete" C++ standard library doesn't count in my book until it runs on Linux. LLVM is still quite incomplete in that regard. (This is LWN after all -- with emphasis on the "L".)

LLVM 3.3 released

talisein — Tue, 18 Jun 2013 20:58:41 +0000

GCC now implements the C++11 _language_, but as of yet falls short on the _standard library_.

LLVM now has both. And IMO is a more honest claim to using the word "complete"; confusion over the competing release announcements is not uncommon, and people who want to try C++11 on GCC will be confused when e.g. <regex> never works as expected--it is stubbed out, meaning it won't give a compile error, but there is no implementation.

To be fair, completing the language portion of the standard is an achievement that GCC deserves to trumpet, but the next sentence in their release announcement should have clarified "work continues on the standard library (help wanted)."

LLVM 3.3 released

kreijack — Tue, 18 Jun 2013 18:49:00 +0000

Before comparing the "quality of code" we should define the metrics. IMHO the judgement parameters should be, in order of importance:
- code correctness
- standard conformity
- performance in the worst case
- performance in the best case

I separate the worst and the best case performance, because I prefer a compiler which doesn't show the *WORST* performance in *ANY* case to one which has the *BEST* performance in *SOME* case.

The bug reports a bad performance against a single case. Something to be investigate, but IMHO not sufficient to take any judgement.

LLVM 3.3 released

farsley72 — Tue, 18 Jun 2013 16:36:03 +0000

Quality of clang generated code still seems to be subpar compared to gcc:
http://llvm.org/bugs/show_bug.cgi?id=16358

LLVM 3.3 released

cry_regarder — Tue, 18 Jun 2013 16:15:29 +0000

C++11 not C++1?:

http://gcc.gnu.org/projects/cxx0x.html

LLVM 3.3 released

rahulsundaram — Tue, 18 Jun 2013 15:59:08 +0000

Depends on how one defines major and full support I guess. The actual status is at:

http://clang.llvm.org/cxx_status.html
http://gcc.gnu.org/projects/cxx1y.html

Neither implement *all* the features but that is not unexpected.

LLVM 3.3 released

intgr — Tue, 18 Jun 2013 15:46:46 +0000

Interesting...

> Support for C++11 ref-qualifiers has been added to GCC 4.8.1, making G++ the first C++ compiler to implement all the major language features of the C++11 standard

> At this point, Clang is the only compiler to support the full C++'11 standard