The SPEC benchmarks are only benchmarks, but *sudden worsenings* in SPEC
output are a good clue that something's just been broken. It's also useful
to keep an eye on C compilation speed (bootstrap-and-test time is also
somewhat useful, but the compiler and testsuite are growing, so that is
expected to rise even if the compiler got no slower).
Posted Nov 2, 2007 11:45 UTC (Fri) by jfj (guest, #37917)
[Link]
Yes. But take for example, auto-vectorization. icc does it and it gets some amazing results in
SPEC. But the fact is that *any real program that really cares about speed cannot depend on
the compiler to auto-vectorize*; its developers will provide custom assembly with MMX for the
critical parts. Now that is a red herring. Maybe LTO as well.
Also, I suspect that commercial compilers spend a lot of effort to look good in SPEC. Maybe
they even identify those benchmark programs and generate predefined assembly, I don't know :)
I generally think that timings on real programs is far more useful. There could be a testsuite
that runs some timings on bzip, gzip, ffmpeg encoding/decoding, etc. SPEC's little programs
are to sensitive to small changes.
Finally, I think that open source would benefit more from new gcc extensions and special
attributes.
gcc is a skyscraper
Posted Nov 2, 2007 13:04 UTC (Fri) by nix (subscriber, #2304)
[Link]
LTO definitely isn't a red herring: most programs consist of more than one
object file and can't easily use --combine (which is far too memory hungry
to be useful anyway: I tried compiling GCC 4.2.1 with --combine and it ran
my 1.5Gb Athlon out of memory...)