Performance [LWN.net]

Performance

Posted Aug 7, 2003 9:42 UTC (Thu) by walles (subscriber, #954)
Parent article: Natively compiled Eclipse

Call me a sceptic, but I'd like to see some numbers backing the claim of the GCJ built Eclipse performing better than a JVM.

I don't doubt for a second that the GCJ build starts up faster than any JVM (as the JVM needs to compile the Java code before it can be run), but it would surprize me if GCJ's runtime performance is anywhere near that of any sane JVM.

Numbers, anyone?

(Log in to post comments)

Performance

Posted Aug 7, 2003 10:07 UTC (Thu) by Carl (guest, #824) [Link]

These are older benchmarks from before some of the Eclipse work (not all of which has been merged back into the main gcc tree yet), but they give an indication of how well gcj does against "the competition":

Performance

Posted Aug 9, 2003 14:49 UTC (Sat) by walles (subscriber, #954) [Link]

The Free VM benchmarks do indeed demonstrate that gcj beats the other Free JVMs. Thus, if you run only Free JVMs, gcj will indeed be an improvement.

Regarding the second set of benchmarks:
* The first one doesn't say anything about for how long the test was run, so it is impossible to say whether the test measures startup time or runtime performance.
* The second one shows that IBM beats gcj if it gets to run longer ("large data"), showing that IBM has better runtime performance than gcj.
* The third shows that IBM is faster than gcj.
* The fourth one "demonstrates the fast startup time for gcj-built programs", but doesn't say anything about runtime performance.

Thus, according to those benchmarks, IBM seems to beat gcj most of the time on runtime performance, even though IBM do bounds checking all of the time which gcj doesn't (look at the build flags for gcj). SUN on the other hand seem to lose fairly vs gcj in some of these benchmarks.

If anything, the above benchmarks hint at that Eclipse might be faster on IBM's VM than on gcj. To have some real evidence of that however, one would need bencmarks run from inside of Eclipse using the same VM instance as Eclipse.

Performance

Posted Aug 7, 2003 11:14 UTC (Thu) by ag (subscriber, #13761) [Link]

This will give you some idea. The Eclipse java compiler can be extracted from Eclipse in order to produce a stand-alone command line tool (ecj).

This is a typical native ecj build (of rhino, in this example). I used the
-repeat option to try to factor out any "fast start" advantage of the native
executable.

$ time find ./ -name \*.java | xargs ecj -bootclasspath /home/green/tools/FSF/HEAD/i/share/java/libgcj-3.4 .jar -repeat 5 -log out.log
Repetition 1/5
Repetition 2/5
Repetition 3/5
Repetition 4/5
Repetition 5/5

real 0m7.136s
user 0m6.660s
sys 0m0.400s

Here's the same compiler being run with IBM's jre...

$ time find ./ -name \*.java | xargs java org.eclipse.jdt.internal.compiler.batch.Main -bootclasspath /home/green/tools/FSF/HEAD/i/share/java/libgcj-3.4 .jar -repeat 5 -log out.log
Repetition 1/5
Repetition 2/5
Repetition 3/5
Repetition 4/5
Repetition 5/5

real 0m20.863s
user 0m19.880s
sys 0m0.710s

An easy way to build ecj for yourself is to get it from here: http://sources.redhat.com/rhug

Performance

Posted Aug 9, 2003 14:23 UTC (Sat) by walles (subscriber, #954) [Link]

You start up the JVM five times. As I said, gcj's startup time should be much better than that of any JVM, and you just demonstrated that.

This doesn't say anything about the runtime performance.

Oops

Posted Aug 9, 2003 14:57 UTC (Sat) by walles (subscriber, #954) [Link]

Sorry, should've read what you wrote in its entirety instead of just brushing you off. Could you try running the same thing with -repeat 1000 instead?

Considering Eclipse is probably run for hours at a time, a seven second benchmark (in the gcj case) may not do IBM's VM justice.

Other than the short time it ran, I'd say that's a reasonable benchmark.