Assuming you have a 64-bit processor, the improvement relative to the 32-bit processor (or the 64-bit processor in a pure 32 bit mode) is basically irrelevant. No surprise there.
The question would be decided in both cases rather by the performance advantages of running 64 bit code with either 32 bit or 64 bit pointers. My understanding is that many Java applications slow down by as much as 20% simply by switching from a 32 bit JVM to a 64 bit JVM on the same machine.
Considering that the JVM implementation is gaining registers and register width, that result is rather remarkable. The only explanation appears to be that the cache impact of larger pointers in a Java environment is severe. Take a look at this for example:
Naturally, the reasonable expectation should be that an x32 JVM would be considerably faster than both current IA-32 and current x86-64 JVMs. An ARM64 JVM with 32 bit pointers should be considerably faster than ARM64 JVM with 64 bit pointers for the same reason.