This should make bisecting a kernel problem fun, especially if the address layout only makes the problem appear on average half the time.
Same with trying to reproduce a hang that locks your system only some of the time. Did you remember to write down the random offset at boot-time just in case?
There can also be interesting performance analysis implications. Randomized userspace processes can vary a few percent in performance when randomized if poorly written (SPEC 2000 perlbench (*cough*)). Things like taking addresses of variables and using them as keys in hash tables, etc.