i think there're quite a few issues conflated in this discussion, let me try to clear them up.
1. purpose of ASLR
i think as the creator of ASLR, i can speak with some authority here: ASLR was *NOT* designed or ever meant to protect against local attacks. really. it was meant as a temporary solution against certain exploit techniques used in remote attacks only.
there're two very simple reasons for this decision: one is that on localhost it's very easy to bruteforce even 32 bits of randomness (if you remember last year's task refcount overflow bug), and we can't get much more than that even for 64 bit userland (note that for remote attacks even 32 bit userland is adequately protected, at least in PaX, despite what the Shacham paper claimed).
the other reason is that anyone able to run an exploit on localhost is much better off by exploiting a kernel bug simply because the kernel is always there whereas the userland process/program may or may not be on the target box (and may not even give root access directly anyway when exploited). not to mention that there's virtually no protection against exploiting kernel bugs, except some attempts in PaX. so it's a lot more economical to develop and use kernel exploits on localhost. this line of reasoning was true back in 2001 as much as it is true (even more so) these days.
2. PRNG for ASLR
for the above mentioned reasons it should be clear that there's no need for a strong PRNG for ASLR's use as the same infoleak that would allow an attack on the PRNG would already destroy the protection provided by ASLR anyway, there's no use in attacking the PRNG itself. one could digress into situations where the attacker leaks randomization info from one process and uses it to reconstruct another's, but that's easy to defend against without resorting to a strong PRNG.
3. fast and 'good enough' PRNG
independently of ASLR, it would still be nice if the kernel provided a 'strong' but fast PRNG device that one could for example use to sanitize a harddrive at raw write speeds, something that isn't possible with /dev/urandom for example. if such a PRNG existed it could then of course be used for ASLR as well but ASLR itself can live with less (ditto for the SSP cookie by the way).