This argument is pure misdirection. There should be little doubt that cryptographically strong PRNGs exist which are just as performant as whatever ridiculous MD4 hack is being used now. Clearly there are cryptographers falling over themselves to try provide the code to Linus & Co.; he's just not hearing it.
Use a strong PRNG now, and worry (or not worry, as it may be) about ASLR entropy later. But who's going to care about the latter if the former would be the weak chain, regardless.
And let's not forget that there's an army of wholly untalented OEM engineers who will follow Linus & Co's lead, and will be (and surely are) using this inadequate PRNG for sundry cases in existing products, mimicking the same, poor calculus Linus uses in his stubborn defense.