Speed is important, but in many cases before you can make it faster you have to make it reliable. Firstly, with a system that's full of race conditions, trying to speed it up will make it more likely that you'll run into more of them, and nobody will be very happy with that. Secondly, many of the speed problems with the current setup are essentially due to sledgehammer workarounds for things that would otherwise be unreliable. Put the system together in a better-thought-out way with more precise constraints on its behaviour and *then* it's a lot easier to make it faster.
(Petter said much the same thing at the boot BoF at DebConf, IIRC.)