I pretty much concur. The userspace loader causes problems even when it isn't being used at all. e.g. I just got a new machine with a Barts GPU in it (ATI Radeon 6870), and since I have a largely non-modular kernel with KMS built in I put all the firmware I thought it would need into CONFIG_EXTRA_FIRMWARE. I forgot one file... so the system hung for something like sixty seconds(!) trying to call out to userspace to load the firmware -- even though PID 1 had not yet been forked off! Of course because this was brand-new hardware and it was right at modesetting time I thought I'd messed something up in the .config or there was something major missing in the kernel's KMS support or something like that.
The whole thing is a complete trainwreck, from the need to dig through source code to find the right names to jam in CONFIG_EXTRA_FIRMWARE through the userspace loading that you can use except in unusual situations such as if you have even a single firmware-using module built into the kernel or if you need even a single firmware-using module to resume from hibernation. (Oh, and how much assistance does the kernel give you in detecting that you have either of those situations? None, that's how much. It just dies without a message at the appropriate time. IIRC there's been talk about fixing the hibernation side of this, but I don't think anything ever came of it.)
This whole thing was designed entirely to let distro vendors produce something without violating the GPL, and it does that -- but unfortunately it makes it bloody hard for the rest of us to produce working systems without digging through the source code if we have anything needing firmware at all, even if we're not using modules for anything.
(Sorry, Matthew, I really don't like to criticise your work -- but this banjaxed-up firmware-loading mess just wasted several hours of my time hunting for a 'lockup' that wasn't, due to a hugely overlong timeout that even the stupidest kernel should not have incurred, on a kernel with no loadable firmware of any sort, before userspace was even running. This is not something that has been tested in the non-modular case with an eye to not being intolerably awful.)