I have had just about every problem you've mentioned. However, I use Slackware, and crappy sounds cards, so I assumed it was "just me". But I've yet to see a LiveCD or something more user-friendly that didn't correctly pick up all my sound cards and just use them properly from the outset (hell, a lot of the LiveCD's get ambitious and play a sound on X starting).
The wrapper layers are annoying in some cases but pretty much everything handles itself now. I haven't had an audio problem in several releases of my distro - it "just works", with whatever I throw at it. The ones that are left are those cases that are caused by external factors. As a case in point, DOSBox and most SDL apps delay their sound by about a second for me in Windows every time (I can manually update all their SDL dll's to a version that doesn't) but on Linux, the same DOSBox/program versions with the same hardware and the same SDL version have never showed a problem.
I think audio was one of the last things to actually be "fixed" in my opinion. I have bad memories of loading soundfonts, editing .asoundrc, fighting with default volume levels, and the volume sliders changing between kernels etc. but everything settled down about two years ago. There'll still be problems, sure, but it's stablised. Hell, I can remember fighting to get automatic recognition of USB keys in KDE and that was the same sort of thing - one upgrade and all my work was in vain.
Sound took longer than it should have, of course it did, but now I don't think there's anything *near* a modern computer that you can't just throw a LiveCD or basic distro at and not have everything "just work". My cheapy-Taiwanese, new-design laptop was fully working in Linux from day one despite obviously never having been designed for Linux (ACPI errors, etc. a wireless card that was only supported in Linux days before the laptop was released, nVidia graphics etc.).
The last four or five major kernel releases, I look at and think "Well, it doesn't support any more hardware, it doesn't do anything I really need, it doesn't change much at all" and only upgrade if I need the security of a more up-to-date kernel.
It'll never be magic and perfect and work for *absolutely* everyone but I think that it's pretty damn good, especially when you compare it to the nightmare of trying to install the right Realtek HDA driver on Windows or similar without a disk.