I got this intel_hda audio driver, with pulseaudio generating 1000s of warnings per second about something or other. It generally doesn't say what the warning is, only that the ratelimiter suppressed thousands of warnings. Great design of ratelimiter!
If I try to watch a video, the sound dies with some pulseaudio-related connection error and then nothing works until I pretty much restart the desktop environment. I guess pulse has a habit of crashing. When it restarts, if it restarts, it sometimes selects the wrong outputs so I notice this because I have to switch output from analog headphone jack back to digital jack. I do this every few days.
On my java app, if I use the default ondemand cpu governor, sound is very choppy, but switching to performance helps (incidentally, this also mostly fixes my video watching problems). I guess ondemand scheduler doesn't work out that the CPU is actually pretty loaded, or maybe the pulseaudio time-based scheduling screws up thanks to variable CPU speed.
So I got numerous crazy issues and I haven't really even tried to make progress pinpointing the causes. I am expecting the fundamental problem is with the intel_hda driver, which has got a ton of issues because it's really an umbrella driver that supports a lot of different hardware, some better and some worse.
I guess my real complaint is just that this shaking pile of cards made out of alsa, pulse and gstreamer doesn't really manage to hide the issues at the bottom of the stack. (And neither can it. Wrappers can't fix problems as a general rule.) However, I could justifiably complain that the pile of junk on top of shaky foundation does make what problems there are somewhat worse.