> because the output then gets mixed with the input in the human ear, and the human ear can hear very short delays
The eye can see very short delays too. Certainly moving the mouse or typing on your screen would be a lot less enjoyable if you had 200ms lag in there. And lag alone can't be that big of a problem with audio -- just standing 20 feet away from the speaker will add 20ms on its own.
> when you are in a game and there is a slight delay between hitting the trigger and hearing the sound, it's not really that big a deal, but if the sound from the speakers is delayed from the sound directly from the audio source, it is very jarring.
OK, so it sounds like that concern isn't directly lag, but rather having two sources of audio that are out of sync. And this can often be accounted for by just delaying the faster source slightly to match the delay of the slower (laggier) one. That's how e.g. video games like Rock Band deal with the fact that modern TVs can have quite large lag -- they let you delay the audio so that the audio and video reach the user at the same time.
Can you provide a more specific example setup where you'd see the problem you describe?