You're still missing the point that the latency only matters if the computer is both taking input (e.g. midi keyboard) and also creating the output (e.g. synthesized audio). Surely you would agree that latency doesn't matter at all if you are e.g. recording singing from a microphone to accompany a guitar track you are playing back. They can be perfectly synchronized in software to match what you heard while singing.
I think my only real error was underestimating how frequently people need to either apply live digital effects or otherwise play live synthesized audio from live external triggers, which sounds like "always" from what people are saying here.