While I agree with your general point, I have two points of
partial-contention.
Most importantly, if you allow password authentication at all, you need to
pay attention to failed logins, and take steps to prevent someone from
eventually getting around to guessing your good password. (I watch my
logs and manually block IP addresses that are trying to brute-force their
way in, but I'm lucky enough not to be getting hit nearly as hard as
others are.)
Also, I've long disliked fail2ban and similar tools because it's too easy
for legitimate users to get blocked. But some variant might be
appropriate, such as one that notifies the sysadmin and asks for approval
before blocking. (Though of course this adds more of the evil complexity
that you rightly argue against.)
I've toyed with the idea of a system that not just penalizes failed
logins, but also gives positive points for successful logins, so that it's
harder for legitimate users to get blocked. And of course, if blocking
does happen, it should eventually expire.