Yeah. What I'm really interested in here is being confident that an
intruder who steals the private key of someone with login rights to my
system cannot use it to log in... but I suppose even passphrases won't
help there, as if they can steal a key they can almost certainly get root
and install a keylogger, and the passphrase is toast.