Don't use passwords, especially for privileged access. SSH provides a two factor system which is adequate for anyone who doesn't find the command line intimidating.
Create user SSH private keys only on machines you have physical control over, or where local policy prohibits such caution, on a few machines where you have the most confidence of their integrity.
Use a passphrase to protect the keys. Use SSH agent software. The agent software uses the passphrase to unlock a key and hold it in memory temporarily. If the key files (stored on disk) are obtained by malicious third parties (e.g. by accessing backups, old drives mistakenly sold without being wiped) they are useless without the passphrase to decrypt them.
You can then use the key to log into machines without presenting a password at all, if the target machine is compromised you reveal only the relatively worthless public key identity of your user. An attacker witnessing hundreds of logins of this sort with full root access on the target machine learns nothing except the identities of frequent users. Contrast this to the situation with passwords, where he learns hundreds of valuable passwords.
Where convenience (or e.g. firewall policy) dictates that you SSH into one security pivot machine A and from there into others (B ... Z), you have two choices which reflect different security beliefs about the machines.
1. SSH to machine A, separate key on that machine for SSH to B ... Z. This keeps security of the set A ... Z separate from security for any other machines you have access to. An attacker in full control of A can authenticate (using the key stored on A) to other alphabetic machines at will but not elsewhere.
2. SSH agent tunnel. Local machine SSH agent is tunnelled to A for authenticating connections from A to other alphabetic machines. This limits power of attacker in full control of A to hijacking your authentication capability when you are logged into A. They can authenticate to any system for which your key validates, not just A ... Z, but the key and its passphrase remain inviolate on your local system.
In either case attackers who somehow have privileges on other alphabetic machines (B ... Z) learn nothing helpful and cannot leverage their privileges to attack other systems via SSH.