Using certificates for SSH authentication

SSH is a well-known mechanism for accessing remote computers in a secure way; thanks to its use of cryptography, nobody can alter or eavesdrop on the communication. Unfortunately, SSH is somewhat cumbersome when connecting to a host for the first time; it's also tricky for a server administrator to provide time-limited access to the server. SSH certificates can solve these problems.

Verification

The OpenSSH project maintains what is, in practice, the reference implementation of the SSH protocol. In the default configuration, its SSH client asks the user to verify the server's identity (its "host key") when first connecting to it, by examining a key fingerprint, which is a secure checksum for the server's public key:

    The authenticity of host 'stamina (10.2.2.176)' can't be established.
    ED25519 key fingerprint is SHA256:cawPF9UX0DwRb5F1kr55JkQ9xNNXZEeqG8MtuV82dRU.
    Are you sure you want to continue connecting (yes/no/[fingerprint])?

The user is asked to verify the host key by comparing long strings of hexadecimal. In the example, the fingerprint is the letters and numbers after SHA256:. This manual check is tedious and error-prone.

There are alternatives to checking the fingerprint manually. The server administrator can publish a list of host keys for their servers, in a format that the client can use in its "known hosts" file. The file could even be updated by a centralized IT department on the user's machine. If there is no centralized IT, the user could update the list themselves. There's also a way to publish the host keys over DNSSEC, to avoid having to update known-host lists on user machines.

But what usually happens is that the user answers "yes" by reflex.

Authentication

The user then needs to authenticate themselves to the server. Traditionally this has been done using passwords, but passwords are a security weakness. Humans just aren't good at remembering secure passwords. SSH allows key-based authentication, which can be much more secure. By default, this works by the user maintaining a file with authorized public keys in their home directory on the server.

At login time, if the user's client can prove to the server that the user has a private key corresponding to one of the keys listed, the user can log in without a password. Overall, this is a big win for security, except for the first time a user accesses the server. How can they add their public key to the list of authorized keys before they can log in?

For this, too, there are alternatives. An organization might centrally collect the public keys of their users. The server administrator could use this to maintain the list of authorized keys for each user. This solves the first-login situation. However, all of these alternative mechanisms need to be added on top of SSH; they also need to be developed and maintained.

In principle, a host's key never changes. Sometimes, it does, however. For example, a server might be replaced, reinstalled, or its domain name changed to refer to a new one. This happens often with virtual machines and in the cloud environment. The server might also be a board used for embedded-system development, where the operating system gets replaced from time to time.

Likewise, a user may need to get a new key for various reasons. They might accidentally post their private key on social media, or just lose the file and not have backups. If nothing else, attacks on cryptography only get stronger and, over time, a strong key will become weaker, resulting in a need to generate a new key, with newer, stronger cryptography.

Certificates

OpenSSH supports an "SSH certificate", which includes an SSH public key along with some metadata related to the key, all cryptographically signed by a trusted party. The certificate is used instead of the SSH public key when connecting, though the corresponding private key is still needed in order to complete the connection; the included signature allows the key to be trusted without manual checking. The trusted party is typically the server system administrator, who controls access to their servers. They create an SSH key dedicated for use as a certificate authority (or CA), and use that key to create host and user certificates. A CA key is just a plain SSH key, created with ssh-keygen, like any other SSH key. The CA key is used to sign (ssh-keygen -s) host and user keys, creating certificates.

The system administrator configures their servers to provide host certificates and to trust user certificates made by the CA, while users configure their clients to trust host certificates signed by the CA. This allows a user to trust a host's identity without having to manually verify the host key. It also allows a server to authenticate a user without passwords, or without the user's public key being in a list of trusted keys for that user. Almost.

There is still one step where some trust needs to be established in a way that isn't automated. How does a CA get a user's public key and how does the user get the CA's public key in a secure way? There's no generic, scalable solution for this, but the problem is somewhat simplified because it only needs to be done rarely.

A solution for this initial exchange of public keys will build on existing trust relationships. Perhaps the parties have OpenPGP keys and can use the Web of Trust for this? Maybe they can access the same intranet of the organization? It might even be enough to rely on TLS and get public keys over HTTPS.

Note that the SSH CA mechanism does not require or support a network of globally trusted certificate authorities the way that TLS does. This means it can be used without paying, or at least involving, a third party, but also means that obtaining the CA key is a manual process. An SSH client does not come with a list of implicitly trusted SSH certificate authorities.

The SSH certificate doesn't work on its own, just like a public key doesn't: the party presenting a certificate must prove (using cryptography) that they have the corresponding private key. The metadata in a certificate makes certificates more useful than plain public keys, however. An SSH public key does not specify which host or user it belongs to, for example, whereas a certificate has a list of "principals" that it vouches for. A principal is a host or user name. The metadata in a certificate can be examined with ssh-keygen -L.

    Type: ssh-ed25519-cert-v01@openssh.com host certificate
    Public key: ED25519-CERT SHA256:V/jJ9f+0sxWuK4Kmjr9x26OUmrQV+pPE8PmnKnFL650
    Signing CA: ED25519 SHA256:6qIHXzaIX8CJ0jO/Iv0rvhZIlwaWxmOVItuD+973HUI (using ssh-ed25519)
    Key ID: "certificate for host stamina"
    Serial: 0
    Valid: from 2022-11-03T10:02:00 to 2023-02-01T10:03:26
    Principals: 
	    stamina
    Critical Options: (none)
    Extensions: (none)

Certificates can have a validity period, which allows them to automatically expire without having to be explicitly revoked. OpenSSH does support revocations, but revocation information can be difficult to communicate to all parties in a fast, reliable way. A common way to sidestep the need for revocation lists is to use short-lived certificates (this is what Let's Encrypt does for TLS certificates). For example, a continuous-integration (CI) worker process might be given a certificate for deployment that only lives for 60 seconds. A student at a university might get one that is only valid for a semester, while an employee might get one that is only valid during working hours of that specific day.

Short-lived certificates are only useful, of course, if a user can get a new one when they need it. For example, a company might store the public keys of all its employees in a secure database and then automatically issue a new one-day certificate to employees every morning. The employees can even retrieve it from an unauthenticated web server. Certificates don't contain any secrets, and they only work with specific private keys, so the transport doesn't necessarily need to be protected strongly.

User certificates can also be tied to specific IP addresses or restrict the commands that the user can invoke. This further limits the damage an attacker can cause if they can steal someone's private key.

Certificate management

Managing the SSH CA infrastructure can be done using only the tools provided by OpenSSH, particularly ssh-keygen. The man page (and the certificates section in particular) has all of the necessary information, though in quite a dense form. There are various HOWTO documents around the Internet to explain things in more accessible ways; I maintain a HOWTO for SSH CA and the OpenSSH Cookbook has a section on SSH CA too.

The SSH CA mechanism first appeared in OpenSSH version 5.4 in 2010, so it has been around for quite some time at this point.

An organization (or even just a solitary system administrator) with many hosts or many users may want to create automation around ssh-keygen. This will allow securely managing a collection of host and user public keys, as well as CA private keys, in order to conveniently and quickly issue certificates on demand. SSH certificates can then be integrated into configuration management with ease. I am in the process of developing such tooling, creatively called sshca, and would appreciate feedback.

Conclusion

SSH certificates are a way to make the use of SSH more convenient and more secure at the same time. The technology is widely available, and implementing it for an organization is straightforward, at least from an SSH point of view. For anyone who is struggling with the key-management aspects of SSH, it is a mechanism that is worth investigating.