The Grumpy Editor's guide to SSH servers

This article is part of the LWN Grumpy Editor series.

Back in March, your editor received some not-entirely-friendly communications from a prominent OpenSSH developer. This person was unhappy about a number of things found in the article about OpenBSD's financial issues, as well as one thing that was absent: a discussion of OpenSSH alternatives. The point which was supposed to emerge from such a discussion is that there are no viable alternatives. Your editor has set out to try to determine if that is truly the situation or not. To that end, this article will look at SSH server implementations; the client side of the picture will be addressed in a future article.

There are a number of things one can look at while evaluating an SSH server. Features, for example: which ciphers are supported, port forwarding features, control over what users can do, PAM integration, etc. One can also look at performance issues; data-heavy SSH sessions can put a significant load on the host system. But the issue which must dominate the others is security. An SSH server is designed to give access - perhaps full, root access - to a suitably authorized user coming in from an arbitrary location on the net. Any vulnerabilities in this server thus have a high probability of turning into a full compromise of the system.

Evaluating security is hard. Certainly one can look at security-oriented features found in a given implementation, and there will be useful information there. But features do not make security; that requires careful coding, extensive code review, and quick response to security issues as they come up. It requires an active development community which continually works to tighten the security of the server. An SSH server which is the subject of a large number of security advisories would make your editor nervous, but a server with a moribund mailing list and no advisories at all would be worse.

With these thoughts in mind, your editor set out to play with the three SSH server implementations he found which are free and under some sort of active development.

Dropbear

Dropbear is an SSH server and client implementation available under an MIT-style license. It runs on just about every Unix-like system, including Cygwin. Dropbear development places a strong emphasis on small size; it is intended for use in embedded systems and other space-constrained situations. The current version of Dropbear is 0.48.1, released on March 12, 2006.

As might be expected in a program which is meant to be small, Dropbear offers fewer features than some others. It can perform X11 connection forwarding (and port forwarding in general), and has options for controlling whether password authentication may be used to log in. There is no configuration file, however, and many of the options available with certain other servers are not implemented in Dropbear.

Dropbear can do passwordless login using RSA or DSA keys. It understands OpenSSH-style authorized_keys files, allowing the same keys to be used with both servers. The key format for host keys is different, however; a script is provided to convert OpenSSH keys into Dropbear's format if needed. Dropbear can be configured to perform password authentication through PAM, though one gets the sense that most installations don't bother.

There is little information available on the ciphers supported by Dropbear. A look at the code, however, shows options for AES-128, AES-256, triple DES, Blowfish, Twofish-128 and Twofish-256.

Dropbear appears to have an active developer and user community. There is a fairly long list of distributions listed as using Dropbear, including OpenWRT, OpenZaurus, Trinux, and Motorola A780 phones. The volume on the mailing list is steady but low - Dropbear users apparently have little to talk about. The last publicly-acknowledged security issue was in March, 2006, when a denial of service problem (which also affected a wide variety of other network servers) was fixed. Prior to that, fully remotely exploitable format string vulnerability was disclosed (and very quickly fixed) in 2003. Another remote vulnerability was disclosed in 2004 and yet another was fixed in early 2005. In December of 2005, a "buffer sizing error" which could enable root access for authenticated users was fixed.

The code base is small - a little over 23,000 lines for both the server and client - but not particularly well commented. The Dropbear code should be relatively easy to audit; the extent to which anybody has done so is unclear, however.

lsh

Lsh comes billed as "a GNU implementation of the secure shell protocols." So, unsurprisingly, it is released under the GPL. Lsh provides both client and server implementations. The current release of lsh is 2.0.3, from May 9, 2006.

The lshd server daemon, like Dropbear, lacks a configuration file; it does have a number of command-line options for controlling options like password authentication and port forwarding. There is support for public-key authentication in lshd, but OpenSSH-format keys must be converted into the lsh format first. The converted key must then be fed to lsh-authorize before the server will recognize it. There does not appear to be an lsh-unauthorize command, making it more challenging than it should be to revoke access for a specific key.

Documentation for lsh is more complete than for Dropbear. From that documentation, one sees that the supported ciphers are AES-256, triple-DES (though it is listed as "3dec"), Blowfish, and ARCFOUR.

Disclosed vulnerabilities in lsh include a file descriptor leak enabling a local denial of service attack (January, 2006), a denial of service problem (March, 2005), and a remotely exploitable buffer overflow (September, 2003). While lsh releases do continue to happen, it is not clear how large the user and developer community really is. The lsh mailing list is dominated by spam, with legitimate messages seemingly being carried at a rate of less than one per month.

Lsh is written in C, but a look at the code gives the impression that the author would rather be using something else. Some sort of preprocessor is used on the code, a memory garbage collector has been implemented, there appears to be some sort of exception mechanism in place, etc. As a whole, the code is harder to read than the Dropbear code, and it is not clear that this code has seen much attention from anybody other than its original author.

All told, your editor would hesitate before committing to lsh; it is far from clear that this tool has the user and developer communities needed to keep it alive and secure into the future.

OpenSSH

OpenSSH is clearly the dominant offering in this area. All available evidence indicates that almost every publicly reachable SSH server is running OpenSSH. This implementation is maintained by the OpenBSD developers; the current release is 4.3 (or 4.3p2 for systems other than OpenBSD) from February, 2006.

If you are looking for features, OpenSSH is the way to go. The sshd_config man page lists a vast number of options controlling authentication mechanisms, ciphers used, user restrictions, file locations, port forwarding, and more. The list of supported ciphers includes ARCFOUR, blowfish, CAST, and several variants of AES. OpenSSH is clearly the most feature-complete of the SSH server implementations; it is also, in many ways, the best documented.

Vulnerabilities disclosed in OpenSSH include a root compromise in 2001 (but only when an obscure configuration option was set to a non-default value), a set of integer and buffer overflow vulnerabilities in 2002 which affected relatively few sites, a remotely exploitable heap corruption bug in 2003, an access restriction bypass vulnerability in 2003, a remotely exploitable PAM-related vulnerability in 2003 (non-default configurations only). The nastiest of these will be the 2003 heap corruption bug, which is thought by some to have been actively exploited for some months prior to being fixed.

It would appear that no OpenSSH server vulnerabilities have emerged since 2003 (there has been one client-side vulnerability since then). As this article is being written, there is some discussion on the OpenSSH list of a number of bugs found by a Coverity scan. Fixes are in circulation, but there does not appear to be much concern that these bugs are exploitable.

The OpenSSH developers clearly take security seriously. The code base is probably the most heavily reviewed of the three implementations discussed here. The OpenSSH server also has a "privilege separation" feature, wherein the bulk of the protocol code (prior to the establishment of the user's session) runs in a separate, unprivileged process. This mechanism will, it is hoped, contain the damage should an exploitable vulnerability turn up in that code in the future.

The handling of the 2002 integer and buffer overflow vulnerability raised some eyebrows; the developers refused to disclose specifics on the vulnerability, insisting, instead, that all users perform a significant upgrade to the current release. They have made it clear that they would do so again:

The fact remains that the OpenSSH developers have earned a high level of trust, and that most users are entirely happy in their place. The OpenSSH mailing list is active, with a steady flow of questions (and patches) from the user community.

OpenSSH is implemented with a significant amount of C code. The code base is written for OpenBSD in particular; the version the rest of us use is the "portable" release which has seen added tweaks to make it run elsewhere. There is a set of regression tests packaged with the code as well.

Conclusion

Your editor began this project with the idea of determining whether there are truly no alternatives to OpenSSH on the server side. Of the two discussed here, only Dropbear looks even remotely viable. For resource-constrained applications dropbear may even be the preferred choice, but it can also be used in any other setting that does not require the larger feature set of OpenSSH. As noted above, your editor is made a little nervous by lsh, and would choose to avoid it.

There are two schools of thought on the OpenSSH monoculture - for that is essentially what it is. Some, including your editor, find the situation a little scary; a serious vulnerability in OpenSSH could be the opening needed for a devastating Internet worm. To these people, some diversity in the OpenSSH ecosystem could only help to make the net as a whole more secure. Others, however, feel that we are better off with a single code base which can benefit from the concentrated auditing and hardening efforts of the entire development community.

One can only hope that there is some merit in the latter view, given that, for most systems, OpenSSH is the only viable choice available.

---

to post comments