At the end of May, five separate open source projects released patches to close the same security hole in their software. This coordinated release and vulnerability handling is a demonstration that "responsible disclosure" can work, especially in open source.
Responsible disclosure is the practice of security researchers discovering a vulnerability and contacting the software vendor to give them a reasonable time to fix it before the vulnerability is published. It contrasts with the policy of "full disclosure" in which security people publish the full details of any vulnerability immediately, in order to get information to the public as quickly as possible. Mostly, these two terms have shown up in the media as part of controversies, or even legal battles, which pit security researchers against software companies and each other.
While the inflammatory confrontations gain most of the news headlines, it
doesn't have to be that way. In fact, among open source projects, it isn't
that way most of the time. The recent multi-product Crypt-DES
vulnerability patch shows that responsible disclosure can and does work
well in the open source world.
The Crypt-DES vulnerability
Robin Xu and Joseph Bonneau at Cambridge University had been investigating how non-ASCII passwords were handled by various systems for more than a year. Bonneau started on this research because of the massive Gawker security breach in 2010. In the course of investigating that, his team uncovered several issues with non-ASCII passwords in commonly used software. While the one at Gawker was quickly addressed — to some degree — he and Xu began a research project on the insecurities introduced by applying algorithms designed for ASCII to Unicode text.
The version of crypt() using
the DES algorithm (hereafter crypt-DES) is a simple irreversible hash
designed to prevent storing passwords in plain text. Introduced in old
Unix days, it had the advantages of easy implementation, portability
between systems and programming languages, computational speed, and is hard
enough to crack that dictionary attacks and social engineering were
generally easier ways to grab passwords. Given the age and limited
computational "strength" of crypt-DES, however, this is no longer true;
brute-force computation of crypt() passwords is easily done.
Programmers are encouraged to use more modern hashing and encryption
algorithms, such as SHA1 and Blowfish. The "extended" DES version was
introduced in BSDi in the early 1990's, improving the algorithm to have a
larger "salt", more rounds of encryption, and also to support passwords longer than eight characters by "folding" them down to eight 7-bit characters using a first round of DES hashing.
The last improvement is the problem which causes the crypt()
vulnerability. Crypt-DES was designed for ASCII characters, and
programmers who upgraded systems to support Unicode didn't really check to
see how crypt-DES would work with Unicode passwords, since by that point
crypt-DES was no longer mainstream. As it turns out, the folding is
broken; the algorithm regards characters containing the byte 0x80 as a
"stop" character and disregards any parts of the password after that byte.
In many Unicode encodings, characters — such as the common character À
— can contain a 0x80 byte, causing all characters after that one to be
disregarded. This means if your password was Àlbanez60, then crypt-DES
would match it with any password beginning with À.
This is also a good illustration of how security is a process and not an end result. Crypt-DES was an adequately secure password hashing approach well into the mid 1990's, which is why people stopped testing it. It was the introduction of popular Unicode-compliant versions of programming languages and databases which has made it less secure than anyone realized.
Having found this issue, Xu, Bonneau, and other Cambridge graduate students spent several weeks examining some common software and found that the defective version of Crypt-DES was still shipping with several open source software packages, among them PostgreSQL and FreeBSD. Having found the vulnerability, they emailed the private security mailing lists for the affected projects.
The PostgreSQL security team received this email on April 24th:
My name is Rubin Xu, a PhD student at University of Cambridge. While my colleague and I were investigating how websites handle non-ASCII passwords, we noticed a glitch in one of the standard DES crypt() implementation which causes certain Unicode passwords to be truncated before being DES digested. Unfortunately PostgreSQL seems to be shipping with the offending code as well.
The Cambridge team had previously contacted a few other projects, including FreeBSD. The FreeBSD and PostgreSQL projects had to decide what to do about patching the vulnerability. For anyone affected by it, an updated version of crypt-DES would require that all affected passwords (ones containing the 0x80 byte) be regenerated. While neither PostgreSQL nor FreeBSD used crypt-DES for system authentication, both supply functions which are used to hash application passwords. Because of this disruption to some users' applications, it couldn't be done casually.
The FreeBSD security team contacted the OpenBSD, NetBSD and DragonflyBSD projects. Rubin Xu's research indicated that PHP's crypt() also had the faulty algorithm, and had attempted to contact the PHP security team without success. Members of FreeBSD contacted them and brought them into the discussion. NetBSD turned out not to be vulnerable.
Coordinating a release
Among the affected projects, this vulnerability was considered moderate in severity, since it only affected a minority of users of each project. Not only did users need to build applications using crypt() with DES, despite other, more modern hashing options being available, but the password vulnerability only affects passwords with Unicode characters including the 0x80 byte. Specifically, the vulnerability was limited to:
BSD users who used the included crypt() library with DES (the default) to support their applications.
PostgreSQL users who installed the optional pgCrypto extension and used its crypt() function with the default DES encryption.
PHP users who used the PHP crypt() function with DES on platforms without native encryption support, most notably Windows.
On the other hand, the vulnerability affects passwords, which means it's specifically a hole in code people have written to secure their systems. That raised this vulnerability from obscure to moderately serious. So FreeBSD filed for a Common Vulnerability and Exposure number (CVE), and the projects began trying to coordinate a release.
From the perspective of the projects, once one project announced a release
became public, it wouldn't take much cleverness for a even a newbie black hat to figure out the vulnerability in other products. That meant coordinating a release date among five different projects. In surprisingly short order, they reached a compromise date of May 30th, which was the earliest reasonable release date. On that date:
The entire timeline from the discovery of the vulnerability to deploying
fixes for multiple projects took about three months. The majority of this
time (about seven weeks) was taken up by the researchers finding and
contacting affected projects. If there's room for improvement in the
process of responsible discovery, it's that finding affected projects or
products and contacting their security teams is slow and time-consuming.
The remaining five weeks is only two weeks longer than the minimum time for
most packaged projects to do a release at all, due to packaging,
scheduling, testing, and coordination requirements. One could easily argue that immediate disclosure would have gotten the news about the vulnerability out much sooner, but it's not clear how that would have benefited affected users until fixes for their software were available.
In relatively short order, five major open source projects were updated to close it. Nobody was threatened, no single project's users or developers were singled out, the security researchers were thanked for their work, and nobody needed to spend more than a few hours of their time getting the fixes made and released. At least from the perspective of software maintainers and regular users, this episode looks like a success.
Why it worked this time
This whole episode had two important factors to make it a relative success: the security researchers were university staff unmotivated by fame or profit, and the open source projects are community non-profits lacking incentives to defer or deny patching security holes. This meant that everyone involved was motivated to fix the vulnerability in the fastest, most responsible way possible.
This is by no means exceptional in the open source world. On the PostgreSQL project today, as with many other open source projects, companies and academic researchers regularly practice responsible disclosure, letting the developers know about a security issue in a reasonable time to fix it. If anything, this is the rule in the non-profit open source world. So why does disclosure cause friction, user exposure, blog wars, and legal threats in the for-profit world?
Well, when you look at failures of security disclosure, the overwhelming
trend is bad faith. Software companies don't want to do expensive releases
and get bad press for security issues, so they put off security researchers forever, or even threaten them. Security people or their employers want fame and attention so they publicize security holes as widely as possible without verification, or giving the vendor a chance to patch issues. Or worse, researchers, companies and agencies participate in a marketplace of secret security exploits.
So, while responsible disclosure can and does work in the non-profit open source world, it's not clear how to transfer these practices to the for-profit world, or even if it's possible to do so. Maybe the answer is simply to use more open source software.
[ Note that MITRE has not updated their CVE database. As such, the CVE
link for the exploit will still show as "pending". ]
to post comments)