Heartbleed aftermath
Resources
One commonly heard assertion is that the OpenSSL project simply does not have the resources needed to maintain a code base of that size (about 300,000 lines of code), complexity, and importance. Companies take OpenSSL for granted, it is said; they make vast amounts of money using it, but do not contribute back to its development. Steve Marquess, the OpenSSL "money guy," put it this way:
There may be value in asking, though, whether support levels are the real problem and, if so, what form the solution would take. There are calls for users and companies to donate to the project, but nobody seems to be pointing out one simple fact: it is a rare development project indeed that operates out of cash donations. It just does not seem to be a sustainable way to keep developers employed working on a project. Companies don't donate cash to OpenSSL because, almost without exception, they do not donate to free software projects at all.
That does not mean that resources are not available for OpenSSL development, though. It is interesting to continue reading in Steve's posting on the subject. The OpenSSL Software Foundation offers consulting services starting at $250/hour. Those rates are apparently not high enough to put off clients:
Given that, one might reasonably conclude that the limiting resource is not money, it's qualified developers. That is a different sort of problem.
Coming up with more qualified developers is not something that will happen overnight. Cryptographic programming is not easy, and the TLS protocols are seemingly not designed to make it any easier. If companies are waiting with contracts in hand for qualified developers to just show up, they may find themselves waiting in vain while the quality of the software continues to suffer. Perhaps there is a better way.
Projects that have large numbers of commercially supported developers are typically not employing those developers through a donation-supported foundation. Instead, those developers are employed directly by the companies that have an interest in the development of the software. In the most successful settings, these companies have in-house programs to bring developers up to speed on the projects they care about the most. That is the sort of support that projects like OpenSSL need; the companies that depend on it need to be directly employing (and training) developers to get this work done. Once that happens, OpenSSL should improve quickly.
The perils of closed enhancements
One of the companies that arguably should be employing OpenSSL developers is Akamai, which uses OpenSSL heavily on its massive content distribution network. On April 11, Akamai proudly told the world that it could not have lost SSL keys via the Heartbleed vulnerability, despite having run vulnerable versions of the software. Why?
The reasoning was that this secure heap, being kept far away from normal working memory, could not be exposed via Heartbleed. Akamai kept this code to itself; it was not contributed back upstream to the OpenSSL project. Or, at least, it was not contributed until Akamai posted it on April 11.
It took less than two days for an independent reviewer to look over the code and conclude that, in fact, it was buggy and offered no protection at all. By late on April 13, Akamai had conceded the point and begun the process of changing out all customer SSL keys. The company deserves credit for facing up to its problems once they were pointed out, but it can't have been Akamai's proudest moment.
Akamai, it seems, had been using its custom allocator for many years. Had that code been contributed upstream, there is a good chance that the bug would have been found a long time ago. Akamai would have benefited with secure code that was actually secure and the damage to the net as a whole caused by the Heartbleed vulnerability would have been considerably reduced. There really is value in this whole open-source development idea, but it seems that this lesson has to be learned the hard way over and over again.
End notes
There does seem to be one influx of new developer effort, though: the OpenBSD project has apparently decided to rip the code apart and clean it up to its standards. The result seems likely to be a cleaner and more secure OpenSSL in OpenBSD, but there is no evidence of any sort of coordination or communication with the OpenSSL project or of any intent to push these changes back upstream. In other words, it seems likely that OpenBSD has just forked OpenSSL. This move may allow some immediate problems to be addressed but, in the long term, this project (which, like most projects, has resource constraints of its own) has just taken on the maintenance of a complex, security-critical body of code. The cost of that decision will have to be paid continuously into the indefinite future.
Finally, we have had a few queries about LWN's server. As LWN readers know, we are strong proponents of running the latest and greatest software with all the newest fixes and features. That is why the main LWN server is running on CentOS 5. First impressions notwithstanding, this version of CentOS does not actually predate the World Wide Web, but it is old enough to have never had a vulnerable version of OpenSSL, so LWN's servers were not vulnerable to Heartbleed for even small period of time. The company that does LWN's credit card processing (Braintree) was vulnerable, but responded quickly once the vulnerability was disclosed. PayPal, our other payment provider, claims to have never been vulnerable but says nothing about why that is true.
In summary: there should be no need to change LWN passwords or to worry about payments made for LWN subscriptions.
The Heartbleed vulnerability is a reminder that there are classes of bugs
that can affect a substantial portion of the Internet. Heartbleed may or
may not have been extensively exploited against people on the net, but, in
a sense, it almost does not matter. The chances that other vulnerabilities
exist and are known to hostile entities would appear to be quite high.
Free software may well offer the best defenses we have against security
problems, but it is clear that those defenses are nowhere near as strong as
they need to be. For years we have been telling the world that it can rely
on our software; now that the world is, to a great extent, convinced of
that fact, we need to accept the responsibility to make sure that this
claim is actually true.
The LWN site is currently under high scraper load, so comment display has been suppressed for anonymous users. If you are a human, you may read the comments by clicking the button below:
Note: you can avoid this step in the future by logging into your LWN account.
