Publicly funded free software security audits [LWN.net]

The static analysis tool once known as the "Stanford Checker" has occasionally shown up here on LWN. The Checker has often been applied to the Linux kernel code base, resulting in the detection (and fixing) of hundreds of bugs before they created trouble on production systems. It is clearly a powerful tool, and it has often been hoped that the Checker would be released as free software. That was not to be, however; instead, it evolved into a proprietary product called "Prevent," offered by a company called Coverity.

The Coverity folks have occasionally posted information on problems found with their software, and those bug reports have been appreciated. It now looks like that stream of information is about to increase; Coverity has announced that it (along with Stanford University) has received a grant from the U.S. Department of Homeland Security to help improve the security of free software. To that end, Coverity Prevent will be run against some 40 free software projects (the release lists the kernel, Apache, MySQL, PostgreSQL, Sendmail, FreeBSD, Mozilla, and GTK) and the results will go into a publicly-available bug database. The project is described as "multi-year"; an initial availability date for the bug database was not provided.

Some people who have yet to fully understand free software have been heard to wonder what benefits come from access to the source. These people may not be programmers, and have no clue what they would ever do with that code. Here is a clear example of why free software is better. All users of the packages analyzed by Coverity Prevent will benefit in a number of ways:

The number of bugs found in each package will be public information, as will how that number changes over time.
Users who are concerned about the security and reliability of the code they use will be able to see just how responsive each project is to the bugs which are found.
Developers will - one hopes - learn from the types of bugs which are consistently found in their packages and get better at avoiding them.
These bugs - many of which are reliability and security problems waiting to happen - will be fixed.

Proprietary software simply is not available for third-party auditing in this manner.

Most of this is not new; the auditing (and fixing) of free software is an ongoing process. The free software community does not, yet, have tools which are as good as Prevent, however, so its regular application to free source should be a good thing. And the bug database should be full of interesting information which will help potential users judge the relative security of the covered projects.

One could argue that the Department's funds would have been better applied to the creation of free tools which perform detailed static analysis of code. Then all projects could benefit from the results. Still, direct government support for free software is rare in the U.S. (especially outside of scientific funding agencies), so this grant looks like a step in the right direction.

There are risks involved in an effort like this. If developers are not responsive to the bugs reported by Prevent, the bug database could become an easy shopping list for malware authors. The bug database also offers some FUD possibilities: similar databases do not exist for proprietary software products. But we should not fear public disclosure of our bugs; it makes us stronger in the end. This project, if it lives up to its potential, will result in a higher-quality, more secure code base for all free software users.

(Log in to post comments)

Watch out for BitMover-like tactics

Posted Jan 12, 2006 6:11 UTC (Thu) by JoeBuck (subscriber, #2330) [Link]

In the past, Coverity has asked people to sign (or click through on) non-compete agreements before seeing the bugs database they generated for free software apps. They did not do this with the Linux kernel, but they did do it for their analysis of another tool (Apache, if I recall correctly) a while back. I believe Linus's "sparse" would fall in the category of competition. Since I, among other things, work in formal verification, I could not accept the terms so I did not look at the Apache data.

I hope that the DHS will not allow Coverity to use taxpayer money to pull such stunts.

All that said, Coverity has excellent technology.

Publicly funded free software security audits

Posted Jan 13, 2006 12:52 UTC (Fri) by mv (subscriber, #17258) [Link]

I'm curious how useful the results will be.

The Coverty checker I think is primarily a checker for possible defects in source code. If it is anything like splint, it will return large numbers of warnings for typical unannotated source code, most not directly related to security.

Fixing the defects obviously has a positive effect for security, but someone still needs to go through and analyse them manually to determine if they are security bugs (and thus warrant security updates etc.).

Coverity's iffy relationship with free software

Posted Jan 13, 2006 16:54 UTC (Fri) by bos (guest, #6154) [Link]

When Dawson Engler's students were developing their checker at Stanford, they did so by modifying gcc rather heavily, and resisted all calls to make their modified version of gcc available outside their group.

They were of course entirely within their rights to do so, but it left a bad taste in many people's mouths, especially as their evasions at the time ("it's too messy to release in its current form") proved to be a holding tactic, so they could acquire venture funding and hook their analysis code to a proprietary front end, letting them ditch gcc and avoid the GPL.

That they have since attempted to play distasteful licensing games with their analyses of other free software is doubly unfortunate, especially as their contributions to the Linux kernel were well received.

It wouldn't have been hard for Coverity to retain the good faith of the free software community, but if they feel like they want to earn it back, they'll have to work hard for it, press releases and DHS funding notwithstanding.