A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World (CACM) [LWN.net]

A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World (CACM)

Posted Feb 16, 2010 20:18 UTC (Tue) by pascal_cuoq (guest, #63637)
Parent article: A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World (CACM)

By sheer coincidence this CACM article is published at about the same time an unfortunate bug afflicts Toyota cars, and a lot of people promptly make a link between the two and comment along the lines of "ah, they should have used static analysis software, how silly of them not to have..."

There is only one little flaw in this reasoning: perhaps they did!

For all we know, Coverity's software has been run on Toyota's hybrid car. Coverity's static analyzers are quite popular, and I would be more surprised to hear that Toyota hasn't bought a license than to hear it has. Assuming the static analyzer was run on this particular piece of software, perhaps the bug was drowned out in a sea of false positives, or perhaps it wasn't detected at all.

Yes, there are bugs that are not detected by Coverity's product. This is one of the key points in the CACM article, in fact. It does not aim to be correct. Giving up correctness against conventional academic wisdom gave them leeway...(just read the article, I wouldn't do it justice).

Testing, or analyzing, software until it is reasonably sure there is no bug left, is doable. In aeronautics, engineers do it, with success, repeatedly. Simply because they have to: there is a certification authority to which they have to explain what they did and why they think it's convincing proof that the probability of a software bug remaining is negligible.

When they do it with assistance from analysis tools, they use correct static analysis tools: tools that do not remain silent in presence of a bug. This, in principle, is at the cost of more false negatives, but the state of the art has advanced enough for the number of false negatives, for well-defined verification activities, in a pre-established context, to remain contained. It's a far cry from the "buy a CD inside a cardboard box and run it on your code" attitude of Coverity, but it works too. Examples of such tools are the now commercially available Astree and the Open Source Frama-C.

There is no such authority and no equivalent obligation for the automotive industry. If you don't like the idea of dying in a car accident caused by software, write to your representative. Don't be too harsh on Toyota, though: they are only doing what everyone else is doing, because they have to to remain competitive. The same thing is bound to happen to another car integrator, unless the rules of the game are changed for everyone.

(Log in to post comments)