A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World (CACM)
Posted Feb 16, 2010 20:18 UTC (Tue) by pascal_cuoq
Parent article: A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World (CACM)
By sheer coincidence this CACM article is published at about the same time an unfortunate bug
afflicts Toyota cars, and a lot of people promptly make a link between the two and comment
along the lines of "ah, they should have used static analysis software, how silly of them not to
There is only one little flaw in this reasoning: perhaps they did!
For all we know, Coverity's software has been run on Toyota's hybrid car. Coverity's static
analyzers are quite popular, and I would be more surprised to hear that Toyota hasn't bought a
license than to hear it has. Assuming the static analyzer was run on this particular piece of
software, perhaps the bug was drowned out in a sea of false positives, or perhaps it wasn't
detected at all.
Yes, there are bugs that are not detected by Coverity's product. This is one of the key points in
the CACM article, in fact. It does not aim to be correct. Giving up correctness against
conventional academic wisdom gave them leeway...(just read the article, I wouldn't do it justice).
Testing, or analyzing, software until it is reasonably sure there is no bug left, is doable. In
aeronautics, engineers do it, with success, repeatedly. Simply because they have to: there is a
certification authority to which they have to explain what they did and why they think it's
convincing proof that the probability of a software bug remaining is negligible.
When they do it with assistance from analysis tools, they use correct static analysis tools: tools
that do not remain silent in presence of a bug. This, in principle, is at the cost of more false
negatives, but the state of the art has advanced enough for the number of false negatives, for
well-defined verification activities, in a pre-established context, to remain contained. It's a far
cry from the "buy a CD inside a cardboard box and run it on your code" attitude of Coverity, but
it works too. Examples of such tools are the now commercially available Astree and the Open Source Frama-C.
There is no such authority and no equivalent obligation for the automotive industry. If you don't
like the idea of dying in a car accident caused by software, write to your representative. Don't be
too harsh on Toyota, though: they are only doing what everyone else is doing, because they have
to to remain competitive. The same thing is bound to happen to another car integrator, unless
the rules of the game are changed for everyone.
to post comments)