| |||||||||||
Compared to commercial?Compared to commercial?Posted Mar 7, 2006 13:50 UTC (Tue) by daniels (subscriber, #16193)In reply to: Compared to commercial? by peterh Parent article: Coverity releases first defect survey results
Actually, Coverity's checker is pretty thorough: it turns up stuff you wouldn't necessarily expect an automated checker to.
(But, of course, there are some false positives, and undoubtedly false negatives. But it's turning up stuff even a skilled human wouldn't necessarily get, and good luck finding someone sufficiently skilled, willing to review the entire X codebase.)
(Log in to post comments)
Compared to commercial? Posted Mar 7, 2006 19:44 UTC (Tue) by kleptog (subscriber, #1183) [Link] Obviously the checker itself needs to be taught how to deal with various aspects of the programs it checks. For example, for PostgreSQL it doesn't appear to recognise that elog(ERROR, ...) never returns, leading to many spurious warnings about using variables inappropriately.
It's still a really nice technology and could help track-down some of the more obscure bugs.
Compared to commercial? Posted Mar 7, 2006 20:12 UTC (Tue) by daniels (subscriber, #16193) [Link] That's just a matter of training, though: it at least appeared to recognise all the cases like that for X. The only false-negative I saw like that was some really hideous error-handling code involving longjmp.
Compared to commercial? Posted Mar 8, 2006 5:32 UTC (Wed) by peterh (subscriber, #4225) [Link] Yes, but you're missing the point. Automated checkers are supposed to be thorough --- they tend to find the sorts of bugs that you wouldn't expect to find yourself.
There's been a reasonable amount of research recently on using reasonably classical compiler-type program analyses, such as abstract interpretation and dataflow analyses to detect bugs (Metal/Coverity checker and Saturn out of Stanford, and I think Cousot and others were doing work on verification of aerospace systems in France, and there are no doubt more that I can't think of right now). The real innovation of Metal is that it has a "find bugs at all cost" mentality, irrespective of the theoretical soundness of what it does. The result is probably quite good as a tool for finding certain classes of bugs. But I doubt it's sensible to make conclusions about the total bug count of a program based on what the checker detects.
|
|||||||||||