Coverity releases first defect survey results

Posted Mar 7, 2006 23:42 UTC (Tue) by tialaramex (subscriber, #21167) [Link]

Figuring out whether any strange code identified as a potential defect by Coverity is actually a bug (causes the software to behave in a way that doesn't match the design/ intention of the programmer) can be fairly difficult. Figuring out whether that bug manifests itself with real-world scenarios is also fairly difficult. Figuring out whether there's a real security problem (often the most serious type of bug) can be so hard that it's safest to assume that all bugs of certain types are security holes even if, in fact, few of them are ever actually exploited. However, the only broadly accepted "proof" that a security hole exists is the "proof of concept" exploit, actual code that breaks the security but has a harmless payload.

Yet, taking the same defect, and turning it into a piece of code that's obviously correct and properly annotated is often fairly trivial. So it may be that you can spend a day or two fixing all the Apache problems found by Coverity, while a colleague takes a week just to examine a single reported defect, then decides that while it is technically a bug, it can only cause any problems in some unlikely corner case.

So all of that means you're right, it's unlikely that anyone will collect such statistics, because doing so is much more work than fixing the defects and has no direct pay-off except to qualitatively validate the usefulness of Coverity.