Everyone insane or what?

Posted Apr 1, 2026 20:27 UTC (Wed) by mb (subscriber, #50428)
In reply to: Everyone insane or what? by pm215
Parent article: The role of LLMs in patch review

>The original Coverity authors had a paper decades back noting the importance of a low false positive rate

What was the number?

to post comments

Everyone insane or what?

Posted Apr 1, 2026 22:08 UTC (Wed) by pm215 (subscriber, #98099) [Link]

Digging out the ACM article I had in mind: https://www.cs.columbia.edu/~junfeng/18sp-e6121/papers/co... the part about false positives is on the last page. They say:

* above 30% is definitely bad
* they aimed for below 20%
* when forced to choose between more bugs and fewer false positives, choose the latter
* the initial reports are really important -- if the first few are bad then the response is "this tool sucks" and people reject it
* "you never want an embarrassing false positive. A stupid false positive implies the tool is stupid"

(My personal experience of Coverity today is that its false positive rate is way higher than I would like.)