Comparing free and proprietary software defect rates

Comparing free and proprietary software defect rates

Posted Feb 13, 2003 19:53 UTC (Thu) by giraffedata (subscriber, #1954)
In reply to: Comparing free and proprietary software defect rates by rakoch
Parent article: Comparing free and proprietary software defect rates

I think the results will be similar everywhere.

Around 1992, there was a research paper on the same topic, with the same results. In that case, a team of programmers fed random input to a whole bunch of Unix programs and noted when the programs crashed. Where they had source code, they debugged the crash and sent the information to the maintainer of the code.

Actually, the paper's main point was the source of the bugs (buffer overrun, etc.) but there was an unmistakeable difference in bug rates between free software and commercial software. At the time, this surprised many of us because the common wisdom in the industry said commercial software was bound to have fewer bugs because of all the investment in testing and because commercial publishers had more to lose from bugs.

But there was another result in that paper which I found much more interesting, which I think explained the phenomenon. The study was a followup on a study done the same way years earlier, which had found the same difference. In the followup, the programmers looked for the same bugs that had been reported in the original study. In commercial software, nearly all of the bugs were still present. In free software, nearly all of the bugs had been removed.

As a software developer for a major software publisher at the time, this didn't surprise me one bit. The software development machine of IBM is not capable of fixing a product just because it learns it's broken. But an individual free software developer not only is capable of releasing a fix, but insists on it as a matter of pride.

--
Bryan Henderson bryanh@giraffe-data.com
San Jose, California

---

(Log in to post comments)