LWN.net Weekly Edition for July 10, 2003

[This article was contributed by Joe 'Zonker' Brockmeier]

Back in February, Reasoning, Inc. released a study that surprised few in the Linux and open source community. Specifically, Reasoning found that the Linux kernel's TCP/IP stack had fewer defects than implementations from vendors with proprietary versions.

This time around, Reasoning has focused on Apache. Reasoning looked at Apache 2.1-dev, released at the end of January this year, and found that release to be about the same quality as commercial software. Reasoning's study was not sponsored by a vendor, nor does the company have any real motivation to find that open source is better or worse than proprietary software. Instead, the company is using studies of open source projects to help promote its testing services. Open source makes an ideal promotional device because Reasoning can actually release the full results to the study, including source code where errors are found.

The company uses a method of automated testing that tests for memory leaks, NULL pointer dereferences, bad deallocations, out of bounds array access and/or uninitialized variables. These are classified as defects. In 58,944 lines of code (LOC) spread out in 360 files, there were a total of 31 defects, or a defect density of 0.53 per thousand lines of code. According to Reasoning, the average defect density for commercial applications is 0.51 per thousand lines of code.

Of 31 defects, 29 of the defects were NULL pointer dereferences and 2 were uninitialized variables -- no memory leaks, bad deallocations or out of bounds defects were found in Apache 2.1-dev. The detailed version of Reasoning's report lists each of the 31 defects, giving the location of the defect, a description and the actual defect in a code fragment taken from the file with the defect.

One might wonder why Reasoning chose to look at a development version of Apache rather than a more mature version that had been out for a while. Certainly, very few people are likely to be deploying a development version of Apache on production sites -- making it less comparable to a release of a proprietary product. Apparently, they decided to review a less mature version of an open source project to point out how the open source development model benefits a project in the long run.

It would be interesting for Reasoning to track Apache's development and compare its quality against proprietary code after it has been released and in use for some time. One suspects that the Apache release would fare better than proprietary projects after it had been formally released and in use for some time.

Obviously, the study doesn't provide the full picture. It only measures certain types of defects, and doesn't take into account the software's features, performance or other qualities. But, at least in the area of software defects, Reasoning's study reflects well on the open source model by demonstrating what many users of open source already know -- that open source produces code of a quality that is at least comparable to proprietary software.

Comments (5 posted)

[This article was contributed by Joe 'Zonker' Brockmeier]

Hewlett-Packard has quietly released a desktop PC featuring Mandrake Linux for small and medium-sized businesses. Last Wednesday the company issued a press release for the Compaq Business Desktop d220, which is available with Windows XP or Linux Mandrake 9.1. MandrakeSoft has also issued a release, which indicates that Mandrake will be on a range of HP's Compaq-branded desktop PCs.

It's encouraging to see one of the major players in desktop PCs getting behind Linux on the desktop. However, it'd be nicer if they were a little more aggressive about the play. HP's release for the d220 desktops doesn't mention that the new line is available with Linux until the sixth paragraph, when one would think that the release of a business desktop machine featuring Linux would be more noteworthy. However, the fact that HP is offering Linux on a desktop machine to SMBs at all is a significant step forward.

A d220 system with an Intel Celeron processor can be had for a mere $327 through HP's site right now, and it's worth noting that a machine with the same specs, but with Windows XP Home Edition, will set SMBs back an additional $50 per machine -- presumably due to the additional cost of adding the Windows license.

It's not exactly world domination, HP is only taking a tentative step in offering Linux to SMB customers on a small slice of its Compaq line. HP's home users, or those looking for a HP or Compaq laptop with Linux pre-loaded, are still out in the cold. (Though there's nothing to stop home users from ordering from HP's small and medium business online store...) But, this small step is necessary to help Linux gain a foothold in the desktop market.

Naysayers and analysts who have continually dismissed Linux as a desktop operating system may have to rethink their position, as it seems unlikely that HP would offer a desktop machine with Linux unless there is sufficient demand for Linux by its business customers, and that HP has decided that Linux is suitable for prime-time on the desktop. If HP is successful with Linux as a desktop offering for SMBs, we can expect to see Dell and others to follow suit very shortly.

Comments (4 posted)