Study: comparing free and proprietary network stacks [LWN.net]

A company called Reasoning has put out a press release on a defect study it did of six different TCP/IP stacks. "Reasoning found 8 defects in 81,852 lines of Linux kernel source code - the fewest number of defects of the various implementations of TCP/IP inspected by Reasoning as part of its study."

to post comments

Study: comparing free and proprietary network stacks

Posted Feb 11, 2003 16:39 UTC (Tue) by bryn (guest, #1482) [Link] (2 responses)

So presumably these 8 defects can be patched for the next kernel release, and Linux would then be the only example of a zero-defect stack....

Study: comparing free and proprietary network stacks

Posted Feb 11, 2003 17:08 UTC (Tue) by robertbrooks (guest, #9557) [Link] (1 responses)

as always FITNR ;o)

It really depends on the nature of the "bugs" they've found in Linux's tcp/ip implementation.

Occasionally I've seen the kernel hackers argue about what it the correct way of handling a given situation on linux-kernel. The choices generally comes down to pragmatism (ie what works best in the real world) and interoperability. Going against standards isn't necessarily the issue though some may argue the standard says one thing it often comes down to interpretation.

It wouldn't suprise me if some of the points they see as bugs are deliberate features of a well tested and working (in the real world) tcp/ip stack.

Study: comparing free and proprietary network stacks

Posted Feb 11, 2003 17:36 UTC (Tue) by ncm (guest, #165) [Link]

Judging from the text of the PR, they are not comparing the stacks' behavior against the RFCs; they are just analyzing the code itself, and detecting such mistakes as using values of uninitialized variables. The defects they find are real, even when they don't necessarily result in noticeable bugs; any small change to the code might change that too. One problem with this process is that it produces false positives, which take engineering attention to winnow out.

Probably a big part of the reason Linux comes out ahead here is that a similar process is already being applied to Linux code by those lunatics at Stanford. One benefit to being important Free Software is that it becomes practical and worthwhile to apply this kind of attention, for graduate credit. Linux really is a part of the academic literature, and benefits correspondingly from the academic attention. (BSD, too, of course, and for much longer.)

various implementations of TCP/IP ?

Posted Feb 11, 2003 19:38 UTC (Tue) by cpeterso (guest, #305) [Link] (1 responses)

And which other TCP/IP implementations did they inspect? FreeBSD? Microsoft?

Other TCP/IP stacks examined?

Posted Feb 12, 2003 8:43 UTC (Wed) by Duncan (guest, #6647) [Link]

The PR didn't say, specifically, altho it did offer the following:

<quote>
Reasoning's study compared six implementations of TCP/IP, the fundamental
protocols underlying the Internet. Besides the Linux kernel, three of the
implementations were part of commercial general-purpose operating systems, and
two were embedded in commercial telecommunications equipment.
</quote>

The "embedded in commercial telecom equip." comment was interesting, as it
could well refer to "life-line rated" equipment known for and relied upon for its
stability. That Linux does better than that, assuming that's indeed what they were
reviewing, is VERY impressive credentials, indeed!

The PR did say the white paper on which it was based is available for free, and
gave the contact/request details, as well as listing the conference where it is to be
presented. Thus, the info on the other stacks studied isn't a big secret, altho they
likely were teasing the reader into requesting it, or attending the conference. It's
equally likely, however, that they didn't outright mention the other implementations
in ordered to avoid undesired raw "my OS is better than your OS" debates in the
coverage, at least until after the formal presentation at the conference.

Study: comparing free and proprietary network stacks

Posted Feb 12, 2003 17:30 UTC (Wed) by md2perpe (guest, #9594) [Link] (1 responses)

Could it just be some variables belonging to a .bss section (thereby being automagically zeroed)?

Study: comparing free and proprietary network stacks

Posted Feb 13, 2003 5:34 UTC (Thu) by Peter (guest, #1127) [Link]

Could it just be some variables belonging to a .bss section (thereby being automagically zeroed)?

Unlikely. For someone sophisticated enough to do a code audit of six network stacks and count bugs, that would be a pretty glaring misunderstanding of C. It's not like zeroing the bss is either undocumented or nonstandard.