Noise is better than bias
Posted Jul 11, 2006 18:31 UTC (Tue) by jimmybgood
In reply to: A survey on kernel quality
Parent article: A survey on kernel quality
I'm going to try real hard once more to explain my point, which seems to have been missed.
If you want to find out something with a survey, a large noisy sample is far better than a small biased sample. If Andrew Morton thinks he's going to find out something by limiting his survey to a small sample of "knowledgeable" respondents, he's making a mistake. He works in an environment where expert knowledge is highly desirable and out of habit, he imagines that experts might be able to tell him whether the kernel is getting buggier.
Even studying the kernel itself with objective code analysis tools is not a valid way to answer his questions, because those tools have recently been applied to the kernel. Many of the bugs those tools can detect have already been fixed, so the bias will tend to make the kernel appear to be less buggy than it really is. A survey is a good approach to finding out if the Linux kernel is getting buggier.
I have no proof that LWN subscribers are biased, but it is generally accepted that professional societies _are_ biased. I think LWN functions more as a professional society than as a social group.
If you want a good survey, procure as many unique responses from as wide a sample as you can get.
to post comments)