| ||||||||||||||||||||||
Re: Reporting bugs and bisection
On Mon, 14 Apr 2008 01:04:12 -0700 > > The steps to be taken are: > > a) agree that we have a problem > I for one do not agree that we have a problem. Based on actual data on oopses (which very clearly excludes other kinds of bugs, so I know I only see part of the story) we are doing reasonably well. Lets look at the 2.6.25 cycle. We got a total of roughly 2700 reports of oopses/warn_ons from users. (This may sound high to those of you only reading lkml, but this includes automatically collected oopses from Fedora 9 beta testers). Out of these 2700, the top 20 issues account for 75% of the total reports. Out of these 20 issues, 9 were from still out of tree drivers (wireless.git and drm.git included in F9). These were caught before they even got close to mainline. The remaining 11 issues can be split in 1) The ones we caught and fixed 2) TCP/IP warnings that DaveM and co are chasing down hard (but have trouble finding reproducers) 3) An EXT3 bug that in theory can cause data corruption, but in practice seems to happen after you yank out a USB stick with an EXT3 filesystem on (so it can't corrupt the disk data). Ted is working on this 4) A bug (double free) that hits in the skb layer, probably caused by a bug in the ipv4 code (a first analysis + potential patch was mailed to netdev this weekend) 5) sysfs "existing file added" warning, mostly in the USB stack (gregkh claims he fixed this recently, I'm not entirely sure he got all cases) And when I look beyond the first 20, the same pattern arises, we fixed the majority of the issues before -rc9. At position 25 we have less than 20 reports per bug. At position 35 we have less than 10 reports per bug. At position 50 we have less than 5 reports per bug. Conclusion there: the bugs people actually hit fall of dramatically; there's a core set of issues that gets hit a lot, the rest quickly gets reduced to noise levels. To me this does not sound like we have a huge quality problem because 1) The distribution of the bugs is such that there is a relatively small set of core issues that are widely hit, and then there's a near exponential drop after that 2) We are fixing the important bugs by and large before they hit a release (important as defined by the number of people actually hitting the bug) I'll be writing a report with more details about this soon with more analysis and statistics (I'll be looking at more detail around the top 25 issues, when they got introduced, when they got fixed etc) -- If you want to reach me at my work email, use arjan@linux.intel.com For development, discussion and tips for power savings, visit http://www.lesswatts.org (Log in to post comments)
Re: Reporting bugs and bisection Posted Apr 17, 2008 8:41 UTC (Thu) by nim-nim (subscriber, #34454) [Link] I'd be less optimistic. When you add a large pool of testers (F9 beta users) that all use the same family of kernels, with more or less the same kernel patches, the same kernel config, the same gcc version, and the same userspace, it's not surprising at all to get a huge spike of a small set of common problems. Does it means the kernel in general is good? No it means the Fedora Devel kernel has a few oopsing bugs (as should be expected just before a Fedora release)
|
||||||||||||||||||||||