| |||||||||||||
How about a distro-provided bisection facility?How about a distro-provided bisection facility?Posted Apr 15, 2008 20:46 UTC (Tue) by JoeBuck (subscriber, #2330)Parent article: Bisection divides users and developers Let's say you're a distro, and a user complains that your shiny, newly released kernel has a major regression. Why couldn't the distro itself provide a bisection-generation facility? This could be some combination of pre-built bisections (maybe for the first 2-3 cuts) and nice packaging to automate bisection generation. Ideally the new kernels could be tested in the context of a live CD distribution, to minimize the risks from running unstable kernels. One could even conceive of a special kernel-testing distro that would run off of a live CD and automate the whole process. The CD (which might be on a USB flash device instead) would just iterate the following process, and the user would only need to wait for the compiles and test for the bug when prompted:
newest_good_kernel = what_I_was_running_before;
oldest_bad_kernel = what_you_shipped_me;
while (more_than_one_rev_between(newest_good_kernel,oldest_bad_kernel)) {
midpoint_kernel = git_bisect(newest_good_kernel,oldest_bad_kernel);
if (big_pipe && someone_has_built(midpoint_kernel))
download(midpoint_kernel);
else
compile(midpoint_kernel);
reboot_and_fire_up(midpoint_kernel);
tell_user_to_test;
if (user_says_the_bug_is_still_there)
oldest_bad_kernel = midpoint_kernel;
else
newest_good_kernel = midpoint_kernel;
reboot_and_fire_up(known_good_kernel); // for the next build
}
bad_patch = compute_diff(newest_good_kernel, oldest_bad_kernel);
send_report(bad_patch, user_comments);
Some types of bugs, such as file system corruption showing up after a while, would be trickier to test for, and the live CD would have to be able to ask for scratch media, be able to reset it to a known state, etc. But if testing can be made easier for interested users, we'll get more testing.
(Log in to post comments)
How about a distro-provided bisection facility? Posted Apr 15, 2008 21:58 UTC (Tue) by jd (guest, #26381) [Link] Some distros (Red Hat and SuSE spring to mind) are big enough that some (but not all) bisecting could actually be done automagically on a server at the distro's HQ. I'm picturing something like this:
This method has several advantages. Firstly, if the bug can be easily repeated, it moves the heavy lifting from users to people who (usually) have more powerful hardware at their disposal. Secondly, by distinguishing hardware-specific and hardware-agnostic bugs, there is automatically more information available for debugging. Thirdly, you really want to get to the final destination of having a way of reporting and filtering bug reports that maximizes both the quantity and quality of what kernel developers get, which means the manual parts have to be minimal and reducable by automation. It also has several disadvantages. More users can bisect than can produce an automatable test plan. It's far harder for an automated system to eliminate non-identical reports that are of the same bug and carry no additional information. Too many automated bug reports may lead to developers ignoring them - and a bug in the bug reporter itself certainly would. So few distros can afford the hardware that would be required to do this well that it would have limited benefit. By necessarily using such high-end hardware, as opposed to what users are likely to have, a lot of hardware-related (and almost all hardware-specific) bugs - which, beween the two, will account for a sizeable fraction of all bugs - cannot be automatically bisected on a remote machine. Automated reporting systems cannot answer additional kernel developer questions or carry out additional testing onthe developers' behalf. Ultimately, the question becomes one of how to get the most results from the most testing, given that testing is something programmers generally avoid if possible and the users most likely to do something funky enough to cause a crash are the ones who don't know what they're doing. The semi-automated method above won't solve that last one, though.
How about a distro-provided bisection facility? Posted Apr 15, 2008 23:05 UTC (Tue) by JoeBuck (subscriber, #2330) [Link] Certainly this is possible (to have people at the distros do the bisection), and in fact it's already being done, but many kernel bugs that escape into released kernels aren't noticed because they only affect users with very specific hardware. And these, I think, are exactly the cases where kernel developers ask the end user to bisect it, because they have no way on their end to make progress.
How about a distro-provided bisection facility? Posted Apr 16, 2008 11:29 UTC (Wed) by michaeljt (subscriber, #39183) [Link] Would the kernel revisions really have to be distributed as source? Perhaps they could be distributed as pre-compiled object files. This would be much quicker for testing, might do away with the need for scratch space (if there was enough RAM available for linking), and space could still be saved by only including the object files which had actually changed since the last revision in a particular revision on the live CD. kexec could be used to load the newly linked kernel.
How about a distro-provided bisection facility? Posted Apr 16, 2008 11:47 UTC (Wed) by michaeljt (subscriber, #39183) [Link] Or perhaps I am thinking too complicated - some sort of binary diffs should do the trick just as well or better.
How about a distro-provided bisection facility? Posted Apr 16, 2008 14:05 UTC (Wed) by nix (subscriber, #2304) [Link] Both of these have the problem that kernel configurations are wildly variable and capable of enormous variation. This would only be practical for a limited set of distro-compiled kernels (thus with known .configs), but it might work for them, and that would still be quite useful (I guess it's more likely that someone who can build their own kernel can bisect it for bugs as well).
How about a distro-provided bisection facility? Posted Apr 16, 2008 14:10 UTC (Wed) by michaeljt (subscriber, #39183) [Link] I think that the original poster was indeed talking about distribution kernels.
|
|||||||||||||