User: Password:
Subscribe / Log in / New account

KS2009: Regressions

KS2009: Regressions

Posted Oct 19, 2009 17:22 UTC (Mon) by dyqith (guest, #31406)
Parent article: KS2009: Regressions

Maybe what we need is a group/company/foundation/person to come up with a huge collection of machines for automatic regression testing for new kernels.
There should be a couple of machines from different categories (servers, laptops, desktops, mobiles) and manufacturers (Dell, HP, etc), and for testing at different levels (configure/compile, static analysis, boot, devices, filesystems, memory, etc).

We have a set standard of tests, and users/devs can add in new tests when new features are needed to be tested.
The output of the tests can then be posted to a bug tracker somewhere.

The way I see it is testers have two problems (time and materials).
Some have lots of time to test things, but no machine to play with;
others have no time, but lots of machines.

(Log in to post comments)

KS2009: Regressions

Posted Oct 19, 2009 19:12 UTC (Mon) by dmk (subscriber, #50141) [Link]

KS2009: Regressions

Posted Oct 19, 2009 21:07 UTC (Mon) by nix (subscriber, #2304) [Link]

We *have* that. The problem is that users tend to have stranger hardware
and stranger configurations than anything that we can actually *test*:
this is as opposed to Ingo's randconfig does-it-boot testing, which while
otherwise excellent doesn't spot problems where drivers not needed for
booting and building kernels don't work properly. I've had at least one
regression with every one of the last four released kernels, and not one
of them was something that would have been spotted by Ingo's randconfig
testing (because that testing had wiped out all the low-hanging fruit

However, all those regressions got squashed *fast* once I reported them:
even the one I didn't realise was a regression until someone else reported
it (the stuck-keys-in-X-on-SMP 2.6.31 locking bug). The key is to report
and test :)

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds