> > How about crowd-sourcing that driver testing then?
> That's exactly what we do, and what we expect when we do the -rc releases.
> You are running them to test that nothing breaks on your machine, right?
Wrong. I am not testing -rc releases, because i have other stuff to do. And i'm not complaining that my hardware doesn't work either, which makes my behavior wholly consistent. Just pre-empting that comment.. :)
There is a lot more that could be done to make this "crowdsourced testing" more effective. Currently it is quite difficult to test out -rc releases. You have to know how to compile a kernel, and how to install and run it in your distribution. Certainly not rocket science, but not easy for the average distro user, which is who you'd need to go after for large-scale outsourced testing.
Just an idea.. What if bootable live test images could be created for -rc releases? Ideally they would need no local storage, but optionally they could use a dedicated partition. The live image could do a whole bunch of tests and send back the results, together with information the hardware the tests ran it. Those could be analyzed for problems. Also the tests could include performance tests.
Every time a new -rc would be released, you'd rebuild the live image, and ask people via G+, Facebook, Twitter, the mailing list, etc, to burn it to a CD and boot their system with it. The CD runs for a few hours overnight, sends back the results, and then says "Thank you, i'm done". I bet you that this could increase your testing base by 10x. Of course you want to be pretty sure that it is safe and put a lot of safeguards in place to make sure it is (which of course doesn't mean you don't need a pretty scary disclaimer before running the CD).