Testing for kernel performance regressions

OSDL tried this, and trust me it failed. The problem is that lots of hardware is expensive to maintain and most tests give no meaningful result. They invested at a least 10 person/years of effort in getting a working test infrastructure and running tests, and the result was only a few bugs that were worth the attention of kernel developers.

Random testing is often better than organized testing! Organized testing works for benchmarks, but the developer in tawain who boots on a new box and reports that the wireless doesn't work is priceless.