| |||||||||||||
More testing is better than less testingMore testing is better than less testingPosted Oct 1, 2012 18:20 UTC (Mon) by oak (subscriber, #2786)In reply to: More testing is better than less testing by sdalley Parent article: KS2012: The future of kernel regression tracking
> More testing is better than less testing.
No, whether that's true depends a lot on the tests and also what you're testing.
I have been in a situation where analyzing results from tests took more time than actually manually finding the bugs *and* fixing them. Eventually we got rid of them. They were quality tests and at wrong level in the stack.
Tests are mostly useful only if:
Preferably they should also be mostly auto-generated so that they get automatically updated with the code, there's less code to maintain and issues with it are more apparent.
Does test code need tests?
(Log in to post comments)
More testing is better than less testing Posted Oct 1, 2012 22:49 UTC (Mon) by sdalley (subscriber, #18550) [Link]
I agree with what you say.
Under the impression there is a relative dearth of formal testing, more testing will obviously be better. I was assuming testing at the appropriate levels in the stack.
It'd already be great to pick lower-hanging fruit like automatic tests for library/programmer/kernel/userspace/filesystem/device interfaces, which must never change without good reason and when they do break, you jolly-well want to know as soon as possible. Regressions Are Bad. If these tests were put into an installable package, then anyone who wanted to help in the testing effort could run them in their own peculiar environment and have the failures forwarded optionally to a central clearinghouse, like Microsoft does with their system crashes.
Harder tests like response latency under varying loads and configurations, memory management fragmentation problems, etc, necessarily have a symbiotic relationship to the code they are testing, and have to be maintained together with them.
More testing is better than less testing Posted Oct 4, 2012 0:02 UTC (Thu) by nix (subscriber, #2304) [Link] Does test code need tests?That depends on its complexity. Testsuite engines are often complex enough to merit it, but it's hard to figure out a way to test most tests except to test the same thing again in a different way and make sure the results of both tests agree. If you know of a less tiresome way that doesn't require doing the same work more than twice (because thinking of a second way to test something is often harder than thinking of the first), I'm all ears.
More testing is better than less testing Posted Oct 4, 2012 0:10 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]
Yes, test code needs tests too. There are projects to do things like finding if tests are efficient. For example, if you comment out (or somehow disable) a part of unit test setup - then the test should fail.
|
|||||||||||||