Why people don't test development distributions

Posted Jul 7, 2009 8:30 UTC (Tue) by lmb (subscriber, #39048)
In reply to: Why people don't test development distributions by nirik
Parent article: Why people don't test development distributions

One could, with reasonable strength of argument, come to the conclusion that software that cannot be automatically tested well is badly designed.

(The counter argument that there are some things that are extremely difficult to test - like usability - does not diminish the assessment that very few software packages get anywhere close an acceptable test coverage.)

Why people don't test development distributions

Posted Jul 7, 2009 12:48 UTC (Tue) by MathFox (guest, #6104) [Link] (3 responses)

It would help a lot if the infrastructural packages came with self-tests that are routinely run before a new version is shipped. (Yes, writing tests takes time; catching the bugs timely saves a lot of time.) For a lot of software it is not-economical to write a 100% coverage automatic test (games!) or the hardware is not available. However that should not stop you from testing the 80-90% that can be tested.

Why people don't test development distributions

Posted Jul 7, 2009 19:32 UTC (Tue) by Max.Hyre (subscriber, #1054) [Link] (1 responses)

It makes a big difference to actually enforce testability. You get your nose rubbed in how much better things work.

In one wonderful (in the sense of getting to follow all those ``best practices'' you read about in books) job, we had automated testing, and were required to build in test points. Which were then evaluated in the code reviews.

Of course, we were building medical devices, and you tend to be a bit more careful when the FDA is looking over your shoulder. :-) I suspect we were within shouting distance of the 80–90 % number.

But that's only 80–90 % number. If that's what we could do under duress, it's hardly surprising we're lucky to get maybe 30 % in the real world. It's human nature (especially programmer nature) to believe both

I made no mistakes this time, and
I can't afford to take the time for real testing.

Until we figure how to overcome those predilictions, we'll have alpha-level ``releases''.

Typo

Posted Jul 7, 2009 19:39 UTC (Tue) by Max.Hyre (subscriber, #1054) [Link]

`Predilections'. So much for level of testing.

Why people don't test development distributions

Posted Jul 8, 2009 11:05 UTC (Wed) by nix (subscriber, #2304) [Link]

prelink *has* self-tests, but perhaps this is a problem that only shows up if ld-linux.so.2 itself is prelinked, in which case you wouldn't see it unless you prelinked the whole system.

(I'm running eglibc 2.10-head and prelink and see no trouble, though. Local RH patch breaking something? What to?)