Left by Rawhide

Posted Jul 16, 2012 21:22 UTC (Mon) by johannbg (guest, #65743)
In reply to: Left by Rawhide by louie
Parent article: Left by Rawhide

"It only suits the project's needs well because the project, collectively, is a total failure on the QA front"

Care to elaborate on that comment?

Btw without proper test cases and without proper debugging procedures hundreds or thousands of people testing the latest code every day is meaningless...

to post comments

Left by Rawhide

Posted Jul 16, 2012 21:29 UTC (Mon) by louie (guest, #3285) [Link] (3 responses)

Btw without proper test cases and without proper debugging procedures hundreds or thousands of people testing the latest code every day is meaningless...

People testing the code gives you data on the parts of the software that people actually use. That's certainly not perfect, and shouldn't be the end-all/be-all of testing, but having people use and report back on the parts that get used is absolutely the best software testing methodology there is, period.

Left by Rawhide

Posted Jul 16, 2012 21:34 UTC (Mon) by johannbg (guest, #65743) [Link] (2 responses)

And this is relevant to my comment how?

Left by Rawhide

Posted Jul 16, 2012 21:53 UTC (Mon) by louie (guest, #3285) [Link] (1 responses)

hundreds or thousands of people testing the latest code every day is meaningless...

Hundreds or thousands of people testing the code is never, ever meaningless. If you think it is, your QA processes are broken.

Context?

Posted Jul 23, 2012 11:06 UTC (Mon) by gmatht (subscriber, #58961) [Link]

I am not sure what the precise context is here. But if the context is having hundreds of thousands of people test software that is known not to work at all, then you aren't going to get much useful feedback. In many projects there is a "latest software that passed basic automated tests" tree, which may be subtly broken but will at least partially work. I don't see any point in having hundreds of thousands of people test anything newer than that; having a thousand "nothing works" bug reports show up at the same time doesn't really help anyone.

Left by Rawhide

Posted Jul 17, 2012 0:02 UTC (Tue) by rgmoore (✭ supporter ✭, #75) [Link]

Users and test cases are different and provide different kinds of information. Test cases are great because they provide reliable, high quality information, but they're limited; nobody has enough resources to test every configuration their users are going to try out. User reports are messy, but they're documentation of real problems that your QA has to fix. The continued existence of user error reports is proof that the kind of disciplined testing represented by proper test cases and debugging procedures is inadequate. If it were enough, no problems would ever get through for users to find them.