User: Password:
Subscribe / Log in / New account

Rolling testing

Rolling testing

Posted Mar 8, 2013 12:37 UTC (Fri) by ibukanov (subscriber, #3942)
In reply to: Rolling testing by man_ls
Parent article: Shuttleworth: Not convinced by rolling releases

> Automated testing is not impossible, it is just harder.

How one can automatically test that, for example, on a random laptop sound works or the video has proper white balance or the laptop can pair with a random bluetooth device in various supported bluetooth applications? Or one can install on a server with some strange BIOS quirks?

In those cases it is best to ask the user. But for that one needs easy to use infrastructure to try new things without breaking stuff that currently works. Plus one has to listen to the feedback. Without these things one simply would not get enough test data for the rapid release.

> Debian testing is a good example of how to do things.

Debian testing works if one releases every couple of years at best. For rapid releases one has to do much better.

(Log in to post comments)

Rolling testing

Posted Mar 8, 2013 12:49 UTC (Fri) by man_ls (guest, #15091) [Link]

You cannot test sound on a random laptop, obviously; but you can check that there is a sound module loaded at boot on a certain set of laptops, not too much problem. Also, for this particular problem Ubuntu is using an industry-standard, well tested kernel for a reason.

For the rest of the distro a couple of machines would be enough to test that packages are downloaded and installed, and then that apps start correctly. Getting detailed info about every bug in a package is quite harder, but that would be an area where a distro has to cooperate with upstream.

Finally, manual testing à la Debian is great. I don't mean "Debian testing" as a concept, but the Debian "testing" distro (right now frozen): packages are uploaded to unstable, then migrate to testing after 10 days if there are no show-stoppers (where the manual part comes in). For me Debian testing it has worked as a rolling distro for the last 10 years.

Rolling testing

Posted Mar 8, 2013 23:02 UTC (Fri) by ibukanov (subscriber, #3942) [Link]

> but you can check that there is a sound module loaded at boot on a certain set of laptops, not too much problem.

How would one pick up laptops? And what about older hardware that may be still popular but it hard to get?

So I just do not see how one can skip user during testing. Surely with unlimited amount of time one can try to solve that problem, but with a more realistic budget spending efforts to simplify testing by users would be more fruitful.

Rolling testing

Posted Mar 8, 2013 23:17 UTC (Fri) by man_ls (guest, #15091) [Link]

Both automated and end-user testing are needed, always. Anyway hardware is the province of the kernel, and they are quite efficient solving this kind of bug. I doubt that Ubuntu patches many sound card bugs nowadays in their kernels.

Rolling testing

Posted Mar 8, 2013 23:43 UTC (Fri) by ibukanov (subscriber, #3942) [Link]

> Anyway hardware is the province of the kernel

I wish that would be true. ALSA has many (way too many IMO) knobs that the user space can influence and both Fedora and Ubuntu touch them. Or consider BlueTooth. The kernel support is rather generic it is up to the application framework to deal with various devices. Unfortunately the result is that one may have two devices that require different Linux distros to get them working...

Copyright © 2017, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds