Shuttleworth: Not convinced by rolling releases [LWN.net]

Shuttleworth: Not convinced by rolling releases

Posted Mar 8, 2013 16:26 UTC (Fri) by ebiederm (subscriber, #35028)
In reply to: Shuttleworth: Not convinced by rolling releases by lmb
Parent article: Shuttleworth: Not convinced by rolling releases

Supply me with a complete automated proof that the package meets all of it's requirements and I might be convinced that automated testing alone is sufficient. Until that time there need to be human beings in the loop who look at things in different ways from the developer giving feedback.

Fundamentally technical solutions alone are inadequate.

(Log in to post comments)

Shuttleworth: Not convinced by rolling releases

Posted Mar 8, 2013 16:37 UTC (Fri) by lmb (subscriber, #39048) [Link]

You will note that I did not include the word "automated" in my comment. That was surprisingly deliberate.

A large number of issues *can* be caught automatically. And I believe overall we're doing way too little automated testing. If you've ever looked at most projects, that is hard to deny; there are huge wins to be had here.

Clearly, not all of the issues can. Neither can everyone run all tests. That requires a distributed approach to testing: running automated tests where they can run (e.g., availability of the hardware or the license key to test compatibility), and perhaps a canary release to a beta tester group whose feedback then (automatically ;-) allows the update for wider distribution. This could (and should) include not just running the code but also source code level patch inspection.

And if the workload is running inside a VM, clone it (and its data), apply patch and see if workload still runs, feed result back.

To anticipate your point, not all analysis of these test results can be automated either. And yes, it should gather not just functional (yes/no) data but also performance, memory usage, etc. Why stop at the simple. ;-)

In short: no, centralized automated tests aren't enough. Even though it'd be amazing if we had more of that already. That does not invalidate continuous delivery.