| |||||||||||||
|
| |||||||||||||
Why people don't test development distributionsWhy people don't test development distributionsPosted Jul 6, 2009 21:51 UTC (Mon) by lmb (subscriber, #39048)Parent article: Why people don't test development distributions
Our editor's experience is part of a death spiral.
As developers know that users won't bother testing the development releases, they develop a "lax" attitude towards pre-submission and sometimes even pre-compile testing. As users know that developers, they are less willing to test; and the loop starts over again.
(From years of experience on enterprise distributions, they exhibit a bit of the same pattern - IHVs and ISVs will only bother testing the RCs, so the alpha and early beta releases are of expectable quality due to the very same mechanism.)
Further, "hey it is a beta" is used as an excuse for substandard quality, lack of regression and feature tests etcetera. Development distributions suffer from a further problem - namely, instead of just testing a single package, one gets the "best" of all test cycles rolled into one, making the overall experience even less pleasant, and even a small failure in a single component can cause a cascade of failures; or, combined, the total sum of all minor annoyances can render the system unusable.
And, of course, packagers are not always knowledgable upstream developers, but cram in updates from the upstream development cycle too - again turning the distribution not only into an integration testbed, but into a whack-a-mole playground.
In my not so humble opinion, this is a terrible misunderstanding of "agile" development and how "release early, release often" should work. And it makes me want to smack down developers with a Test-Driven-Development book.
The only way around this is to take all developers and force their own development distribution down their own throats. And to enforce better quality assurance, as in public floggings of developers who submit packages which they clearly have not run themselves at the very least.
But to say that this is making me grumpy is not quite true. I'm a reasonable man, so it makes me bloody furious.
(Log in to post comments)
Why people don't test development distributions Posted Jul 7, 2009 6:43 UTC (Tue) by malor (subscriber, #2973) [Link]
Even worse, then you get teams that notice and decry the lack of testing, and then trick users into doing it, like KDE did with 4.0. They've explicitly stated that they called it 4.0 to get more testing. They did some mental gymnastics, claiming that putting "this is unstable!" in the release notes meant it was okay, but in practically the same breath, they admit that they called it 4.0 to get testers. In other words, they weren't "lying", they were just taking advantage of people who didn't read the README. They knew that 4.0 would get tested because a .0 release has an implied level of quality and release-worthiness. In no way, shape, or form, was this quality level met by the code released at the time.
This is pure scorn of a user community... the people using KDE are the same people making the other shit that KDE depends on. Shipping them a broken desktop just screws them up and slows them down. Their time is worth just as much, if not more, than the KDE dev team's. But it's pretty apparent that the KDE people don't think that's true. Rather, their need for testers outweighs every other consideration -- so they'll fool people, and then point to the README as justification.
I have no doubt that yet another apologist will show up and say "no we didn't!", but yes you did. Anyone involved with making that decision who didn't actively campaign against it is a selfish asshole.
Why people don't test development distributions Posted Jul 7, 2009 9:16 UTC (Tue) by dholland (subscriber, #14680) [Link]
"a .0 release has an implied level of quality and release-worthiness"
I know more people who say ".0 - it'll be buggy, don't touch it" than I do people who say ".0 - new release! it'll be great! install it now!"
So maybe the implication is there - but not necessarily the one you mean.
Why people don't test development distributions Posted Jul 7, 2009 12:15 UTC (Tue) by fb (subscriber, #53265) [Link]
Seriously, can you honestly define what your friends call "buggy" wrt .0 releases?
I sincerely suspect it is something like: "there will be a number of dumb bugs no one found yet. In two or three weeks time there will be a bugfix release. Use that one".
KDE 4.0 was "buggy" more along the lines of: "It doesn't work. Features were all removed. The core libs are still in full development. Please don't mention the applications. Developers are fully aware of it. They expect to bring it to 'work' within a year's time, meaning that it will take close to 2 years (or more) for it to work again."
Why people don't test development distributions Posted Jul 16, 2009 15:34 UTC (Thu) by Wol (guest, #4433) [Link]
The *most* *important* feature of 4.0 was the API freeze.
App developers weren't writing/testing apps because the underlying libraries were a moving target. 4.0 was the release that said "this target has stopped moving, please start porting your apps".
So OF COURSE there were no apps worth speaking of on 4.0.
Cheers,
Why people don't test development distributions Posted Jul 7, 2009 12:16 UTC (Tue) by etrusco (subscriber, #4227) [Link]
Don't be silly; this is more of a Windows joke than anything else.
Why people don't test development distributions Posted Jul 7, 2009 12:14 UTC (Tue) by nix (subscriber, #2304) [Link]
What are you talking about? Modulo one unfortunate ommission (in the release notes, whoops), KDE 4.0 was always, always considered a stable *library* release, i.e. a release for *app developers*.
Now can we finally shut up about KDE 4.0? Nobody wants to hear this skeleton of a horse be flogged yet again.
Why people don't test development distributions Posted Jul 7, 2009 12:41 UTC (Tue) by malor (subscriber, #2973) [Link]
ie, if caught with pants down, redefine pants.
Why people don't test development distributions Posted Jul 7, 2009 13:37 UTC (Tue) by nix (subscriber, #2304) [Link]
My understanding is that 4.0 was meant to be a libraries-are-frozen app-developers-are-go release from long before the actual release date. It's just a shame that everyone was so used to this that they didn't think to mention it in the release notes!
'Redefine' implies post-facto. This is more like 'define'.
Why people don't test development distributions Posted Jul 7, 2009 13:20 UTC (Tue) by fb (subscriber, #53265) [Link]
Don't you think it is quite trollish to tell somebody to "shut up about KDE 4.0" during a discussion about testing development distributions?
IMHO KDE-4.0 is an text book example of software release planing and management failure. Should it be kept from being discussed **in the context of release testing and planing** because it embarrasses some? I don't think so.
Why people don't test development distributions Posted Jul 7, 2009 13:36 UTC (Tue) by nix (subscriber, #2304) [Link]
It's not because it embarrasses people. It's because it's *boring*. We've been over this dead horse until only a pile of bones is left.
Why people don't test development distributions Posted Jul 9, 2009 22:58 UTC (Thu) by johnflux (guest, #58833) [Link]
As a KDE developer - I agree. The whole KDE 4.0 fiasco was a mess.
But the developers who are embarrassed by it all just keep quiet and get to work trying to make the code better, leaving behind the noisy people who claim that KDE isn't to blame.
|
|
||||||||||||