| |||||||||||||
|
| |||||||||||||
Why people don't test development distributionsWhy people don't test development distributionsPosted Jul 6, 2009 23:12 UTC (Mon) by sjlyall (subscriber, #4151)Parent article: Why people don't test development distributions
Is automated build images a possible answer?
I know automation won't pick up everything but I would have thought the prelink and the gdm bugs mentioned would have been picked up by even simple "apply the update and reboot" testing.
To tell the truth I thought most distributions already did this.
(Log in to post comments)
Preling but not gdm Posted Jul 7, 2009 0:16 UTC (Tue) by khim (subscriber, #9252) [Link] I know automation won't pick up everything but I would have thought the prelink and the gdm bugs mentioned would have been picked up by even simple "apply the update and reboot" testing. Prelink will certainly be detected, but gdm... no luck: bug there killed your session when you tried to update, offline upgrade worked fine (it just killed the session of the unfortunate user who tried to use the system at the time). The real solution is shown by Gentoo: package is added to the system in "masked" state. It's possible to use it - but you need to specifically ask for it. Once enough "success stories" are obtained the mask is removed and all "unstable" users are upgraded. Works good so far: I certainly never seen unstable Gentoo system which refused to even boot! P.S. Number of success reports differes for type of package: for core packages (like baselayout, prelink, or glibc) it can be "few months of testing", but for fringe package like tofrodos it can be "one user other then the packager"...
Preling but not gdm Posted Jul 7, 2009 14:49 UTC (Tue) by michaeljt (subscriber, #39183) [Link]
> The real solution is shown by Gentoo: package is added to the system in "masked" state. It's possible to use it - but you need to specifically ask for it.
You echo my thoughts. I'm sure many users would be happy (even eager) to test certain unstable packages that they are interested in. Compare that to the number who would be happy to work with an unstable distribution (I for one would be very reluctant to). The only thing that Gentoo is lacking in this respect is a way to pull in the dependencies of the unstable packages you wish to test without having to remove the stable versions that your other packages are using.
Preling but not gdm Posted Jul 7, 2009 19:05 UTC (Tue) by cry_regarder (subscriber, #50545) [Link]
Fedora already has this. It is called bodhi. See:
http://bodhi.fedoraproject.org
But the discussion is about development __distributions__ not development __packages__.
Preling but not gdm Posted Jul 9, 2009 19:40 UTC (Thu) by yokem_55 (subscriber, #10498) [Link]
Gentoo, while it is rare for a stable or even unstable update to completely hose a system, a couple of years ago they pushed through an upgrade to expat which bumped the .so version number, which in turn hosed everything depending on expat. On top of this, the usual tool for fixing missing libraries (revdep-rebuild) had some nasty weaknesses that made fixing one's system decidedly non-trivial. I personally had no idea what expat was or how much of a system depended on it (nearly everything that ran in X to start). There are still people running into this as the support thread for the issue in the Gentoo forums shows.
Preling but not gdm Posted Jul 12, 2009 7:56 UTC (Sun) by dirtyepic (subscriber, #30178) [Link]
> The real solution is shown by Gentoo: package is added to the system in
> "masked" state. It's possible to use it - but you need to specifically > ask for it. Once enough "success stories" are obtained the mask is > removed and all "unstable" users are upgraded.
We do? ;) There are only a couple situations I can think of where a new package/version would be added to the tree in a masked state:
- you know it's going to break things and you need guin- er, testers to smoke out the worst of the bugs (eg. gcc-4.4)
...unless by masking you're actually referring to keywording (~arch / arch). Then yes, packages need to be in ~arch (unstable) a minimum of one month before they can be stabilized. We don't need success stories, just a lack of bug reports. Stabilization is done by dedicated arch-tester teams, so you will always have at least one other set of eyes on a package before it hits stable.
IMO Gentoo generally manages to keep the unstable tree in working order and usable enough for everyday work. I think this can be attributed in part to the fact that we don't do releases and therefore don't have that period of churn that conventional distros do where everything is getting updated at once. Basically, we don't have a development cycle; we're always in development.
Why people don't test development distributions Posted Jul 7, 2009 0:33 UTC (Tue) by nirik (subscriber, #71) [Link]
The prelink issue only would have been picked up if it was installed and the regular nightly
cron job was run... granted a automated test could have run that manually, but it's not normal right after an update.
Why people don't test development distributions Posted Jul 7, 2009 8:30 UTC (Tue) by lmb (subscriber, #39048) [Link]
One could, with reasonable strength of argument, come to the conclusion that software that cannot be automatically tested well is badly designed.
(The counter argument that there are some things that are extremely difficult to test - like usability - does not diminish the assessment that very few software packages get anywhere close an acceptable test coverage.)
Why people don't test development distributions Posted Jul 7, 2009 12:48 UTC (Tue) by MathFox (guest, #6104) [Link]
It would help a lot if the infrastructural packages came with self-tests that are routinely run before a new version is shipped. (Yes, writing tests takes time; catching the bugs timely saves a lot of time.) For a lot of software it is not-economical to write a 100% coverage automatic test (games!) or the hardware is not available. However that should not stop you from testing the 80-90% that can be tested.
Why people don't test development distributions Posted Jul 7, 2009 19:32 UTC (Tue) by Max.Hyre (subscriber, #1054) [Link] It makes a big difference to actually enforce testability. You get your nose rubbed in how much better things work.In one wonderful (in the sense of getting to follow all those ``best practices'' you read about in books) job, we had automated testing, and were required to build in test points. Which were then evaluated in the code reviews. Of course, we were building medical devices, and you tend to be a bit more careful when the FDA is looking over your shoulder. :-) I suspect we were within shouting distance of the 80–90 % number. But that's only 80–90 % number. If that's what we could do under duress, it's hardly surprising we're lucky to get maybe 30 % in the real world. It's human nature (especially programmer nature) to believe both
Typo Posted Jul 7, 2009 19:39 UTC (Tue) by Max.Hyre (subscriber, #1054) [Link] `Predilections'. So much for level of testing.
Why people don't test development distributions Posted Jul 8, 2009 11:05 UTC (Wed) by nix (subscriber, #2304) [Link]
prelink *has* self-tests, but perhaps this is a problem that only shows up if ld-linux.so.2 itself is prelinked, in which case you wouldn't see it unless you prelinked the whole system.
(I'm running eglibc 2.10-head and prelink and see no trouble, though. Local RH patch breaking something? What to?)
|
|
||||||||||||