| |||||||||||||
Goals of bug triageGoals of bug triagePosted Mar 3, 2009 18:20 UTC (Tue) by man_ls (subscriber, #15091)In reply to: Goals of bug triage by DeletedUser32991 Parent article: Ubuntu now offering mainline kernel builds I cannot believe what I am reading is indeed serious. And if you believe that the main competition in the creation of free software is about attention, that is a good strategy indeed. [...] At the same time, syncing with Debian every six months ensures that in the long term the most annoying bugs are fixed whether or not bug triaging marks them as invalid too early.To paraphrase: triaging is just a way of bonding (clueless) people to Ubuntu. Hordes of attention-seeking fanboys (as fb termed them) get to think they are doing something important, and Ubuntu gets to look merry. Meanwhile bug fixing is left to Debian, which does the hard work in the background and without getting any deserved recognition. This is exactly what people were accusing Ubuntu of, a couple of years ago; and Ubuntu engineers have fought bravely to dispel this image of merry leeches with several interesting projects. But bug management is an area where Ubuntu was supposed to provide something to the community. Something of lasting value that the community could use and benefit from. Now it seems that Launchpad is a nice playground, and triaging is not an engineering procedure but smoke and mirrors for people to play with. In my limited experience people pay good money for solid engineering, not for attention or for merry feelings (and in the latter case they already have Apple and Microsoft). I hope Ubuntu can correct its course before their revenue stream dries out.
(Log in to post comments)
Goals of bug triage Posted Mar 3, 2009 18:53 UTC (Tue) by DeletedUser32991 ((unknown), #32991) [Link] I think your paraphrase is exaggerating what I said, which was more (in the part you chose to reword) bonding is a big part of the benefit Ubuntu gets from its bug triaging and the risk associated with premature bug-closing is much reduced compared to the other big distros because Debian fixes to annoying bugs find their way into Ubuntu in time. In particular, I would prefer if you omit clueless, even in parentheses. This is largely unrelated to what Ubuntu does and does not on the engineering side. I am sure that Ubuntu also contributes fixes and packages to Debian. But what do you think that the bugs fixed for the first time shortly before and during Debian's lenny freeze mean to Ubuntu 8.04, 8.10, and 9.04? They will likely be fixed in 9.04 regardless of whether they were addressed in Ubuntu or not.
Goals of bug triage Posted Mar 3, 2009 19:54 UTC (Tue) by man_ls (subscriber, #15091) [Link] But bug triaging is not, and should never be, a means to encourage participation of non-technical (OK, not clueless) people. There are blogs, meetings, fora and technology fairs where such people can contribute. Bug triaging is a most technical occupation.As you probably know triaging originates in the medical profession, from when scarce resources can be devoted to treat huge masses of patients, either wounded in battle or victims of some epidemia. A medic first triages patients using three or four colors: green, yellow, red and black. You do not want to try to waste time resuscitating someone without a chance (black); and you do want to treat urgent patients (red) before stable ones (yellow), or even patients already cured (green). Now imagine if a well-meaning but ignorant volunteer was to do the triaging in order to save medical time. It would result in wasting time on unnecessary procedures and letting critical patients die. Somehow it would look a bit like Colin Watson's post, and that is not the proper way to contribute. Just wishing that bugs will be fixed in Debian (and then patches will somehow get into Ubuntu) is not useful; bugs fixed in Debian do not necessarily percolate to Ubuntu. Just hoping that Ubuntu contributes something ("fixes and packages") back to Debian is not useful either. There has to be a willful dedication to contribute in a technically competent manner. A fanatic dedication to quality Control in Debian is precisely what sets it apart from other distributions, and Launchpad just seems to generate noise.
Goals of bug triage Posted Mar 3, 2009 20:22 UTC (Tue) by Requiem (guest, #51519) [Link]
"Debian fixes to annoying bugs find their way into Ubuntu in time."
Debian fixes bugs sure, but Ubuntu is drawing from Unstable, once the major bugs get fixed, those packages get moved into testing, and new packages get put into stable (not right away, but the point stands), so Ubuntu isn't seeing all the bugs Debian fixes at the testing--> stable step, and misses half the bug fixes in the Unstable --> testing step.
Goals of bug triage Posted Mar 3, 2009 22:05 UTC (Tue) by gevaerts (subscriber, #21521) [Link]
I think you totally missed how debian works
Debian branches Posted Mar 4, 2009 10:40 UTC (Wed) by man_ls (subscriber, #15091) [Link] I think that what gevaerts means is that Debian versions do not work that way. I also used to think that bugs are corrected in the transitions from unstable to stable, but after some research found otherwise. For clarity let's call unstable, testing and stable 'branches', and keep 'version' for packages.Versions are entered into the unstable branch, and after a suitable period they go to testing. They stay there as long as the current version is in testing, or until a new version comes along. Every 18-36 months a new branch (including versions of all packages) becomes stable. But if a problem is found in a package, a new version of the package that corrects that problem comes again through unstable, and eventually promotes again to testing (or not if further problems are found). No changes are made between unstable and testing; patches are always applied before entering unstable. If Ubuntu draws from the unstable branch they will eventually gather all fixes. Of course it can take 18 to 36 months to get a new version with those fixes, so with a lifecycle of 6 months most of the time Ubuntu is on its own.
Goals of bug triage Posted Mar 3, 2009 20:28 UTC (Tue) by michaeljt (subscriber, #39183) [Link]
> triaging is just a way of bonding (clueless) people to Ubuntu. Hordes of attention-seeking fanboys (as fb termed them) get to think they are doing something important, and Ubuntu gets to look merry.
Is it such a bad thing if Ubuntu is aiming to be accessible outside of the usual Linux audience, and trying to help those people contribute something back? I think that at least one of the things that "bonds" the more traditional audience to Linux is the feeling that they are "part of it" and one of the myriad small forces driving its evolution. Why should less technical people not be able to feel the same? It would be nice though if Canonical found more effective ways of getting them involved in the bug process, like say more reproducing and less marking as invalid. I'm sure that they could be a massive benefit to the FOSS community.
Goals of bug triage Posted Mar 3, 2009 21:20 UTC (Tue) by jspaleta (subscriber, #50639) [Link]
Indeed.. being able to feel like you are making an impact drives interest. Ubuntu's processes do a very good job of that..driving interest. Its very easy to feel like you are making an impact, especially when your leadership is telling you every step of the way that what you are doing is a positive action which is helping.
But is it helping? I don't think I've seen a metric from a developers standpoint that indicates that triage in Launchpad is helping them do their work more efficiently. Do Ubuntu developers feel that the triage is helpful? Do Debian developer find it helpful? Do upstream projects find it helpful? No one wants to be told that what they are doing isn't making a positive impact, but at the same time you don't want a large pool of people making more work developers just for the sake of feeling involved.
Perception is not necessarily reality. The trendable metrics I've seen coming out of Launchpad focus on the number of open and closed bugs. And that's fine. If the most important thing is keeping the rate of growth of open bugs down that is a perfectly fine metric. But it is not a metric which directly tracks impact on developer efficiency or impact on the rate of valid bugs fixed. Are developers fixing valid bugs faster when there is more triage involvement? Do triagers help or hurt the efficiency of developers to address valid bugs and to generate valid fixes with patchsets? I don't know. The available metrics don't try to answer that question.
All I know for sure is that the Ubuntu triagers are doing a very good job of closing bugs and keeping the open bug count growth way down. But how much of that are inappropriate closures? <1%? >10%? I don't really know. I don't expect them to be perfect and if the triagers are only doing 1% inappropriate tagging that's very very good. I think they are making an honest effort to make a positive impact. I don't think people would spend their time and encourage other people to spend their time on this sort of thing unless they really felt that it was a positive impact. But sometimes its worth taking a moment and being self-critical and to question your own assumptions in an effort to self-assess. Hopefully Colin's voicing of frustrations will provide the opportunity for that sort of moment. We don't really know how widespread the problem is that Colin chose to blog about. Is it systemic? Or is it a few overzealous people who just need to be re-trained? Is it primarily new people with low karma or is it veterans with high karma? We don't know.
My understanding Launchpad's karma system doesn't have a concept of negative karma. Every action you take scores you positive karma, but I don't think there is any effort in place to track whether your action is making more work for people or is helping people work more efficiently. Negative karma for mistakes would help gain a better view of how widespread the problem is. If its systemic, you'll see a lot of negative karma build up and you could then adjust the process with a sequence of small corrections and try to reduce the negative karma build up. If its just a few people, you'll be able to see and address that with re-training as needed on an individual basis without calling them out publicly. If its primarily new people, you can adjust processes which stress more mentoring. If its veterans...you have a deep cultural problem which needs to be addressed through some triager/developer cross communication and try to reorient and understanding of workflow needs across the groups.
-jef
Goals of bug triage Posted Mar 3, 2009 21:37 UTC (Tue) by michaeljt (subscriber, #39183) [Link]
That negative karma thing sounds like a good idea in one way, but a very dangerous one in another. Perhaps something like that should be done "in private" by Ubuntu, with negative karma something entirely separate from positive, and which only Canonical employees are allowed to assign and view (that is, that the people with the negative karma and all other non-employees are not even aware of it). That way, they would get some idea of where things are going wrong, but without upsetting their user base.
Who knows, perhaps they are already doing this? :)
Goals of bug triage Posted Mar 3, 2009 21:53 UTC (Tue) by jspaleta (subscriber, #50639) [Link]
I would agree that negative karma could be used for "evil" if it were made widely available on an account by account basis. Aggregate information could be made publicly available however. But tracking wrong action can be as important as right action..as long as you are prepared to use that information constructively to inform your process adjustments. I would shy away from making it a Canonical-only set of information and more of a leadership sensitive information for both external and corporately sponsored leadership.
What you don't want is bad practises going on for so long without a way to address them via normal process feedback resulting in developers like Colin feel like they need to make a public statement about the problems. I doubt he enjoyed putting people on the spot, but if triagers are making more work for him and other developers..that's a real problem. The developers are the critical resource, everything else needs to be tuned to help them be more efficient. The sort of statement Colin made is a red flag moment indicating the the processes aren't working. It draws attention to the problem but it has too little detail to know the scope of the problem.
The first step in addressing any problem is being aware of a problem. Negative karma would just be one way to seeing human process problems develop in a systematic and unbiased way. If you think its worth discussing more, feel free to write it up in Brainstorm and see if anyone takes notice. I'll even let you take credit for the idea.
-jef
|
|||||||||||||