| |||||||||||||
Goals of bug triageGoals of bug triagePosted Mar 3, 2009 10:50 UTC (Tue) by DeletedUser32991 ((unknown), #32991)In reply to: Ubuntu now offering mainline kernel builds by fb Parent article: Ubuntu now offering mainline kernel builds
However, having people triage bugs does create a bond to Ubuntu. For better or worse, that is at least half of everything done in Ubuntu. People like Jono make sure that your hear about how great Ubuntu is at least once a week, and if you look at events like global bug jam, the largest item in the three point instructions is "make some noise", moving on to the success story, you again see that reporting on the bugs fixed is the least important part. And if you believe that the main competition in the creation of free software is about attention, that is a good strategy indeed. Since part of Canonical's revenue probably is off stuff based on Ubuntu, hiring people that can generate that much attention for Ubuntu may be one of their best investments for both Ubuntu and themselves, even more important than any single of the (excellent, at least those that I know) technical people. At the same time, syncing with Debian every six months ensures that in the long term the most annoying bugs are fixed whether or not bug triaging marks them as invalid too early. Of course, if everyone deployed that strategy, people would get spammed in "look at how great we are" everywhere, but in the current environment where people usually blog about technical things or rant about how they not like someone else's stuff, Ubuntu gets to look like an island of happiness. Mind you, probably a good portion of the bug triage also results in the correct resolution.
(Log in to post comments)
Goals of bug triage Posted Mar 3, 2009 13:23 UTC (Tue) by fb (subscriber, #53265) [Link]
You seem to imply that one of the main purposes of having launchpad tracking bugs is to generate buzz. I don't know if that is a Ubuntu/Cannonical strategy. If it is, I think it is a huge mistake.
The goal of a bug tracking system is to track bugs, and not to generate buzz. Well written bug reports, and wish-list requests are being thrown away by people who haven't got a clue.
A good -working out of the box- distribution generates buzz. A handful of clicking fanboys happy because they clicked away at this many bug cases, leads to happy fanboys, and frustrated users. With the added bonus that people with the expertise to diagnose problems get frustrated, and cease from participating in the process.
Goals of bug triage Posted Mar 3, 2009 14:37 UTC (Tue) by nye (guest, #51576) [Link]
Note: As I originally composed this post it was far more harshly worded. I've changed that because I felt it was unconstructive, but what remains falls short of indicating the ire this inspires in me.
So far as I'm aware the purpose of Launchpad is indeed to generate buzz. Certainly if it's intended to be used to fix bugs then it's a total failure.
The biggest reason I'm not using Ubuntu any more is not because of the numerous major bugs that slip into every release (though they do give a bad impression about the quality of Linux when Ubuntu is all most people have heard of), nor is it the increasing focus on users so different to me that helping them requires me to fight with the system more with each release (though that is increasingly frustrating), but it's because the way they handle bugs is so atrocious that I can only infer that anyone involved in Ubuntu QA doesn't care about what they're producing. If it turns out that this is in fact because of a load of hangers-on 'helping' despite not being competent to do so, then that really needs to be stopped.
Goals of bug triage Posted Mar 3, 2009 15:18 UTC (Tue) by michaeljt (subscriber, #39183) [Link]
I have to say that (personal feeling only) they are improving. There was a point where I was close to dropping Ubuntu. Now I notice that many of the most annoying bugs which make it into releases are at least getting fixed within the first two to three months (previously they were simply never fixed). A small improvement, but enough for me to wait a bit to see what else improves. As mentioned above, if they were to move to vanilla kernels entirely instead of brewing their own thing (in fact if they were to brew their own thing a bit less generally - apparently they are finally dropping usplash and will use Fedora's one instead) they might also have more resources free to squash bugs. Hopefully this article indicates a step in that direction.
Goals of bug triage Posted Mar 3, 2009 16:03 UTC (Tue) by fb (subscriber, #53265) [Link]
Let me give you some anecdotal evidence of the contrary. Dell selling laptops with Ubuntu, the "Tier-1 hardware vendor selling Linux from 'well known distribution'" poster child.
The brightness keys on Dells have stopped working properly for a year already (https://bugs.launchpad.net/bugs/207473). That bug has a bunch of "invalid" duplicates, therefore the large number of annotated fixes.
At the end of the day, Dell laptops shipped with Ubuntu still don't have a fix for this year old regression.
[...]
Anecdotal for sure, but 'Dell with Ubuntu' is still their poster child success story.
Goals of bug triage Posted Mar 3, 2009 17:55 UTC (Tue) by DeletedUser32991 ((unknown), #32991) [Link] I'd say that generating buzz with everything they do is an integral part of the Ubuntu success, and ultimately I think that this is based on the strategy (after all, Canonical's Ubuntu community managers - or whatever the job title of Jono, Daniel, Jorge is - play an integral role in that). And really, in terms of turning random users into helpers of one sort or another - be it helping users on forums, be it as guerrilla marketers, ... - they are a highly successful. They have so many of highly spirited, well-meaning people wanting to talk about how they love Ubuntu and help other people that they can drown a fair amount of criticism in positivity. Now, I specifically do not want to say that they are doing bug triage for the purpose of giving non-technical people a task in order to increase the bond with Ubuntu, but I do positively believe that they view every activity that people can participate in as an opportunity to do the bonding as well. Mind you, this plasters over a lot of things that were considered problematic elsewhere: Ubuntu (the distro) is split into main and universe (reminds me of Fedora Core and Community(?) before they merged): only 50-some people being in Ubuntu-cored-dev (uploading to main+universe) and an additional 70 uploading to universe, i.e. only a tiny fraction of their community is empowered to actually upload packages. Yet I have heard few complaints about people feeling disempowered in the Ubuntu project. So I think "generating buzz" really is the very key to Ubuntu's strategy and success. Thus, looking at every activity - in addition to the immediate objective - helps a lot at understanding the mechanics of Ubuntu. Of course, I am probably the second most biased person on earth to discuss fixing bugs and the role of a community in that, so take it with a grain of salt.
Goals of bug triage Posted Mar 3, 2009 18:20 UTC (Tue) by man_ls (subscriber, #15091) [Link] I cannot believe what I am reading is indeed serious.And if you believe that the main competition in the creation of free software is about attention, that is a good strategy indeed. [...] At the same time, syncing with Debian every six months ensures that in the long term the most annoying bugs are fixed whether or not bug triaging marks them as invalid too early.To paraphrase: triaging is just a way of bonding (clueless) people to Ubuntu. Hordes of attention-seeking fanboys (as fb termed them) get to think they are doing something important, and Ubuntu gets to look merry. Meanwhile bug fixing is left to Debian, which does the hard work in the background and without getting any deserved recognition. This is exactly what people were accusing Ubuntu of, a couple of years ago; and Ubuntu engineers have fought bravely to dispel this image of merry leeches with several interesting projects. But bug management is an area where Ubuntu was supposed to provide something to the community. Something of lasting value that the community could use and benefit from. Now it seems that Launchpad is a nice playground, and triaging is not an engineering procedure but smoke and mirrors for people to play with. In my limited experience people pay good money for solid engineering, not for attention or for merry feelings (and in the latter case they already have Apple and Microsoft). I hope Ubuntu can correct its course before their revenue stream dries out.
Goals of bug triage Posted Mar 3, 2009 18:53 UTC (Tue) by DeletedUser32991 ((unknown), #32991) [Link] I think your paraphrase is exaggerating what I said, which was more (in the part you chose to reword) bonding is a big part of the benefit Ubuntu gets from its bug triaging and the risk associated with premature bug-closing is much reduced compared to the other big distros because Debian fixes to annoying bugs find their way into Ubuntu in time. In particular, I would prefer if you omit clueless, even in parentheses. This is largely unrelated to what Ubuntu does and does not on the engineering side. I am sure that Ubuntu also contributes fixes and packages to Debian. But what do you think that the bugs fixed for the first time shortly before and during Debian's lenny freeze mean to Ubuntu 8.04, 8.10, and 9.04? They will likely be fixed in 9.04 regardless of whether they were addressed in Ubuntu or not.
Goals of bug triage Posted Mar 3, 2009 19:54 UTC (Tue) by man_ls (subscriber, #15091) [Link] But bug triaging is not, and should never be, a means to encourage participation of non-technical (OK, not clueless) people. There are blogs, meetings, fora and technology fairs where such people can contribute. Bug triaging is a most technical occupation.As you probably know triaging originates in the medical profession, from when scarce resources can be devoted to treat huge masses of patients, either wounded in battle or victims of some epidemia. A medic first triages patients using three or four colors: green, yellow, red and black. You do not want to try to waste time resuscitating someone without a chance (black); and you do want to treat urgent patients (red) before stable ones (yellow), or even patients already cured (green). Now imagine if a well-meaning but ignorant volunteer was to do the triaging in order to save medical time. It would result in wasting time on unnecessary procedures and letting critical patients die. Somehow it would look a bit like Colin Watson's post, and that is not the proper way to contribute. Just wishing that bugs will be fixed in Debian (and then patches will somehow get into Ubuntu) is not useful; bugs fixed in Debian do not necessarily percolate to Ubuntu. Just hoping that Ubuntu contributes something ("fixes and packages") back to Debian is not useful either. There has to be a willful dedication to contribute in a technically competent manner. A fanatic dedication to quality Control in Debian is precisely what sets it apart from other distributions, and Launchpad just seems to generate noise.
Goals of bug triage Posted Mar 3, 2009 20:22 UTC (Tue) by Requiem (guest, #51519) [Link]
"Debian fixes to annoying bugs find their way into Ubuntu in time."
Debian fixes bugs sure, but Ubuntu is drawing from Unstable, once the major bugs get fixed, those packages get moved into testing, and new packages get put into stable (not right away, but the point stands), so Ubuntu isn't seeing all the bugs Debian fixes at the testing--> stable step, and misses half the bug fixes in the Unstable --> testing step.
Goals of bug triage Posted Mar 3, 2009 22:05 UTC (Tue) by gevaerts (subscriber, #21521) [Link]
I think you totally missed how debian works
Debian branches Posted Mar 4, 2009 10:40 UTC (Wed) by man_ls (subscriber, #15091) [Link] I think that what gevaerts means is that Debian versions do not work that way. I also used to think that bugs are corrected in the transitions from unstable to stable, but after some research found otherwise. For clarity let's call unstable, testing and stable 'branches', and keep 'version' for packages.Versions are entered into the unstable branch, and after a suitable period they go to testing. They stay there as long as the current version is in testing, or until a new version comes along. Every 18-36 months a new branch (including versions of all packages) becomes stable. But if a problem is found in a package, a new version of the package that corrects that problem comes again through unstable, and eventually promotes again to testing (or not if further problems are found). No changes are made between unstable and testing; patches are always applied before entering unstable. If Ubuntu draws from the unstable branch they will eventually gather all fixes. Of course it can take 18 to 36 months to get a new version with those fixes, so with a lifecycle of 6 months most of the time Ubuntu is on its own.
Goals of bug triage Posted Mar 3, 2009 20:28 UTC (Tue) by michaeljt (subscriber, #39183) [Link]
> triaging is just a way of bonding (clueless) people to Ubuntu. Hordes of attention-seeking fanboys (as fb termed them) get to think they are doing something important, and Ubuntu gets to look merry.
Is it such a bad thing if Ubuntu is aiming to be accessible outside of the usual Linux audience, and trying to help those people contribute something back? I think that at least one of the things that "bonds" the more traditional audience to Linux is the feeling that they are "part of it" and one of the myriad small forces driving its evolution. Why should less technical people not be able to feel the same? It would be nice though if Canonical found more effective ways of getting them involved in the bug process, like say more reproducing and less marking as invalid. I'm sure that they could be a massive benefit to the FOSS community.
Goals of bug triage Posted Mar 3, 2009 21:20 UTC (Tue) by jspaleta (subscriber, #50639) [Link]
Indeed.. being able to feel like you are making an impact drives interest. Ubuntu's processes do a very good job of that..driving interest. Its very easy to feel like you are making an impact, especially when your leadership is telling you every step of the way that what you are doing is a positive action which is helping.
But is it helping? I don't think I've seen a metric from a developers standpoint that indicates that triage in Launchpad is helping them do their work more efficiently. Do Ubuntu developers feel that the triage is helpful? Do Debian developer find it helpful? Do upstream projects find it helpful? No one wants to be told that what they are doing isn't making a positive impact, but at the same time you don't want a large pool of people making more work developers just for the sake of feeling involved.
Perception is not necessarily reality. The trendable metrics I've seen coming out of Launchpad focus on the number of open and closed bugs. And that's fine. If the most important thing is keeping the rate of growth of open bugs down that is a perfectly fine metric. But it is not a metric which directly tracks impact on developer efficiency or impact on the rate of valid bugs fixed. Are developers fixing valid bugs faster when there is more triage involvement? Do triagers help or hurt the efficiency of developers to address valid bugs and to generate valid fixes with patchsets? I don't know. The available metrics don't try to answer that question.
All I know for sure is that the Ubuntu triagers are doing a very good job of closing bugs and keeping the open bug count growth way down. But how much of that are inappropriate closures? <1%? >10%? I don't really know. I don't expect them to be perfect and if the triagers are only doing 1% inappropriate tagging that's very very good. I think they are making an honest effort to make a positive impact. I don't think people would spend their time and encourage other people to spend their time on this sort of thing unless they really felt that it was a positive impact. But sometimes its worth taking a moment and being self-critical and to question your own assumptions in an effort to self-assess. Hopefully Colin's voicing of frustrations will provide the opportunity for that sort of moment. We don't really know how widespread the problem is that Colin chose to blog about. Is it systemic? Or is it a few overzealous people who just need to be re-trained? Is it primarily new people with low karma or is it veterans with high karma? We don't know.
My understanding Launchpad's karma system doesn't have a concept of negative karma. Every action you take scores you positive karma, but I don't think there is any effort in place to track whether your action is making more work for people or is helping people work more efficiently. Negative karma for mistakes would help gain a better view of how widespread the problem is. If its systemic, you'll see a lot of negative karma build up and you could then adjust the process with a sequence of small corrections and try to reduce the negative karma build up. If its just a few people, you'll be able to see and address that with re-training as needed on an individual basis without calling them out publicly. If its primarily new people, you can adjust processes which stress more mentoring. If its veterans...you have a deep cultural problem which needs to be addressed through some triager/developer cross communication and try to reorient and understanding of workflow needs across the groups.
-jef
Goals of bug triage Posted Mar 3, 2009 21:37 UTC (Tue) by michaeljt (subscriber, #39183) [Link]
That negative karma thing sounds like a good idea in one way, but a very dangerous one in another. Perhaps something like that should be done "in private" by Ubuntu, with negative karma something entirely separate from positive, and which only Canonical employees are allowed to assign and view (that is, that the people with the negative karma and all other non-employees are not even aware of it). That way, they would get some idea of where things are going wrong, but without upsetting their user base.
Who knows, perhaps they are already doing this? :)
Goals of bug triage Posted Mar 3, 2009 21:53 UTC (Tue) by jspaleta (subscriber, #50639) [Link]
I would agree that negative karma could be used for "evil" if it were made widely available on an account by account basis. Aggregate information could be made publicly available however. But tracking wrong action can be as important as right action..as long as you are prepared to use that information constructively to inform your process adjustments. I would shy away from making it a Canonical-only set of information and more of a leadership sensitive information for both external and corporately sponsored leadership.
What you don't want is bad practises going on for so long without a way to address them via normal process feedback resulting in developers like Colin feel like they need to make a public statement about the problems. I doubt he enjoyed putting people on the spot, but if triagers are making more work for him and other developers..that's a real problem. The developers are the critical resource, everything else needs to be tuned to help them be more efficient. The sort of statement Colin made is a red flag moment indicating the the processes aren't working. It draws attention to the problem but it has too little detail to know the scope of the problem.
The first step in addressing any problem is being aware of a problem. Negative karma would just be one way to seeing human process problems develop in a systematic and unbiased way. If you think its worth discussing more, feel free to write it up in Brainstorm and see if anyone takes notice. I'll even let you take credit for the idea.
-jef
|
|||||||||||||