Not logged in
Log in now
Create an account
Subscribe to LWN
Pencil, Pencil, and Pencil
Dividing the Linux desktop
LWN.net Weekly Edition for June 13, 2013
A report from pgCon 2013
Little things that matter in language design
I think you underestimate the attention-deficit of Ubuntu bug traigers.
Ubuntu now offering mainline kernel builds
Posted Mar 3, 2009 9:58 UTC (Tue) by fb (subscriber, #53265)
I don't know who was the genius that decided to give non-programmers this kind of autonomy. But it sure was a bad idea.
There are 3 things that bug triagers do in Ubuntu:
1. Mark similar but fundamentally different bugs as duplicates. It does not matter if you explicitly say the bugs are not duplicates. It does not matter if you explain the bugs are not the same. They don't read the bug thread, they don't investigate the bug. They mark it "duplicate".
2. No matter how much information you enter, there will be "Ubuntu bug triager" that will come to your bug to mark it incomplete. The amount of detail or information, really does not matter. Just mark it "incomplete".
3. Once a next release comes. Someone will come and ask if the bug is still present, and if can be fixed. No matter that all steps to reproduce the bug are still be there. They won't check, they won't read. They just copy and paste some standard text into the bug report, wishing to mark the whole bug as invalid, if you don't answer them in time.
Goals of bug triage
Posted Mar 3, 2009 10:50 UTC (Tue) by DeletedUser32991 ((unknown), #32991)
However, having people triage bugs does create a bond to Ubuntu. For better or worse, that is at least half of everything done in Ubuntu. People like Jono make sure that your hear about how great Ubuntu is at least once a week, and if you look at events like global bug jam, the largest item in the three point instructions is "make some noise", moving on to the success story, you again see that reporting on the bugs fixed is the least important part. And if you believe that the main competition in the creation of free software is about attention, that is a good strategy indeed. Since part of Canonical's revenue probably is off stuff based on Ubuntu, hiring people that can generate that much attention for Ubuntu may be one of their best investments for both Ubuntu and themselves, even more important than any single of the (excellent, at least those that I know) technical people. At the same time, syncing with Debian every six months ensures that in the long term the most annoying bugs are fixed whether or not bug triaging marks them as invalid too early. Of course, if everyone deployed that strategy, people would get spammed in "look at how great we are" everywhere, but in the current environment where people usually blog about technical things or rant about how they not like someone else's stuff, Ubuntu gets to look like an island of happiness.
Mind you, probably a good portion of the bug triage also results in the correct resolution.
Posted Mar 3, 2009 13:23 UTC (Tue) by fb (subscriber, #53265)
The goal of a bug tracking system is to track bugs, and not to generate buzz. Well written bug reports, and wish-list requests are being thrown away by people who haven't got a clue.
A good -working out of the box- distribution generates buzz. A handful of clicking fanboys happy because they clicked away at this many bug cases, leads to happy fanboys, and frustrated users. With the added bonus that people with the expertise to diagnose problems get frustrated, and cease from participating in the process.
Posted Mar 3, 2009 14:37 UTC (Tue) by nye (guest, #51576)
So far as I'm aware the purpose of Launchpad is indeed to generate buzz. Certainly if it's intended to be used to fix bugs then it's a total failure.
The biggest reason I'm not using Ubuntu any more is not because of the numerous major bugs that slip into every release (though they do give a bad impression about the quality of Linux when Ubuntu is all most people have heard of), nor is it the increasing focus on users so different to me that helping them requires me to fight with the system more with each release (though that is increasingly frustrating), but it's because the way they handle bugs is so atrocious that I can only infer that anyone involved in Ubuntu QA doesn't care about what they're producing. If it turns out that this is in fact because of a load of hangers-on 'helping' despite not being competent to do so, then that really needs to be stopped.
Posted Mar 3, 2009 15:18 UTC (Tue) by michaeljt (subscriber, #39183)
Posted Mar 3, 2009 16:03 UTC (Tue) by fb (subscriber, #53265)
The brightness keys on Dells have stopped working properly for a year already (https://bugs.launchpad.net/bugs/207473). That bug has a bunch of "invalid" duplicates, therefore the large number of annotated fixes.
At the end of the day, Dell laptops shipped with Ubuntu still don't have a fix for this year old regression.
Anecdotal for sure, but 'Dell with Ubuntu' is still their poster child success story.
Posted Mar 3, 2009 17:55 UTC (Tue) by DeletedUser32991 ((unknown), #32991)
I'd say that generating buzz with everything they do is an integral part of the Ubuntu success, and ultimately I think that this is based on the strategy (after all, Canonical's Ubuntu community managers - or whatever the job title of Jono, Daniel, Jorge is - play an integral role in that).
And really, in terms of turning random users into helpers of one sort or another - be it helping users on forums, be it as guerrilla marketers, ... - they are a highly successful. They have so many of highly spirited, well-meaning people wanting to talk about how they love Ubuntu and help other people that they can drown a fair amount of criticism in positivity.
Now, I specifically do not want to say that they are doing bug triage for the purpose of giving non-technical people a task in order to increase the bond with Ubuntu, but I do positively believe that they view every activity that people can participate in as an opportunity to do the bonding as well.
Mind you, this plasters over a lot of things that were considered problematic elsewhere: Ubuntu (the distro) is split into main and universe (reminds me of Fedora Core and Community(?) before they merged): only 50-some people being in Ubuntu-cored-dev (uploading to main+universe) and an additional 70 uploading to universe, i.e. only a tiny fraction of their community is empowered to actually upload packages. Yet I have heard few complaints about people feeling disempowered in the Ubuntu project.
So I think "generating buzz" really is the very key to Ubuntu's strategy and success. Thus, looking at every activity - in addition to the immediate objective - helps a lot at understanding the mechanics of Ubuntu.
Of course, I am probably the second most biased person on earth to discuss fixing bugs and the role of a community in that, so take it with a grain of salt.
Posted Mar 3, 2009 18:20 UTC (Tue) by man_ls (subscriber, #15091)
And if you believe that the main competition in the creation of free software is about attention, that is a good strategy indeed. [...] At the same time, syncing with Debian every six months ensures that in the long term the most annoying bugs are fixed whether or not bug triaging marks them as invalid too early.
This is exactly what people were accusing Ubuntu of, a couple of years ago; and Ubuntu engineers have fought bravely to dispel this image of merry leeches with several interesting projects. But bug management is an area where Ubuntu was supposed to provide something to the community. Something of lasting value that the community could use and benefit from. Now it seems that Launchpad is a nice playground, and triaging is not an engineering procedure but smoke and mirrors for people to play with.
In my limited experience people pay good money for solid engineering, not for attention or for merry feelings (and in the latter case they already have Apple and Microsoft). I hope Ubuntu can correct its course before their revenue stream dries out.
Posted Mar 3, 2009 18:53 UTC (Tue) by DeletedUser32991 ((unknown), #32991)
I think your paraphrase is exaggerating what I said, which was more (in the part you chose to reword) bonding is a big part of the benefit Ubuntu gets from its bug triaging and the risk associated with premature bug-closing is much reduced compared to the other big distros because Debian fixes to annoying bugs find their way into Ubuntu in time. In particular, I would prefer if you omit clueless, even in parentheses.
This is largely unrelated to what Ubuntu does and does not on the engineering side. I am sure that Ubuntu also contributes fixes and packages to Debian. But what do you think that the bugs fixed for the first time shortly before and during Debian's lenny freeze mean to Ubuntu 8.04, 8.10, and 9.04? They will likely be fixed in 9.04 regardless of whether they were addressed in Ubuntu or not.
Posted Mar 3, 2009 19:54 UTC (Tue) by man_ls (subscriber, #15091)
As you probably know triaging originates in the medical profession, from when scarce resources can be devoted to treat huge masses of patients, either wounded in battle or victims of some epidemia. A medic first triages patients using three or four colors: green, yellow, red and black. You do not want to try to waste time resuscitating someone without a chance (black); and you do want to treat urgent patients (red) before stable ones (yellow), or even patients already cured (green). Now imagine if a well-meaning but ignorant volunteer was to do the triaging in order to save medical time. It would result in wasting time on unnecessary procedures and letting critical patients die. Somehow it would look a bit like Colin Watson's post, and that is not the proper way to contribute.
Just wishing that bugs will be fixed in Debian (and then patches will somehow get into Ubuntu) is not useful; bugs fixed in Debian do not necessarily percolate to Ubuntu. Just hoping that Ubuntu contributes something ("fixes and packages") back to Debian is not useful either. There has to be a willful dedication to contribute in a technically competent manner. A fanatic dedication to quality Control in Debian is precisely what sets it apart from other distributions, and Launchpad just seems to generate noise.
Posted Mar 3, 2009 20:22 UTC (Tue) by Requiem (guest, #51519)
Debian fixes bugs sure, but Ubuntu is drawing from Unstable, once the major bugs get fixed, those packages get moved into testing, and new packages get put into stable (not right away, but the point stands), so Ubuntu isn't seeing all the bugs Debian fixes at the testing--> stable step, and misses half the bug fixes in the Unstable --> testing step.
Posted Mar 3, 2009 22:05 UTC (Tue) by gevaerts (subscriber, #21521)
Posted Mar 4, 2009 10:40 UTC (Wed) by man_ls (subscriber, #15091)
Versions are entered into the unstable branch, and after a suitable period they go to testing. They stay there as long as the current version is in testing, or until a new version comes along. Every 18-36 months a new branch (including versions of all packages) becomes stable. But if a problem is found in a package, a new version of the package that corrects that problem comes again through unstable, and eventually promotes again to testing (or not if further problems are found). No changes are made between unstable and testing; patches are always applied before entering unstable.
If Ubuntu draws from the unstable branch they will eventually gather all fixes. Of course it can take 18 to 36 months to get a new version with those fixes, so with a lifecycle of 6 months most of the time Ubuntu is on its own.
Posted Mar 3, 2009 20:28 UTC (Tue) by michaeljt (subscriber, #39183)
Is it such a bad thing if Ubuntu is aiming to be accessible outside of the usual Linux audience, and trying to help those people contribute something back? I think that at least one of the things that "bonds" the more traditional audience to Linux is the feeling that they are "part of it" and one of the myriad small forces driving its evolution. Why should less technical people not be able to feel the same? It would be nice though if Canonical found more effective ways of getting them involved in the bug process, like say more reproducing and less marking as invalid. I'm sure that they could be a massive benefit to the FOSS community.
Posted Mar 3, 2009 21:20 UTC (Tue) by jspaleta (subscriber, #50639)
But is it helping? I don't think I've seen a metric from a developers standpoint that indicates that triage in Launchpad is helping them do their work more efficiently. Do Ubuntu developers feel that the triage is helpful? Do Debian developer find it helpful? Do upstream projects find it helpful? No one wants to be told that what they are doing isn't making a positive impact, but at the same time you don't want a large pool of people making more work developers just for the sake of feeling involved.
If a lot of developers feel like Colin...this could be a problem. But his frustrations could be an anomaly. There needs to be a less personal accounting of impact of triage on developer efficiency which takes Colin's experience and aggregates it with other developers.
Perception is not necessarily reality. The trendable metrics I've seen coming out of Launchpad focus on the number of open and closed bugs. And that's fine. If the most important thing is keeping the rate of growth of open bugs down that is a perfectly fine metric. But it is not a metric which directly tracks impact on developer efficiency or impact on the rate of valid bugs fixed. Are developers fixing valid bugs faster when there is more triage involvement? Do triagers help or hurt the efficiency of developers to address valid bugs and to generate valid fixes with patchsets? I don't know. The available metrics don't try to answer that question.
All I know for sure is that the Ubuntu triagers are doing a very good job of closing bugs and keeping the open bug count growth way down. But how much of that are inappropriate closures? <1%? >10%? I don't really know. I don't expect them to be perfect and if the triagers are only doing 1% inappropriate tagging that's very very good. I think they are making an honest effort to make a positive impact. I don't think people would spend their time and encourage other people to spend their time on this sort of thing unless they really felt that it was a positive impact. But sometimes its worth taking a moment and being self-critical and to question your own assumptions in an effort to self-assess. Hopefully Colin's voicing of frustrations will provide the opportunity for that sort of moment. We don't really know how widespread the problem is that Colin chose to blog about. Is it systemic? Or is it a few overzealous people who just need to be re-trained? Is it primarily new people with low karma or is it veterans with high karma? We don't know.
My understanding Launchpad's karma system doesn't have a concept of negative karma. Every action you take scores you positive karma, but I don't think there is any effort in place to track whether your action is making more work for people or is helping people work more efficiently. Negative karma for mistakes would help gain a better view of how widespread the problem is. If its systemic, you'll see a lot of negative karma build up and you could then adjust the process with a sequence of small corrections and try to reduce the negative karma build up. If its just a few people, you'll be able to see and address that with re-training as needed on an individual basis without calling them out publicly. If its primarily new people, you can adjust processes which stress more mentoring. If its veterans...you have a deep cultural problem which needs to be addressed through some triager/developer cross communication and try to reorient and understanding of workflow needs across the groups.
Posted Mar 3, 2009 21:37 UTC (Tue) by michaeljt (subscriber, #39183)
Who knows, perhaps they are already doing this? :)
Posted Mar 3, 2009 21:53 UTC (Tue) by jspaleta (subscriber, #50639)
What you don't want is bad practises going on for so long without a way to address them via normal process feedback resulting in developers like Colin feel like they need to make a public statement about the problems. I doubt he enjoyed putting people on the spot, but if triagers are making more work for him and other developers..that's a real problem. The developers are the critical resource, everything else needs to be tuned to help them be more efficient. The sort of statement Colin made is a red flag moment indicating the the processes aren't working. It draws attention to the problem but it has too little detail to know the scope of the problem.
The first step in addressing any problem is being aware of a problem. Negative karma would just be one way to seeing human process problems develop in a systematic and unbiased way. If you think its worth discussing more, feel free to write it up in Brainstorm and see if anyone takes notice. I'll even let you take credit for the idea.
Users doing bug triaging
Posted Mar 3, 2009 13:01 UTC (Tue) by epa (subscriber, #39769)
I haven't used Launchpad's bug tracking, but I suspect the problem is in user interfaces that present bugs as bad things or todo items, and closing bugs as a good thing. Instead of the aim of the process being to improve the software, the aim becomes to close as many bugs as possible. You get a good feeling from marking a bug 'dup' or 'wontfix' or 'needinfo' and it disappears from your todo list. It feels like an annoyance when the pesky reporter comes back and asks for it to be reopened.
In my view the most useful thing that non-programmers can do with a bug report is to reproduce it. Not to ask the reporter whether it still occurs in the latest version, but to follow the recipe themselves and report whether they can verify the bug. If the instructions on how to trigger the bug are not clear, then of course the triager can ask for clarification.
Making a bug reproducible is often the hardest part of the process; once you have a test case for reliably making a bug appear, fixing it is often trivial.
So I would suggest the visual rewards given by the bug tracker for making progress on bugs should be weighted towards activities that add value. A bug's possible states might go through
unconfirmed, but has a recipe to reproduce it
confirmed (by N separate people)
test case written
test case confirmed as valid by upstream
Each of these states is better than the previous one and could be displayed appropriately (perhaps a 'golden bug' for one that has a test case). However it works, the interface needs to give the perception that while bugs are a bad thing, bug *reports* are a very good thing, to be nurtured, tested and reproduced, not closed.
Posted Mar 3, 2009 18:39 UTC (Tue) by iabervon (subscriber, #722)
Personally, I think bug trackers should have support for a "me too" feature, where you find a bug in the database that you think is happening to you, and file an intentional duplicate (already marked as such) with your particular conditions. Then people trying to fix the bug get a bigger pool of reporters who might be able to test possible fixes (and find out about variations that don't affect the outcome, which is useful for eliminating ideas), and people who find their problem in the database can track it more directly.
And, of course, it should count more to close a bug (successfully) with a lot of duplicates, if those duplicates verify the fix.
Posted Mar 3, 2009 21:13 UTC (Tue) by fb (subscriber, #53265)
IMHO both in theory as in practice there is a problem. In theory each bug should have its own report. In practice often there is already a lot of tracking info in one bug, which gets lost ni the cross fire after merging. Second, it leads to conflicting reports, which makes it confusing for whoever is trying to verify, or fix it.
> Personally, I think bug trackers should have support for a "me too"
> feature, where you find a bug in the database that you think is happening
> to you, and file an intentional duplicate (already marked as such) with
> your particular conditions.
Actually launchpad has something like you describe, as users can subscribe to a bug. So you as a developer could choose to look first at the one bug with 25 subscribers, and not at the one with just 2.
Posted Mar 3, 2009 21:14 UTC (Tue) by fb (subscriber, #53265)
Posted Mar 3, 2009 14:57 UTC (Tue) by cortana (subscriber, #24596)
Instead of upgrading my parents' computer when I am next home, I'm going to install Debian instead. The only problem will be explaining where the Firefox logo went...
Posted Mar 4, 2009 21:08 UTC (Wed) by oak (guest, #2786)
I switched from Ubuntu (Feisty) to Debian (Lenny) half a year ago and
about only thing that's worse is that no Flashplayer (neither Open Source
nor proprietary) works properly in Konqueror (I'm using KDE3 desktop).
There's no update applet for KDE(3) either, but I'm doing my updates with
Synaptic anyway, so it's not a problem (IMHO viewing package changes is
much better in Gnome Synaptic than KDE Adept).
Posted Mar 5, 2009 12:45 UTC (Thu) by nye (guest, #51576)
What are the symptoms?
Are you using AMD64? If so, are you using the 64-bit plugin? Also do you have konqueror-nsplugins installed, and are you using gtk-qt-engine? I know that the KDE3 version of gtk-qt-engine causes Flash to crash in KDE4, which I assumed was due to version incompatibilities, but it's possible that it just doesn't work with Flash in Konqueror at all.
Posted Mar 5, 2009 21:27 UTC (Thu) by oak (guest, #2786)
Crashes and freezes.
> Are you using AMD64 / konqueror-nsplugins
32-bit (6 years old Athlon XP). Have nsplugins. KDE is v3.
Posted Mar 9, 2009 13:53 UTC (Mon) by nye (guest, #51576)
Posted Mar 3, 2009 16:42 UTC (Tue) by jengelh (subscriber, #33263)
Posted Mar 5, 2009 9:57 UTC (Thu) by Lovechild (guest, #3592)
This kind of experience tends to leave a bad taste in my mouth, especially since I feel kinda bad personally if I delay much interacting with users who file bugs against my packages. They took the time to tell me about a problem, clearly it was important enough to them to complain, I should take the time to reply. If it's a packaging problem I can fix it, if not I can get the bug in as good a shape as possible and push it upstream.
Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds