Goals of bug triage

But is it helping? I don't think I've seen a metric from a developers standpoint that indicates that triage in Launchpad is helping them do their work more efficiently. Do Ubuntu developers feel that the triage is helpful? Do Debian developer find it helpful? Do upstream projects find it helpful? No one wants to be told that what they are doing isn't making a positive impact, but at the same time you don't want a large pool of people making more work developers just for the sake of feeling involved.
If a lot of developers feel like Colin...this could be a problem. But his frustrations could be an anomaly. There needs to be a less personal accounting of impact of triage on developer efficiency which takes Colin's experience and aggregates it with other developers.

Perception is not necessarily reality. The trendable metrics I've seen coming out of Launchpad focus on the number of open and closed bugs. And that's fine. If the most important thing is keeping the rate of growth of open bugs down that is a perfectly fine metric. But it is not a metric which directly tracks impact on developer efficiency or impact on the rate of valid bugs fixed. Are developers fixing valid bugs faster when there is more triage involvement? Do triagers help or hurt the efficiency of developers to address valid bugs and to generate valid fixes with patchsets? I don't know. The available metrics don't try to answer that question.

All I know for sure is that the Ubuntu triagers are doing a very good job of closing bugs and keeping the open bug count growth way down. But how much of that are inappropriate closures? <1%? >10%? I don't really know. I don't expect them to be perfect and if the triagers are only doing 1% inappropriate tagging that's very very good. I think they are making an honest effort to make a positive impact. I don't think people would spend their time and encourage other people to spend their time on this sort of thing unless they really felt that it was a positive impact. But sometimes its worth taking a moment and being self-critical and to question your own assumptions in an effort to self-assess. Hopefully Colin's voicing of frustrations will provide the opportunity for that sort of moment. We don't really know how widespread the problem is that Colin chose to blog about. Is it systemic? Or is it a few overzealous people who just need to be re-trained? Is it primarily new people with low karma or is it veterans with high karma? We don't know.

My understanding Launchpad's karma system doesn't have a concept of negative karma. Every action you take scores you positive karma, but I don't think there is any effort in place to track whether your action is making more work for people or is helping people work more efficiently. Negative karma for mistakes would help gain a better view of how widespread the problem is. If its systemic, you'll see a lot of negative karma build up and you could then adjust the process with a sequence of small corrections and try to reduce the negative karma build up. If its just a few people, you'll be able to see and address that with re-training as needed on an individual basis without calling them out publicly. If its primarily new people, you can adjust processes which stress more mentoring. If its veterans...you have a deep cultural problem which needs to be addressed through some triager/developer cross communication and try to reorient and understanding of workflow needs across the groups.

-jef