LWN.net Logo

Two articles from Dag Wieers

Dag Wieers finds problems in bug tracking systems. Dag was looking for bug reports in Launchpad, for tools he had written. What he found were a few bug reports for new issues for which he had not been informed. "Not only is this a lost opportunity, it is a bad service to both upstream and the user itself. Without a bugtracking system, users would directly contact upstream. Now with Launchpad users report their bugs and nothing is done with them. Not by the maintainer and not by (unaware) upstream. And they are not being send to Debian (their upstream) either. And this is not specific to Launchpad per se, I have similar remarks for Fedora's bugzilla or OpenSUSE." In a followup article he proposes a Google index for Red Hat bugzilla.
(Log in to post comments)

Two articles from Dag Wieers

Posted Dec 16, 2008 21:09 UTC (Tue) by tseaver (subscriber, #1544) [Link]

Although it isn't true for Dag, some upstreams are hostile to getting bug reports from foreign bug trackers, particularly Launchpad. The Wikipedia entry for "barnburner" seems to explain the phenomenon of the Free Soil Farmers pretty well:

http://en.wikipedia.org/wiki/Barnburner

Two articles from Dag Wieers

Posted Dec 16, 2008 22:28 UTC (Tue) by tetromino (subscriber, #33846) [Link]

I may be missing something obvious - but what do barnburners and free soil have to do with Dag Wieers and bug tracking?

Two articles from Dag Wieers

Posted Dec 17, 2008 0:02 UTC (Wed) by louie (subscriber, #3285) [Link]

You missed something; I think the idea is that not taking bugs from upstream bug trackers is cutting off your nose to spite your face (or 'barnburning'). That could well be the case in some cases (I'm sure there is some reflexive anti-Canonicalism out there).

But in other cases, I know at least some projects (particularly small ones) have decided that the signal-noise ratio is too poor (and the volume too high) to make taking things from launchpad or other upstreams worthwhile. I realize 'not taking all bugs' is anathema to some people, but a bug tracker with too much noise in it means developers don't use it- and from there you are just on a vicious circle, where it gets used less and less. Much better to raise the barrier to entry if it means the tool will actually get used. (Ideal, of course, to have both low barriers to entry and lots of participation, but reality isn't always so nice and tidy.)

Two articles from Dag Wieers

Posted Dec 17, 2008 0:23 UTC (Wed) by jspaleta (subscriber, #50639) [Link]

Is Ubuntu's newest efforts at triage helping with signal to noise in launchpad? Things like the 5-a-day bug triaging effort and the upstream bug report. These are certainly processes and tools that can be replicated in other bug trackers, but are they worth it? Do we know yet? Are efforts like that making signal to noise better for upstream project dev who have been so far been reluctant to take launchpad bugs?

What if upstream projects were given the choice to receive emails from a distribution level bug tracker (which every one that happens to be bugzilla or launchpad or whatever) about a subset of bugs only after they have gone through distribution level triage?

-jef

Two articles from Dag Wieers

Posted Dec 17, 2008 0:34 UTC (Wed) by louie (subscriber, #3285) [Link]

I really don't know if things are better or worse overall; I don't follow it very closely. I can say that I do sense that launchpad is now big/sprawling enough to say that you can't generalize about it as a whole- some components I've filed bugs in launchpad against seem well triaged and quickly responded to, others... not so much. (At least on paper, I will say that the 5-a-day and upstream reports seem conceptually solid and worth exploring for all upstreams.)

Assuming that downstream triage knew what they were doing and did it consistently, only going to upstream after triage is reasonable. Those are big 'ifs', though :) Especially for small projects that don't have the resources to do much fix-up after the fact- they are going to be very sensitive if they are hit with low-quality bugs. Oh, and I'd say that email is So Yesterday; push for the downstreams to get to bugzilla 3.2 and use xml-rpc or something similar.

Two articles from Dag Wieers

Posted Dec 17, 2008 1:50 UTC (Wed) by jamesh (subscriber, #1159) [Link]

A fair bit of work has been put into interacting with upstream projects in Launchpad, with a number of options available:

1. Launchpad can track bugs in both distribution context and upstream context, so a number of upstream projects have adopted Launchpad for bug tracking. When this is the case, a bug can be tracked simultaneously by distro and upstream developers.

2. There is an API to let Launchpad interact with remote bug trackers. When an appropriate plugin is installed in the remote bug tracker, Launchpad can forward bugs and synchronise comments between bugs in each tracker giving similar benefits to (1).

3. If the project has not installed a plugin, it is up to distro developers to forward bugs manually. Once this is done, the upstream bug can be recorded and the upstream status will be tracked.

4. In cases where the upstream doesn't have a public bug tracker, it is possible to register an upstream bug email address for a project. This provides a way for distro developers to forward information in a controlled fashion.

So these things are certainly possible. Naturally, the options that involve automatic forwarding (as opposed to a human doing it) need some action by the upstream: anything else would be spamming.

Two articles from Dag Wieers

Posted Dec 17, 2008 4:11 UTC (Wed) by louie (subscriber, #3285) [Link]

1. Launchpad can track bugs in both distribution context and upstream context, so a number of upstream projects have adopted Launchpad for bug tracking. When this is the case, a bug can be tracked simultaneously by distro and upstream developers.

And as you know, I think it is reprehensible that Canonical has attracted projects to run their projects in a non-free honeypot. Go ahead and use it for your own distro, but inducing others to move in is unhealthy for free software.

As for the rest, automated tools are terrific, and I do wish Fedora and others using free tools had pioneered similar syncronization work. But like I said originally, it is only part of the story- there has to be commitment (on both ends) to do proper triage and handling of incoming bugs; no amount of email, xml-rpc, or plugins can help you with that. (I tend to think launchpad's horrifically bad UI makes this problem worse by frequently making it difficult to tell what a bug is actually filed against but that is a rant for some other time.)

Two articles from Dag Wieers

Posted Dec 17, 2008 9:23 UTC (Wed) by jamesh (subscriber, #1159) [Link]

> And as you know, I think it is reprehensible that Canonical has attracted
> projects to run their projects in a non-free honeypot. Go ahead and use
> it for your own distro, but inducing others to move in is unhealthy for
> free software.

The free software aspect of your complaint should be a non-issue in around six months, based on the recent announcements.

As far as inducement goes, the projects that I helped migrate were not interested in running their own bug tracker: either they were previously using a hosted system (e.g. SourceForge), or they no longer wanted to maintain their existing tracker software (e.g. Zope). One thing they were concerned about was access to the data, which is covered by Launchpad's web services API and the offer to provide bulk data exports on request.

I think your argument that projects are induced to use Launchpad would carry more weight if there wasn't active work to improve interaction with remote trackers. There certainly are trade offs to the use of hosted systems, but projects adopting Launchpad are generally aware of the risks and benefits.

Two articles from Dag Wieers

Posted Dec 17, 2008 12:41 UTC (Wed) by epa (subscriber, #39769) [Link]

Sourceforge is also a 'non-free honeypot', as is Google Code. The newer crop of project hosting sites seem to be the same: GitHub appears to be non-free by your definition, likewise Sun's Kenai, and so on. What project hosting sites meet your standards, and why?

(I can think of GNU's Savannah, for which you can download the Savane source code. But it still doesn't give you the ability to modify the code that is running on the site and being used to manage your project. The only way to get true freedom, surely, is to host the project on a box you manage yourself.)

Two articles from Dag Wieers

Posted Dec 17, 2008 18:55 UTC (Wed) by skvidal (subscriber, #3094) [Link]

I don't think luis would encourage the use of sourceforge or google code, either.

Notably, all the code is open and freely available for http://fedorahosted.org.

-sv

Two articles from Dag Wieers

Posted Dec 20, 2008 15:33 UTC (Sat) by cortana (subscriber, #24596) [Link]

<a href="http://alioth.debian.org/">Alioth</a> has been provided by Debian for just this purpose for some time. :)

Two articles from Dag Wieers

Posted Dec 17, 2008 17:09 UTC (Wed) by nevyn (subscriber, #33129) [Link]

> As for the rest, automated tools are terrific, and I do wish Fedora and
> others using free tools had pioneered similar syncronization work.

Well they didn't create their own proprietary solution, but Fedora has had bugzilla integration with upstreams for years now (which was the only usable game in town when they started), and are active upstream. Of course Canonical are mostly a PR company, so get all the credit for everything...

I'm unsure if trying to tie RT, issue-tracker, BZ, launchpad, trac, etc. together is going to be viable, long term. They are just too different and in the cases I've seen it the "integration" looks much worse than a "normal" bug or just copying and pasting the bug ... and thus. is easier on the reporter, but worse for the fixer.

Two articles from Dag Wieers

Posted Dec 17, 2008 17:53 UTC (Wed) by jspaleta (subscriber, #50639) [Link]

If the current LP plugin approach with other tools doesn't work well as its being implemented right now, that doesn't mean its not going to result in something more useful down the road. An API for LP isn't a bad thing.

What's bad is that we seem to be growing different APIs for different services...which causes lots of duplication of effort when writing code for communicating with them.

What is really needed is for all the bug tracking tool developers to get into a room (real or virtualized) and talk about a common and extendable API specification that will work to share information across tool instances and services. The LP devs have a particular idea about what that should look like and are going ahead and implementing something on their own and providing an initial set of plugins for some external tools to work with that API. Providing plugins for other tools plugins for your customized API is an interesting way to avoid trying to have the consensus discussion about what a common API needs to look like.

Is the LP API any better or worse than the existing bugzilla xmlrpc based service? or the trac xmlrpc service which you can get as a plugin for trac? Projects like the eclipse based Mylyn already make use of these existing xmlrpc services if instances have them turned on (no LP plugin for Mylyn yet.) Just having LP have an API to communicate with isn't a shockingly innovative idea. I would hope that as part of the discussion moving forward someone will publish a comparison between LP API and bugzilla xmlrpc service. A Mylyn developer working on connector code might be a good person to compare and contrast the APIs.

-jef

Two articles from Dag Wieers

Posted Dec 17, 2008 18:31 UTC (Wed) by hppnq (subscriber, #14462) [Link]

And as you know, I think it is reprehensible that Canonical has attracted projects to run their projects in a non-free honeypot. Go ahead and use it for your own distro, but inducing others to move in is unhealthy for free software.

This keeps ... bugging me. You are sitting on your high horse. You are wearing your GPL T-shirt. You have checked that the evil Canonical are still using non-free software. You are ready to set out on today's crusade.

Why then would you write such a comment using the non-free honeypot deployed by LWN?!

Don't get me wrong: I am genuinely baffled. What is the big difference for you?

Two articles from Dag Wieers

Posted Dec 17, 2008 16:04 UTC (Wed) by dcbw (guest, #50562) [Link]

I personally wouldn't want to receive un-triaged direct reports from Launchpad, because Ubuntu carries quite a few patches against NetworkManager for customizations that aren't upstream for various reasons. They also use versions which aren't updated with bug fixes that have already been committed to NM git. Thus I get lots of redundant bugs in gnome bugzilla (where NM bugs live) where all I can say is "already fixed, please ask your distro to patch or update" or "that's an Ubuntu specific customization that is not upstream, sorry!". Makes for a much worse signal to noise ratio *with* some triage already going on in LP.

defining triage

Posted Dec 17, 2008 16:14 UTC (Wed) by louie (subscriber, #3285) [Link]

*good* triage would sort those out for you. If it isn't resolving these issues (patches, old versions, etc.), it isn't worth calling triage.

Two articles from Dag Wieers

Posted Dec 17, 2008 19:47 UTC (Wed) by jspaleta (subscriber, #50639) [Link]

Question for you as an upstream project developer.

Since Ubuntu has implemented the 5-a-day triaging and other initiatives are you seeing a change in the Ubuntu relevant bugs in NM upstream? Are things getting better in any way that matters to you as an upstream developer? Is the throughput volume into the upstream tracker more or less than before? Are the quality of the reports making it through the Ubuntu QA processes getting better, more signal, less noise?

We've seen discussion about how Ubuntu's renewed effort at triage is affecting the launchpad numbers:
http://mdzlog.wordpress.com/2008/10/29/ubuntu-quality/
http://poelcat.wordpress.com/2008/10/31/credit-to-ubuntu/

But what we don't know is how the changes in Ubuntu QA processes are affecting upstream. I'm all for replicating things like 5-a-day across all distributions, if its an effective aid for upstream developers. And that's the part I don't see a discussion about, are the triage processess that Ubuntu have put together helping upstream bug quality? Maybe its too early to tell yet.

-jef

Two articles from Dag Wieers

Posted Dec 17, 2008 20:16 UTC (Wed) by nix (subscriber, #2304) [Link]

Good grief, Jef, will you just shut up about this? Post on your own blog,
or something, but by my count nearly a fifth of the comments on *all of
LWN* over the past week have been your incessant bleeding complaints about
everything you think Canonical is doing wrong, generally conducted with an
absolute minimum of actual research.

Get a new hobby. A *private* hobby, for preference.

Two articles from Dag Wieers

Posted Dec 17, 2008 20:28 UTC (Wed) by louie (subscriber, #3285) [Link]

These are honest, legitimate questions- he's asking 'hey, this looks interesting, does it work?' If you're lumping this in with the rest of jef's Ubuntu-bashing (which even I agree is tiresome) I think you also need to take a deep breath and step away from the keyboard and consider getting a better hobby of your own.

Two articles from Dag Wieers

Posted Dec 17, 2008 21:24 UTC (Wed) by nix (subscriber, #2304) [Link]

TBH I barely bother reading jspaleta's stuff anymore. If his posts contain
the strings 'Ubuntu', 'Canonical' or 'Shuttleworth' it is probably not
worth reading. Flooding does that.

ad hominem

Posted Dec 18, 2008 15:55 UTC (Thu) by mmcgrath (subscriber, #44906) [Link]

> TBH I barely bother reading jspaleta's stuff anymore. If his posts contain the strings 'Ubuntu', 'Canonical' or 'Shuttleworth' it is probably not worth reading. Flooding does that.

Trying to discredit an individual does not mean that what he is saying is incorrect nor does it mean he should not ask questions.

ad hominem: yeah right

Posted Dec 18, 2008 17:05 UTC (Thu) by hppnq (subscriber, #14462) [Link]

Just from how these posts look people should be able to deduce that not all of the 200 question marks point to serious, relevant, and informed questions.

The reasonable request was, and still is, to please ask these "questions" somewhere else. Sometimes I get the feeling that I have ended up in a Fedora evangelists forum, and I did not subscribe to that, as much as I like Fedora. Which is rapidly changing by sheer association.

Every time someone mentions Launchpad, Ubuntu or the bloody NetworkManager we get the same predictable flood of reactions. Yeah yeah, my own included. But I have a point. ;-)

ad hominem

Posted Dec 18, 2008 22:23 UTC (Thu) by nix (subscriber, #2304) [Link]

It's an expression of exasperation, not an ad hominem attack. I'm not
saying 'Jef is attacking Canonical therefore he is a bad person and eats
baby seals', I'm saying 'please do it somewhere else'.

Two articles from Dag Wieers

Posted Dec 17, 2008 22:34 UTC (Wed) by bkor (guest, #27950) [Link]

It is like the boy who cried wolf one too many times.

Two articles from Dag Wieers

Posted Dec 18, 2008 15:57 UTC (Thu) by mmcgrath (subscriber, #44906) [Link]

> It is like the boy who cried wolf one too many times.

Wasn't the boy right in the end though? IIRC everyone suffered for ignoring him.

Two articles from Dag Wieers

Posted Dec 17, 2008 20:35 UTC (Wed) by jspaleta (subscriber, #50639) [Link]

For the record that wasn't a complaint. It wasn't even a back handed one.

I am genuinely interested in knowing if the new triaging initiatives Ubuntu has been working on are making a positive impact on upstream interaction. I think the new initiatives like the 5 a day project are honest efforts to positively impact bug quality. I want the new initiatives to be effective. But understanding if the are effective means getting feedback from upstream developers, as well as bug reporters.

If Dan, as an upstream developer is going to comment here about his previous interactions as an upstream receiving reports from distributions, I think it only fair to get him to comment specifically about the impact of the new ideas. This isn't a setup. I'm hopeful that if he takes the time to look he'll be able to say he's seen a positive change in the quality.

The feedback loop has to be closed. If the initiatives are helping coordinate reporting with upstream that were having problems, great, we can start replicating those processes in other distributions. If they aren't helping upstreams be more effective, then better to get feedback sooner rather than later so the interaction can be adjusted.

-jef

Two articles from Dag Wieers

Posted Dec 17, 2008 0:22 UTC (Wed) by sbergman27 (subscriber, #10767) [Link]

He's not referring to Dag, who wants *more* interaction with downstream bug tracking systems. He's talking about those projects which are hostile with regards to interaction with Ubuntu^WCanonical^WLaunchpad.

It's behavior that does not surprise me, as the community seems to have built-in self-defeatist daemons running in the background. Whenever one entity in the community gets things *too* right, and becomes *too* popular as a result, they become active to work against its success. Barn-burning. Cutting off one's nose to spite one's face. There are many analogies to describe the all-too-common phenomenon.

Linux's success on the server has been had *in spite of* those daemons. (Red Hat is the new Microsoft, didn't you know?) Any true success for Linux on the desktop will also have to be *in spite of* them. Interestingly, successful embedded Linux players seem largely ignored by them.

Two articles from Dag Wieers

Posted Dec 17, 2008 17:33 UTC (Wed) by tseaver (subscriber, #1544) [Link]

My reference was to the FSF's policy of refusing to allow links
to bugs in Launchpad. The analogy to the "barnburners", who were
radical Whigs, so named because they would burn their own barn
(i.e., destroy the Whig party) to deal with an infestation of rats.

The barnburner Whigs allied with the Free Soil party to nominate
Martin Van Buren for the presidency in 1848; the pun on FSF seemed
apropos.

bugzilla and google

Posted Dec 17, 2008 0:08 UTC (Wed) by louie (subscriber, #3285) [Link]

I agree that in principle it would be good if bugzilla were google-able, but Bugzilla being google-bot'd is roughly the most effective DDOS you can imagine. I'm really not sure what exactly it is that makes the two such a disastrous pairing, but it is bad enough that bugzilla ships by default with a robots.txt that prevents spidering. (There have been performance improvements in recent releases, but I'm not sure anyone wants to test those in this way.) That is certainly why bugzilla.GNOME isn't spidered; I imagine it is the same for fedora.

bugzilla and google

Posted Dec 17, 2008 5:23 UTC (Wed) by dag- (subscriber, #30207) [Link]

Why not then create a static page of bugzilla with all the content and clickable links to the fenced (robots.txt) bugzilla ?

Why hold back all that useful information that may help people and let your users in the cold and leave them all by themselves ? If it is not in Google, it barely exists for the majority of people.

Maybe thinking out of the box is required ;-)

bugzilla and google

Posted Dec 17, 2008 7:19 UTC (Wed) by drag (subscriber, #31333) [Link]

Ya. I know that search provided by Redhat's bugzilla stuff is pretty horrible.

Also end users are not qualified to determine which piece of software is causing a problem. There are lots of dependencies.

So if I am having problems with SDL applications crashing under pulse-audio, which package should I search for? SDL-audio, SDL, PA, or what? Typical end users are not going to understand this problem.

So you want to do searches based on errors getting spitted out and such things. But that's not really possible. So that leads to a lot of people not filing bugs.

--------------------------------

For example in Fedora 10 I am finding the Intel drivers very troublesome. I never had issues with them crashing applications or locking up my system on Debian, but with many games (such as Nexuiz) will reliably cause lockups. So what do I file against? Nesuiz? The DRI drivers? The kernel? Just xserver in general? Or what? I don't know for certain if it's video drivers, it could be audio causing the crash for all I know. Doing the libgl_debug verbose stuff doesn't really provide useful information.

I can guess at what to file a bug against, and narrow it down to may 2 or 3 different possible packages that should get fixed. But I don't really know all the details about how X and OpenGL works. It could be anything for all I know.

So I can try like I've done in the past and just file bugs and hope soembody pays attention and I don't create a redhearing or waste people's valuable time because I don't understand everything going on... but I've always been ignored (not in Fedora, but in general) for these sort of graphics-related bugs, so it's best that I just ignore the problem and fart around with compiling custom drivers to try to make the problem go away on my own.

Good search would probably help with that. Then I could reliably know if people filed bugs against this issue before.. and then I could feel good about filing a bug even if it's very misleading.

It would certainly help cut down on duplicate bugs...

bugzilla and google

Posted Dec 17, 2008 16:29 UTC (Wed) by dag- (subscriber, #30207) [Link]

I agree with you. Your example is spot on.

But where you are very vague on what this search consists of, I think it is important that this search is the search most people use, either as part of the browser, or as part of their own habit.

Which (unless they are community members) is seldom the Bugzilla or Launchpad search, even when it would perform better and were more user-friendly.

That is why I believe Google is the solution here. Though I would not exclude other or future search engines. It was just convenient to use Google in this context.

bugzilla and google

Posted Dec 17, 2008 18:29 UTC (Wed) by iabervon (subscriber, #722) [Link]

My dream bug tracking system would be a local program that you run, and it would take you through a tech support script starting extremely generic ("What program are you having a problem with?") and pulling in scripts and info about different projects as needed. So it would be something like:

What's wrong? Computer locks up
Does a particular program trigger it? Nexuiz
Anything else? (other games)
(system looks up those games, finds they're all OpenGL)
How about glxgears? Yes
(and so on)

The ultimate result would be that it would either end up filing a new bug report for some project or some projects, or it would lead you to an existing bug report, which might tell you that it's fixed in versions of things you don't have installed (now we check the package manager or file a distro bug to get them to ship the working version), or that there's a workaround, or that it's still entirely open.

One major source of script questions would actually be existing bug reports, where it's trying to determine if your bug matches them or is different, and when it fits a small number of existing reports, you'd see those reports; it's effectively a search at that stage. But if it doesn't find anything, it turns into the process for filing a bug.

bugzilla and google

Posted Dec 17, 2008 17:57 UTC (Wed) by iabervon (subscriber, #722) [Link]

Ideally, bug trackers wouldn't be have to be spidered. The web pages are a thin layer over a database; they'd do better to just have the database exported, and Google could deal with it in a structured fashion and probably get better results than just looking at the pages (if a search term is a version number, look in the version fields and not in the descriptions, etc). There's no need to render each bug's page and then ignore (or worse, respond to) the page's rendering.

bugzilla and google

Posted Dec 17, 2008 21:30 UTC (Wed) by raf (guest, #35151) [Link]

Seems like a good opportunity to use OWL (or similar). All it takes is a thin layer that generates OWL/RDF documents instead of HTML, which would be crawled by an RDF-aware spider. I've toyed with this idea myself for integrating my company's private Bugzilla with other internal (proprietary) issue-tracking systems.

bugzilla and google

Posted Dec 17, 2008 22:39 UTC (Wed) by bkor (guest, #27950) [Link]

Another reason why bgo isn't indexed is that some developers don't like it. See http://bugzilla.gnome.org/show_bug.cgi?id=142505#c16.

e.g. with Bug-Buddy suddenly your bugreport is easily searchable. This while it hasn't been for years. If there are objections to indexing from highly respected developers, then making it indexed by Google is counter effective (gain search interface, but annoy at least one developer).

Two articles from Dag Wieers

Posted Dec 17, 2008 3:21 UTC (Wed) by yarikoptic (subscriber, #36795) [Link]

It just boils down to the burden of package maintenance, which someone needs to take on their shoulders; thus there should be a person interested in having a package in good standing.

Most of the packages in Ubuntu are packaged by Debian developers or maintainers, which spent their spare time packaging it, thus, most often, they are eager to see their 'baby' doing well. Especially in universe/multiverse of Ubuntu, most of the packages are just grabbed from Debian, with sometimes semi-automated patching for various Ubuntu-specifics (like adjustment of Maintainer field, which I was recommended to do the same for the package in Debian ;-)), and there is barely any person who feels that those packages are 'their', and that makes them exist like poor orphans without appropriate care. Needless to say that noone cares to inform upstream or Debian maintainer about a glitch in the package or the software, unless glitch is a very pronounced one and impacts plentiful of users.

Another 'bad favor' done by Ubuntu for the users, imho, is their rapid release cycles, which for proper maintenance would require even more of human power. I know that some of them are LTS and some are not... but now explain that to the users as well. I monitor few Debian packages of mine in Ubuntu (what if some real bug ever would be reported?) and often I see the lengthy lists of reports "I have the bug too!!! What could I do", after the problem has already been resolved in the current release. that requires additional 10 or 20 messages until someone buzzes MOTU to sync the version. That is simply pathetic, and shows the shortcomings of the maintnance process.

Two articles from Dag Wieers

Posted Dec 17, 2008 15:56 UTC (Wed) by drag (subscriber, #31333) [Link]

What would be nice is better binary compatibility between distributions so that the packaging can be done upstream, rather then having multiple redundant packaging efforts being done by distributions for their own benefit.

After all upstream folks know best about the optimal dependencies and build options. RPMs and Deb files are fundamentally the same items.. sure they are different formats, but the sort of information required to build either one is going to be about the same.

So have the build systems people use optimally just spit out deb and rpms files instead of just binaries.

Then the distribution's job would be to gather up the software, bundle it together, and work out any conflicts and fixes to packages and submit those fixes back to upstream. Add them as patches to the source code and build system.

that way package maintainers work much more closely (in general) with the software developers then the distribution developers.

Really a few years ago there were all sorts of reasons to have very big variations in linux operating systems. Nowadays the 'linux plumbing' stuff is something that should be standardized across all major Linux distributions and distributions then can stand out on the quality and configurations they offer to end users.

In other words, take the classic computer science approach to scalability, and push packaging out in a distributed manner. Nowadays it should be doable.. people have a lot better understanding of the efforts and things that it takes to get good packages. Things like Yum vs Apt-get or Deb vs RPM or whatever.. they are just mirror images of each other. Different software, but with the same goals and whatnot.

Two articles from Dag Wieers

Posted Dec 17, 2008 16:21 UTC (Wed) by dag- (subscriber, #30207) [Link]

As a packager I disagree. Distributions differ a lot, not only between the different vendors/communities but even between different releases. Even if you are able to standardize all distributions wrt. initscripts, path locations, installation tools. You have the differences between services.

Upstream does not know about all the possible differences, and neither do the individual packagers. Nobody is really interested to learn about the differences either, so this is a lost proposition.

But what is important is that common bugs between distributions are tied to each other and that upstream has an overview of what bugs are still open, which ones are already fixed in newer releases.

The real benefits I (as upstream) have out of this is that the more information is shared and the more discussions are being made between users and upstream, the better a problem is understood, the higher the chance a bugfix is already provided and the more likely the real fix solves all the issues better.

Even wrong bugfixes and different viewpoints help a lot in the process.

BTW If we are talking about package issues, sometimes a packaging issues is not identified as one to the packager and sometimes it is not identified as on by upstream. So even packaging issues may require the help of both sides. That is why I am a strong believer that upstream should be aware of all issues, including those that are believed to be packaging issues.

This is how it works

Posted Dec 17, 2008 6:08 UTC (Wed) by dwaynebailey (guest, #49311) [Link]

I produce a South African English spell checker as part of a much larger body of localisation work and development. I don't package for upstream but have worked many times to encourage others to get it packaged.

I've seen two instances that are frustrating, I get the two confused, so think of them as observations.

The first was being forwarded a link to a Launchpad bug. Lucky for me I knew the German developer who forwarded it. I found a bug that was at least a few months old. Full of comments, some from clearly frustrated users, all from South Africa. A bit like your next door neighbour contacting some agency across the world to get me to fix "our" fence.

The other was seeing a similar bug report but this time finding it simply because I was searching for a something related to what I do. What a surprise to see a bug report that could have helped many users.

For this piece of work their really isn't an upstream. I don't have an upstream bugzilla, just an email address. That might complicate things. For me the clear thing missing is that in packaging we should be creating the link between the package and upstream. Whether that is plugins to connect bug trackers, a simple textual description of what to do to push bugs upstream, a maintainer that is more proactive, or simply ensuring that upstream is always CC'd on bugs. I don't really care but we should ensure we create that link, these links being missing is what creates these frustrations.

Two articles from Dag Wieers

Posted Dec 17, 2008 20:39 UTC (Wed) by jengelh (subscriber, #33263) [Link]

Claims, claims. Let's have some statistics. Caution: This post may include sarcasm.

Project: pam-mount
Source: mailing list archives
Total posts: 840 as of today. Period: a bit more than 3 years when it moved from just-a-homepage to SF. Occassional duplicates may be contained.

[From me: 383 (~46%)]
From the Fedora maintainer[1]: 79 (~9%)
From the Debian maintainer[1]: 76 (~9%)
From the Gentoo[2] maintainer: 10 (~1%)
From the SUSE[3] maintainer: 5 (<1%)
From someone[1] at Canonical: 0.000000000
Users (the rest): 287 (~34%)

[1] They keep me busy with patches.
[2] I give them no time to accumulate patches.
[3] They virtually have no patches.
[4] Humble apologies for not having present the name of the Ubuntu proxy. Probably because there never was one.

Now let's look at the Bugzillas... number of open issues and number of comments on the oldest issue still open. I am trying to find a metric for issues piling up and nobody answering, so here goes. Note that comments may include my own:

Debian BTS: 1, 4
Redhat BZ: 5, 3
SUSE BZ: 7, 13
Gentoo BZ: 1, 5
Unbunt LP: 10, 3.

Now we have to ask ourselves whether this could be within noise, that is, whether pmt is a magnet too small. Looking at the total number of BZ entries might shed some light. Realistically, you should divide the number from above's paragraph by the total number of bugs over all time to get the ratio of bugs open on the average^Wthis day or so (someone care doing deep math analysis like Sourceforge's “average open days”?):

Debian: 143 if I did it right
Redhat: 20
SUSE: 50
Gentoo: 26
Launchpad: 10

Two articles from Dag Wieers

Posted Dec 17, 2008 22:48 UTC (Wed) by jspaleta (subscriber, #50639) [Link]

You'll have to help me out. Which specific claims are you trying to address with numbers. I want to make sure I have the right context.
If I had a better idea of the questions you were trying to ask I could probably craft a set of bugzilla queries across multiple compnents for Fedora's stuff beyond pam_mount. I've been looking for a good excuse to play with python_bugzilla.

Just as an aside 3 of those Red Hat bug hits are essentially the same bug report, filed for different Fedora release targets F8,F9,F10. But that's only observable on visual inspection there's no way to know that until you look.

The oldest bug should have been addressed with the available update to F9 but the bug is still open because someone experiencing the problem hasn't confirmed the fix and the bug wasn't cross linked from the update service for auto-closure. This is a candidate for basic triage closure like a 5-a-day program.

The 5th bug and the one with the most comments is maybe a real one that impacts upstream, but its going to be hard to reproduce or to confirm a fix for because its triggered by a config file format conversion from the older config file format to the new xml file format, that runs at package install time. File format conversions are always a fun corner case.
https://bugzilla.redhat.com/show_bug.cgi?id=463429

For that sort of subtle hard to confirm bug, at what point do you as an upstream want to be poked?

-jef

Two articles from Dag Wieers

Posted Dec 20, 2008 12:08 UTC (Sat) by jengelh (subscriber, #33263) [Link]

>You'll have to help me out.

That something may be wrong with Launchpad or something around it.

>Just as an aside 3 of those Red Hat bug hits are essentially the same bug report,

Yes. If the number of bugs were greater, the statistical noise would be smaller. I chose to leave it as is. But, the "noise" hints to something different: that there never was a security-fix release in Ubuntu. Great, Scott.

>For that sort of subtle hard to confirm bug, at what point do you as an upstream want to be poked?

If they cannot figure it out on their own within like, 1 or 2 days, going upstream is essential. Otherwise, they risk not getting any answer *at all*, because bugs just rot in Launchpd and upstream is then already months away from the conversion, uninterested in old bugs as they have been ironed out by the other distros.

Bugzilla and Compatriots are The Spawn of The Devil

Posted Dec 18, 2008 2:05 UTC (Thu) by dkite (guest, #4577) [Link]

These systems are always developed or imagined by someone who can pay
hordes of determined employees to sift through mounds of useless
information.

Somehow, free software projects have adopted the technology without the
means to provide the necessary manpower to keep the thing useful.

So we end up with, I know, A DataBase. Let's make it Searchable. Easy to
Use. Encourage Responsible Users to Report Bugs. So eager users find a real
problem, research, write out a legible and reasonable bug report, put it in
the appropriate database, and then sit and wait. And wait.

On the other end, you have a time constrained developer, interested in
making his stuff better. (S)He ends up spending his precious hours sorting
through mounds of useless or incomplete or incomprehensible bug reports. By
the time he finds a valid report, emails the reporter for clarification the
user has moved on, forgotten, or ...

Bugs require communication between the developer and the one using the
software. The bug report mechanisms either inhibit the communication or
encourage/allow too much.

I don't know the answer other than cultivating a small but diverse user
base of available and helpful testers. And as the project stabilizes and
the small user base finds no flaws, use them to build their own secondary
user bases to further test and pass issues on to the developer. And so on.
Known problems won't clog the system. Any communication will be
constructive or filtered out. If the traffic gets too high, change email
addresses and start again.

This isn't a technological problem. It is a human problem, and most
'systems' seem to ignore that. Or depend on countless drones to keep the
damn thing running.

Derek

Two articles from Dag Wieers

Posted Dec 30, 2008 6:49 UTC (Tue) by dag- (subscriber, #30207) [Link]

For those people reading up on this article/thread, apparently Red Hat has come around and opened up their Bugzilla for Google and providing it with an index (sitemap) of all bugreports in their robots.txt somewhere around 23/12/2008.

As a result, queries like this one:

http://www.google.be/search?q=dstat+site%3Abugzilla.redha...

now work fine. This is particularly good news for both of the articles I wrote. Upstream can search for problems with their project (for both Fedora and RHEL) and users of Fedora and RHEL can now search for common errors and resolutions.

I am grateful to Red Hat for doing this, even if it was not my article that ignited this. (Having worked in big companies, it is hard to imagine that they turned around and implemented this in little more than a week)

Copyright © 2008, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds