Ubuntu debuts its Upstream Report [LWN.net]

By Jake Edge
October 1, 2008

Ubuntu has taken some heat over the years for its relationship with upstream projects, but the distribution seems determined to change that impression. To that end, Ubuntu has started by looking at bugs and bug reporting between the distribution and upstream projects. The visible result is the beta release of the Ubuntu Upstream Report, which displays the progress of getting bugs upstream.

Users of Ubuntu report lots of bugs in the software they use but, for the most part, those bugs aren't in any way specific to Ubuntu; they tend to also exist in the upstream project. Ubuntu collects its bugs at Canonical's Launchpad web site which allows linking those bugs to bugs in the bug tracking system of an upstream project. Once the link—or watch as it is called in Launchpad—is established, updates to the upstream bug's status will be reflected in the Ubuntu bug as well.

That capability has been available for some time, but as Ubuntu looked at ways to improve how well their bugs were flowing upstream, they needed a way to measure how well watches were being used. Canonical's Ubuntu community manager Jono Bacon describes the idea behind the report:

In terms of this project, I was keen to see graphs that show the number of upstream bug linkages going on, the total number of open vs. upstream bugs and how many bugs are fixed elsewhere. We could use these graphs to determine our progress in improving our bug workflow, but this was not enough - we also needed raw data about which projects needed the most focus. Which projects were struggling the most with bug figures? Which projects were not forwarding bugs upstream? Which projects didn't have an upstream bug tracker registered in Launchpad? We had all the answers to these questions in Launchpad, but no means of gathering them. To fix this, we created the Ubuntu Upstream Report.

The report ranks Ubuntu projects by the number of open bugs, while also showing how many have progressed towards upstream. Bugs in Ubuntu get triaged by the Ubuntu bug team, with some of them getting classified as "upstream"—meaning that they exist in the project itself, rather than just Ubuntu's build. Upstream bugs that are linked to a bug in the projects bug tracker are considered "watch" bugs. Each successive stage shows the difference between the previous, both as a number and a percentage so that it is easy to see how bugs are being handled as well as where the bottlenecks are. This dashboard-style interface also allows sorting by column and retrieving lists of bugs by following the numeric links.

The report was created by Jorge Castro, who is in charge of external project developer relations for Canonical. The tool has multiple uses, as Castro explains:

We wanted to provide a tool that not only shows upstreams how well we're linking and forwarding bugs, but a day-to-day tool for maintainers to see where there are targets of opportunity to forward to upstream. And lastly, for triagers we wanted to provide real-time working "bug lists" that you can work through if you want to help be the bridge that connects the downstream Ubuntu Package to the upstream project.

Part of the idea is for the report to be used by participants in Ubuntu's 5-A-Day initiative. 5-A-Day is an effort to make the Ubuntu bug list better by encouraging users and developers to work on five bugs each day. Users can do things like try to reproduce the bug, cleaning up and adding more information to the report; while developers can triage bugs or look at patches to the upstream project to see if they are needed for Ubuntu. The report will also help those who are running or participating in Bug Jams—focused efforts to gather people together to move Ubuntu bugs along.

Linking to existing upstream bugs or creating new ones for problems that Ubuntu users find can be helpful for projects. Some projects will find it more helpful than others, as Bacon notes:

If we do link a bug upstream, we had no firm idea how useful an upstream actually find our bug data. Our discussions suggested very mixed reactions - a small project is likely to have a very different perspective on bugs than a large project. Just think about this in purely quantitative states - a small project will likely get fewer bugs, and these bugs can probably be dealt with by a small collection of volunteers. This is unlikely to scale to something like the Linux kernel or OpenOffice.org.

One of the problems, of course, is the one-way nature of the watch link—Ubuntu sees changes to the upstream bug, but the reverse is not true—as projects have to come looking in Launchpad for updates. There is also resistance to using Launchpad because it is not free software, though that is slated to change by mid-2009. Overall, this new report and the focus on improving upstream relations are very welcome, but tracking bugs only goes so far; fixing upstream bugs is an important, but missing, piece.

In order to not be seen as just a consumer of upstream software, one needs to not only report bugs, but fix them as well. For all of the various bug-related efforts that Ubuntu is sponsoring, there is very little mention of actually fixing problems and sending patches upstream. There are tools like Harvest that make it easier to find upstream patches—bug fixes and enhancements for possible inclusion in the Ubuntu packages—but the focus is clearly on improving Ubuntu, as opposed to improving the software ecosystem that makes up the distribution.

It is important to remember that the efforts so far are just a start; Ubuntu is working on additional projects to improve its upstream relations. One gets the sense that they have heard the criticisms and are working to address them. Like it or no, Ubuntu has its own way of doing things which may mean it takes longer than some would like, but it certainly looks to be headed in the right direction.

to post comments

smoke screen?

Posted Oct 1, 2008 17:56 UTC (Wed) by s0f4r (guest, #52284) [Link] (22 responses)

This feels like a bit of a hastily put-together smoke screen for the publicity effect. It's unfortunate since it will yet again put the ubuntu users up against the upstream developers more, after all the flame blogging that has been going on recently.

Greg KH righteously slapped canonical in the face and told them to get their act together and fix the process. Maybe it was not polite, but it was based on indisputable numbers. I don't care how much ubuntu contributes to gnome personally. gnome != linux. Ubuntu would switch away from gnome to something else if it could in a jiffie.

This project aims at showing how well ubuntu is forwarding bugs upstream without actually helping upstream at all. The net effect is that ubuntu users will even more get the idea that upstream is the problem and causing their bugs, while ubuntu can sit at the sideline passing the blame.

<sarcasm>
Great way to make upstream developers appreciate you, ubuntu!
</sarcasm>

smoke screen?

Posted Oct 1, 2008 18:16 UTC (Wed) by JoeBuck (subscriber, #2330) [Link] (3 responses)

If Ubuntu contributed nothing to the kernel but contributed big-time to Gnome and freedesktop.org projects, that would still be a huge service. While it wouldn't help those who are only interested in Linux as a server or embedded OS, it would still be a big contribution.

However, upstream shouldn't be asked to deal with Lauchpad; Ubuntu contributors to upstream should deal with making sure that bug reports are handled properly in upstream projects' bug-tracking systems (likewise for other distros).

smoke screen?

Posted Oct 2, 2008 3:54 UTC (Thu) by jamesh (guest, #1159) [Link] (2 responses)

When a bug watch has been added in Launchpad, it means someone has triaged the distribution bug report and either (a) linked it to an existing bug in the upstream or (b) filed a bug upstream and linked to that. When this is working correctly, the upstream developer wouldn't be required to look directly at Launchpad (but could if they wanted to).

This is basically the same way Debian maintainers interact with upstream, but with a bit more automation of communication with upstream bug trackers (and more automation planned for the future).

The report gives a good indication of how well that process is going.

smoke screen?

Posted Oct 2, 2008 5:23 UTC (Thu) by pabs (subscriber, #43278) [Link] (1 responses)

Debian has bts-link, which basically does the same thing, but without the report:

http://bts-link.alioth.debian.org/

smoke screen?

Posted Oct 2, 2008 5:33 UTC (Thu) by jamesh (guest, #1159) [Link]

Sounds cool. I wonder if the Launchpad plugins for Trac and Bugzilla could be extended to allow bts-link to communicate with them?

smoke screen?

Posted Oct 1, 2008 18:31 UTC (Wed) by jspaleta (subscriber, #50639) [Link] (1 responses)

I think I need to give Ubuntu the benefit of the doubt with regard to this.
Metrics are important. Everybody could probably be doing a better job on metrics. This maybe be a valuable first step for Ubuntu to get their head around their upstream interaction and start to build a process to push patches more directly.

What I am more concerned about is the fact that this is yet another piece of functionality which is deeply tied to launchpad, and thus not a tool that can be easily propagated in a way that others can use and more importantly extend.

Even if Ubuntu doesn't know how to turn the corner and make effective use of this sort of reporting to get patches moving upstream, other groups may, or may be able to extend launchpad with better upstream patching support so that launchpad becomes more of a two-way communication conduit. It's unfortunately that Canonical continues to keep their most powerful community interaction tools as implemented as part of launchpad...closed.

Would there be benefit to everyone, both Ubuntu and upstream, if launchpad were openly developed such that people could begin incorporating direct upstream patch forwarding? Like how transifex does things for translations. Canonical's failed to provide those sort of direct upstream interaction features that transifex provides, even though the Ubuntu translation community has been asking for this sort of support for years now. And since launchpad isn't open, the community can't dig in and extend the launchpad functionality for themselves.

And now we have the new metric reporting. It's actually sort of cool, and its actually still not open to be extended, as far as I know. You know, maybe there are community people out there who have the desire and the understanding of upstream project processes that Canonical lacks. Hell, maybe those people are in the current Ubuntu using community. And maybe if launchpad were opened right now, this very second, those people would be able to dig in and extend launchpad specifically to include the direct upstream patch forwarding interactions in a way that makes sense and could be tied to the metrics reporting. But Canonical built launchpad in the way that they did specifically to centralize control into Canonical's hands and did not include a facility to push upstream directly.

So as a result, if others see value in this sort of metric reporting and want to also track upstream patch forwarding, they will have to re-invent the wheel completely, instead of being able to start with launchpad as it is and extend it. What this shows is that Canonical continues to want sole control over what community interaction means, for everybody.

Anyone hear anything newer than August 28th concerning an approved roadmap for opening Launchpad? All the references I have found pre-date the Aug. 28th bug ticket comment stating that the roadmap was not approved yet.

https://bugs.launchpad.net/launchpad-foundations/+bug/506...

-jef

smoke screen?

Posted Oct 2, 2008 4:22 UTC (Thu) by jamesh (guest, #1159) [Link]

Interacting with upstream trackers is a difficult process. If done wrong, it is very easy to shift from being considered a useful service to being considered spam.

So any automated system that started sending automated unsolicited messages to all the community's bug trackers would probably get blocked pretty quickly. So at present Ubuntu bugs are generally manually forwarded upstream by a human.

Once you rule out unsolicited forwarding, the other option is for upstreams to opt in. If they want this, then some plugins were recently made available to help here. At the moment, the plugins allow for comments to bugs to be pushed in both directions for linked bug reports, but in future could allow some level of automation to forwarding bugs upstream (you'd probably still want some human involvement, so massive numbers of duplicates on the distro side don't flood the upstream tracker).

smoke screen?

Posted Oct 1, 2008 18:31 UTC (Wed) by jwb (guest, #15467) [Link] (3 responses)

For a linux kernel developer to complain about any other project's bug tracker is ironic indeed. The standard linux-kernel response to any bug report, no matter how obvious and serious, is as follows. "Could you please test 2.6.42-rc2-pre6-mm17+git20080909153242.8818?" And although the kernel has its own bugzilla, kernel developers seem to ignore it as a policy.

I happen to think Launchpad is very helpful both to developers and users. Users think it is a hassle to 1) find the upstream bug tracker, 2) register for an account there, 3) file the bug, and 4) drive the process through to the end. The fourth item is especially true when the upstream has a bug tracking system that they assiduously ignore. It's very frustrating to file bugs in GNOME's bugzilla or Freedesktop.org's bugzilla or the linux kernel bugzilla, only to see them rot for years on end. At least when you file them in Launchpad you get a good feel for how the distro managers are dealing with the upstream's defects.

Right now I have 53 bugs in Launchpad and only one of them applies to Ubuntu proper. The rest are all upstream bugs, and from my perspective as a user I'm completely satisfied with Launchpad's ability to drive the resolution upstream.

smoke screen?

Posted Oct 2, 2008 8:38 UTC (Thu) by dgm (subscriber, #49227) [Link] (2 responses)

> The standard linux-kernel response to any bug report, no matter how obvious and serious, is as follows. "Could you please test 2.6.42-rc2-pre6-mm17+git20080909153242.8818?"

What else could one say? Most kernel bugs are related to specific hardware (that kernel's developers usually don't have).

> And although the kernel has its own bugzilla, kernel developers seem to ignore it as a policy.

Correct me if I'm wrong, but wasn't the kernel's bugzilla an experiment to see if it helped? (and I think some kernel developers thought it was not worth the hassle).

smoke screen?

Posted Oct 3, 2008 16:32 UTC (Fri) by jengelh (subscriber, #33263) [Link] (1 responses)

Some developers, if not most, don't find it very amusing to register at Bugzilla, especially since there is no way to easily un- and reregister. For them, and that includes me, mail is so much easier.

smoke screen?

Posted Oct 16, 2008 15:30 UTC (Thu) by Duncan (guest, #6647) [Link]

The kernel bugzilla has certainly worked quite well for me. I've filed
and had fixed several -rc bugs that nobody else seemed to be running
into, that may well have made it into the full release were it not for my
report... which might not have been filed had I to try to figure out the
proper mail interface and rules. At least bugzilla is reasonably
familiar for those who run testing and early releases and regularly file
bugs elsewhere, thus already knowing a bit about the process, hopefully
therefore filing better bugs as a result.

Kernel bugzilla therefore expands the feedback loop, opening it to people
whose testing and bug reporting otherwise wouldn't be heard. When the
object is a stable kernel release and the testers are all volunteers,
that's a GOOD thing. =:^)

Duncan

Re: Ubuntu debuts its Upstream Report

Posted Oct 1, 2008 18:39 UTC (Wed) by tajyrink (subscriber, #2750) [Link]

> Maybe it was not polite, but it was based on indisputable numbers.

Hmm, weren't the numbers very much disputable? Omitted one of the most obvious numbers, ie. the number of employees in companies, Canonical's 150 (a lot less before) vs. other's thousands. Blaming a small company for their big success is a bit funny. The grandmas and grandpas using Ubuntu cannot be turned into kernel patches.

I find your post a bit trollish, since whatever all the aspects that have been discussed to death, the issue is very disputable and Greg's ramblings very biased-sounding against his employer's competitor. It's about 50% that Canonical has some problems that could be fixed with time and (hopefully) company growth and 50% that Greg has a bit of a problem with rudeness that should not be tolerated in communities since it creates bad spirit.

Regarding the topic, I find it a welcome start to a more fluent upstream handling. I think previously effort and thinking has gone to interacting with Debian, but it's also useful to try to contribute more directly to upstreams.

smoke screen?

Posted Oct 1, 2008 18:56 UTC (Wed) by whiprush (guest, #23428) [Link]

Hi,

I've been working on this report with Graham Binns for a good portion of this release cycle and it has always been made available at the same public URL. I purposely didn't advertise it much because until recently it wasn't very useful - it is now to a point where it can be useful which is why I blogged about it.

I don't understand where you get the idea that this puts ubuntu users "against" upstream developers, the point of the report is to point out which bug reports are in an upstreamable state so that we can push those to upstream bugtrackers.

--
jorge

smoke screen?

Posted Oct 1, 2008 19:21 UTC (Wed) by apokryphos (guest, #42130) [Link] (5 responses)

Actually, suggesting that they contribute considerably to GNOME in comparison to other companies instead of the Kernel is _actually_ a smoke screen. Companies like Novell and Red Hat employ significantly more developers to consistently work on GNOME and KDE, so if you did a graph with those contributions comparing the different downstream projects, I doubt there would be much difference at all. For example, SUSE have significantly more developers working on OpenOffice.org alone than entire desktop team at Canonical

Greg KH's speech was quite simply that Ubuntu are not the upstream innovators. Critics who complained that he didn't consider the GNOME projects still do not change this fact, and critics who suggest that Canonical is a smaller company are side-stepping the issue. Launchpad is part of the problem, but one of the main problems IMO is one of erroneous attribution. For example I have seen many Ubuntu users suggesting to others that they created Compiz -- a project that actually cost Novell thousands of dollars.

Companies like Red Hat and SUSE should be getting tremendous credit for consistently paying free software developers to work on the Linux server/desktop day-in-day-out, and this so frequently does not happen.

smoke screen?

Posted Oct 1, 2008 20:13 UTC (Wed) by bfields (subscriber, #19510) [Link] (3 responses)

a project that actually cost Novell thousands of dollars.

Might want to count those zero's again....

how many zeroes

Posted Oct 1, 2008 21:12 UTC (Wed) by louie (guest, #3285) [Link] (2 responses)

RH and Novell (and to a lesser but still very significant extent) have invested many millions (probably in the low tens of millions at this point) in GNOME, gtk, X, freedesktop, etc.

how many zeroes

Posted Oct 1, 2008 21:14 UTC (Wed) by jspaleta (subscriber, #50639) [Link]

I wonder... how much has Canonical sunk into launchpad specifically over the years?

-jef

how many zeroes

Posted Oct 2, 2008 19:49 UTC (Thu) by louie (guest, #3285) [Link]

Hrm, that 'and ____ to a lesser but still significant extent' should have a 'Sun' in the blank there.

smoke screen?

Posted Oct 2, 2008 9:17 UTC (Thu) by kripkenstein (guest, #43281) [Link]

> For example I have seen many Ubuntu users suggesting to others that they created Compiz -- a project that actually cost Novell thousands of dollars.

What do a few misinformed users have to do with Canonical or Ubuntu? I've never seen either claim they created Compiz.

smoke screen?

Posted Oct 1, 2008 19:49 UTC (Wed) by flewellyn (subscriber, #5047) [Link]

I see it as precisely the opposite. They are effectively putting themselves on public notice, putting their process of communicating with upstream projects in the public view. This not only encourages others to see exactly what and how the Ubuntu developers are communicating, but also encourages other users and developers to get involved with the process, and encourages the Ubuntu people themselves to make sure they do a good job of it. It's a move towards greater transparency and more public participation, the diametric opposite of a smoke screen.

smoke screen?

Posted Oct 2, 2008 18:16 UTC (Thu) by jsbarnes (guest, #4096) [Link] (1 responses)

As upstream for one of the Ubuntu packages (xf86-video-intel) I can say
that this is definitely not a smokescreen. My interactions with the Ubuntu
packagers have been great. They file good quality bug reports, are
conscientious about getting needed info, and often attach patches we can
apply directly. That's not to say other distros don't do the same (the
Debian, Gentoo and Mandriva guys are also very helpful), but Ubuntu
definitely does the right thing here.

Interestingly Fedora/Red Hat don't interact much with our bug system (i.e.
I rarely see Fedora bugs get reported to the upstream bug tracker), which
is a shame, but that's more than offset by direct developer participation
from the Red Hat guys (people like airlied, mjg59 and ajax put a lot into
our driver).

smoke screen?

Posted Oct 2, 2008 19:27 UTC (Thu) by jspaleta (subscriber, #50639) [Link]

I think this drills down to an important underlying point. Which parties are contributing bits of code into upstream. Isn't that the fundamental friction being expressed?

If Canonical wants to get ahead of the perception problem, they need to be able to track patch flow as well as ticket comment flow. Downstream supplied patched do matter, and may not show up in the the commit stats in the same way that direct developer involvement does. You obviously can't directly compare upstream developer activity done on company time, to volunteer patch submission efforts. So watching how patches come in via a bug tracking interface isn't going to tell the whole story but it may help turn the corner on Canonical's upstream relation issue.

-jef

smoke screen?

Posted Oct 2, 2008 19:34 UTC (Thu) by lysse (guest, #3190) [Link]

> Ubuntu would switch away from gnome to something else if it could in a jiffie.

(k|x|ed)ubuntu

Ubuntu debuts its Upstream Report

Posted Oct 1, 2008 18:35 UTC (Wed) by rahvin (guest, #16953) [Link] (32 responses)

The effort is appreciated, but where does it list that the Ubuntu developer submitted a patch to fix the problem? I thought the complaint wasn't that they weren't tracking bug reports, but more importantly, that they weren't submitting their changes, or any bug fixes upstream.

To me it's one thing to submit a bug report upstream, it's something entirely different to note the bug and submit a patch upstream.

Ubuntu debuts its Upstream Report

Posted Oct 1, 2008 18:56 UTC (Wed) by dlang (guest, #313) [Link] (31 responses)

why do you expect ubunto to submit a patch for every bug they report?

RedHat doesn't supply a patch for every bug that they report to the kernel. sometimes they do, but I wouldn't even say that it's the majority of the time.

taking that attitude that you shouldn't submit bug reports unless you also submit patches is a great way to kill off bug reports, by driving your users away.

nothing in the GPL (or any other opensource license) requires you to create patches, let alone require you to create patches based on problems that people you provided the opensource software to ran into.

they require that you release changes that you do make, and it's strongly encouraged that you push those changes upstream, but if a distro is distributing software and there is a bug in the upstream code I don't see any reason why all the different distros should be expected to create a patch before submitting the bug. in fact I see the idea of them doing so as a massive duplication of effort. if the bug is in the upstream package the bug reports should go to the upstream package where the experts there can look at all the reports from all the different distros and fix the problem once instead of having programmers at the distros who are not familiar with the codebase all scrambling to produce a patch to submit upstream

the fact that launchpad is only a one-direction link is probably as much a limitation of the upstream bug managers as it is of launchpad. none of the upstream bug managers have hooks in them to feed data into launchpad, so they would be reduced to screen-scraping websites (and if they are lucky e-mails) to find changes to pull them back into launchpad.

I think the fact that they can do the one-direction links a big step forward from the common situation with other distros. hopefully the various bug management tools can start creating api's so that they can notify each other about updates. this is a good first step.

Ubuntu debuts its Upstream Report

Posted Oct 1, 2008 19:22 UTC (Wed) by gmb (guest, #54452) [Link] (15 responses)

the fact that launchpad is only a one-direction link is probably as much a limitation of the upstream bug managers as it is of launchpad. none of the upstream bug managers have hooks in them to feed data into launchpad, so they would be reduced to screen-scraping websites (and if they are lucky e-mails) to find changes to pull them back into launchpad.

We on the Launchpad Bugs Team have been aware of this problem for a while. One of the stated aims of Launchpad is to provide a means by which free software projects can intercommunicate, whether or not they use Launchpad for their hosting or bug tracking or translations, etc.

To help resolve this and make the communications between Launchpad and upstream bug trackers two-way, we've recently released plugins for two of the major OSS bug trackers, Bugzilla and Trac. The plugins, which work on Bugzilla 3.0 and above and Trac 0.10 and above, add an XML-RPC API to the upstream bug trackers that install them. Launchpad auto-detects the plugins and can then use that API to:

Synchronise comments both ways between LP and the upstream tracker (so comments on the upstream tracker get imported into Launchpad and replies to those comments in Launchpad get forwarded back upstream).
Tell the upstream tracker which Launchpad bug links to one of its bugs (the upstream tracker will then display a link to the Launchpad bug)
Push bugs up to the upstream tracker (this currently isn't enabled; we want to be absolutely sure that the plugins work as they should before we start pushing bugs upstream).
Automatically import bugs from the upstream tracker (again, this isn't currently enabled for the same reason as the previous entry).

Also, the plugin API makes Launchpad <-> upstream communications a lot more efficient than they are at the moment.

Some upstreams (bugs.xine-project.org, twistedmatrix.com/trac to name two) have already installed the plugins, but we're eager to find more projects willing to help us test them (whilst at the same time hopefully benefiting themselves by making it easier for their users to inform them of bugs via Launchpad).

You can find information about the plugins on the Launchpad help wiki:

For the Bugzilla plugin: /Bugs/BugzillaPlugin
For the Trac plugin: /Bugs/TracPlugin

You can also grab the plugins themselves and file bugs against them through Launchpad:

Ubuntu debuts its Upstream Report

Posted Oct 1, 2008 19:54 UTC (Wed) by zooko (guest, #2589) [Link]

Sweet! I'm totally going to do this for http://allmydata.org/trac/tahoe

please don't violate rules of upstream sites

Posted Oct 1, 2008 20:50 UTC (Wed) by JoeBuck (subscriber, #2330) [Link] (13 responses)

For FSF sites, including bugzilla.gcc.org, links to Launchpad aren't allowed as long as it is non-free software. So please delay implementing that feature until Launchpad is free. If you do it anyway, the GCC web team will have to take any links to Launchpad out again, since we agreed to implement the FSF's linking policy. That policy isn't my preference, and I can't speak for any other projects; Gnome tends to take a looser view, but you'd need to ask them if this is OK. Other projects are likely to object as well.

Alternatively, just make Launchpad free software and all these problems go away.

please don't violate rules of upstream sites

Posted Oct 1, 2008 20:57 UTC (Wed) by dlang (guest, #313) [Link]

as I read the message you would have to install the module in your bugtracker system.

but even if this wasn't the case, how do you justify telling another project not to develop or implement any feature?

Absurd rules of upstream sites

Posted Oct 1, 2008 21:39 UTC (Wed) by tseaver (guest, #1544) [Link] (3 responses)

Such a policy is ridiculous and absurd on its face, because it is contrary to the the fundamental principle of free software, which is that users share information about, and fixes and improvements to the software they use. Must links to CVE entries, for instance, must be blocked as well? How would that help the users running the vulnerable-but-free software about which the CVE had been created?

What about bugs originating in systems where the bug tracker is free software, but which happen to be running on a non-free OS? (How would you know? Why would you care?) To take the point further: should GCC ignore bugs from Mac users, who happen not to run a free OS?

Talk about cutting off your nose to spite your face!

Absurd rules of upstream sites

Posted Oct 1, 2008 22:55 UTC (Wed) by JoeBuck (subscriber, #2330) [Link] (2 responses)

GCC has, from the beginning, supported proprietary operating systems, and it's the primary compiler used on the Mac, though Apple seems to be planning a switch to LLVM.

The policy I was talking about only affects links: an FSF site isn't supposed to link to a page that promotes proprietary software.

Absurd rules of upstream sites

Posted Oct 2, 2008 1:42 UTC (Thu) by tseaver (guest, #1544) [Link] (1 responses)

> The policy I was talking about only affects links: an FSF site isn't
> supposed to link to a page that promotes proprietary software.

Let's get specific. Here is a Launchpad bug page for Zope (the bug is assigned to me):

https://bugs.launchpad.net/zope2/+bug/143422

In what way does that page "promote proprietary software"? Its entire purpose is to help improve a piece of *free* software (Zope); the only mentions of Launchpad are the home page link (the logo at the top), and a "Help us improve Launchpad" link at the very bottom.

The report contains valuable information, including a patch, about a bug in an application I help maintain. Rejecting a link to this page because LP happens to be closed would be a ludicrously irresponsible policy. Making the report and patch inaccessible to the maintainers of a putative upstream application which applied such a policy would be a disservice both to the maintainers and to the users of the software.

Note that I *do* wish LP were opened up, as I believe its value would be increased. I *don't* think rejecting a link to a bugreport in LP would be sane: it would actively harm the free software ecosystem.

Absurd rules of upstream sites

Posted Oct 2, 2008 17:41 UTC (Thu) by JoeBuck (subscriber, #2330) [Link]

Launchpad is itself proprietary software. I didn't make the rule, I'm just telling you what the rule is. Directing someone to Launchpad is seen by the FSF as promoting Launchpad.

Canonical has promised to free Launchpad, and when they do, this issue will be moot.

please don't violate rules of upstream sites

Posted Oct 1, 2008 22:24 UTC (Wed) by gmb (guest, #54452) [Link] (1 responses)

For FSF sites, including bugzilla.gcc.org, links to Launchpad aren't allowed as long as it is non-free software. So please delay implementing that feature until Launchpad is free.

I'm sorry, perhaps I wasn't clear in my original post. For this functionality to work with a Bugzilla or Trac instance, that maintainers of that instance need to install the requisite Launchapd API plugin. Without the plugin, the API isn't available. Since we can assume that bugzilla.gcc.org won't install the plugin (at least until Launchpad is released as free software), this isn't a problem.

Alternatively, just make Launchpad free software and all these problems go away.

We're working on it ;)

please don't violate rules of upstream sites

Posted Oct 1, 2008 22:49 UTC (Wed) by jspaleta (subscriber, #50639) [Link]

Do you have a publicly communicated roadmap later than Aug 28, 2008?
The last thing I read in launchpad's bug tracker is that the roadmap has not been approved.. and I can't seem to find a blueprint or anything describing the roadmap.

If you have an approved roadmap or blueprint or whatever you want to call it....publish it... and provide a link here please.

-jef

please don't violate rules of upstream sites

Posted Oct 2, 2008 7:59 UTC (Thu) by epa (subscriber, #39769) [Link] (4 responses)

What does it mean for a website to be free software? I cannot make a
change to the code for bugzilla.gcc.org and start running it on the site.
I can download a tarball of the Bugzilla source code, but the
bugzilla.gcc.org site remains unmodifiable by me; it is no more free
software than Linux running on a TiVo.

The only people who have the freedom to change bugzilla.gcc.org for their
own requirements are the site's maintainers. And indeed the maintainers of
the Launchpad site have freedom to change their own code, run modified
versions, and even share it with others if they choose to. They currently
choose not to share it, but nothing in RMS's philosophy forces you to
distribute your code to others - just that if you do distribute it, the
others must have the same rights you do. If I made my own private version
of Apache with a few changes, and ran it only on my own website without
giving a copy to anybody else, would the FSF policy forbid linking to that?
What about linking to a website that uses a couple of CGI scripts? Must
those scripts be published somewhere first?

If the policy is only to link to websites implemented using free software
which is also available for public download, that is consistent, but
extremely strict even by FSF standards. Surely even the FSF's own site
uses some scripts and configuration files they have not made public.
Presumably it would rule out linking to a Bugzilla page that happens to be
hosted on AIX or on Linux running inside a non-free virtual machine like an
IBM mainframe.

What does it mean for a website to be free software?

Posted Oct 2, 2008 21:46 UTC (Thu) by grantingram (guest, #18390) [Link] (2 responses)

What does it mean for a website to be free software?

I think that is an excellent question! I've never quite understood the objection to Launchpad - as far I can see it is a web site. Though admittedly I only report bugs in it.

I generally don't demand the source code to websites that I read (like LWN for example!) but perhaps there is a point where the website becomes complex and intermingled enough with your own data that one should... I'm not sure where or how I would draw the line though.

What does it mean for a website to be free software?

Posted Oct 3, 2008 9:22 UTC (Fri) by hppnq (guest, #14462) [Link] (1 responses)

Take a look at the GNU APGLv3. It is meant to address the issue that software (not so much data) can be "locked up" in a website.

What does it mean for a website to be free software?

Posted Oct 3, 2008 9:27 UTC (Fri) by hppnq (guest, #14462) [Link]

What's in an acronym: AGPLv3 of course.

please don't violate rules of upstream sites

Posted Oct 9, 2008 20:07 UTC (Thu) by oak (guest, #2786) [Link]

News at FUD.com: RMS "Tivo"s FSF bugzilla; users can get the sources to
it, but they cannot install their modified sources/binaries there! :-)

please don't violate rules of upstream sites

Posted Oct 9, 2008 11:21 UTC (Thu) by azrael (guest, #53640) [Link]

"For FSF sites, including bugzilla.gcc.org, links to Launchpad aren't allowed as long as it is non-free software."
Man, I've heard that FSF is paranoid but that is just plain retarded.
I think they should also not link to Google, cause that's not free software too.

Ubuntu debuts its Upstream Report

Posted Oct 1, 2008 20:55 UTC (Wed) by JoeBuck (subscriber, #2330) [Link] (12 responses)

"why do you expect [Ubuntu] to submit a patch for every bug they report?"

The expectation is that they (and all distros) will submit a patch upstream for every bug that they fix, as well as to communicate with upstream about any confirmed bug that upstream doesn't know about. The situation to avoid is where a distro (any distro) fixes bugs for its own users and doesn't work to get them upstream so that others benefit as well. Upstream might disagree with a given fix, of course.

Sometimes this can't happen immediately, but if all distros do this, then all distros benefit.

Ubuntu debuts its Upstream Report

Posted Oct 1, 2008 20:58 UTC (Wed) by dlang (guest, #313) [Link] (11 responses)

that's not what people are saying.

they are complaining the ubuntu is making bug reports without also sending patches to fix them, they aren't claiming that ubuntu is making fixes and refusing to submit them.

Ubuntu debuts its Upstream Report

Posted Oct 1, 2008 21:18 UTC (Wed) by nevyn (guest, #33129) [Link] (10 responses)

No, they have said that Canonical/Ubuntu doesn't participate in upstream development. This says nothing about the number of bugs they send upstream, or how much development they do for themselves. I've heard that they do quite a bit of work on their packages and never bother pushing it even to debian -- but that is a side argument.

Personally I wouldn't care much if they did 10x the development they pushed upstream, locally, and 100x the bugs than they did fixes ... as long as they did their "fair share" of the upstream work, and generally stopped taking credit for the work Red Hat, IBM and Novell are doing for them.
This is mainly an economics argument, in that if they keep getting all the credit/mindshare and thus. a sizable number of the users ... development will freezeup as Red Hat/Novell will not be able to afford to spend money they aren't making back.

Ubuntu debuts its Upstream Report

Posted Oct 1, 2008 22:04 UTC (Wed) by kragil (guest, #34373) [Link] (9 responses)

??? Logic ??? I really fail to see your reasoning.

Canonical isn't making money. They loose lots of money .. (on a pretty consistent basis so far)

So up until now Ubuntu is essentially a charity organisation that sends out CDs to everyone and develops a _very_ popular distro.

RH and Novell (mostly due to the help from MS though) are making lots of money on the other hand. So it is only fair that they do more development.

That is how the system should work IMNSHO.

Ubuntu debuts its Upstream Report

Posted Oct 1, 2008 22:45 UTC (Wed) by jspaleta (subscriber, #50639) [Link] (4 responses)

Let's connect some dots....
Ubuntu is popular....
Canonical controls the core infrastructure that makes it possible for Ubuntu to be popular....
That infrastructure that Ubuntu relies on is not currently replicable without Canonical as it is closed source....
Canonical is not profitable, nor do we know for sure if its on a path to profitability....

Draw your own conclusions about the sustainability of what Canonical's approach to managing community interactions. The situation with translation handling continues to be a hallmark example. Ubuntu specific improvements such as translations are bound up in that closed infrastructure and are not flowing into upstream projects as smoothly:

http://brainstorm.ubuntu.com/idea/125/
http://brainstorm.ubuntu.com/idea/13571

And that's just upstream translations, which arguably is one of the lowest hanging fruit of community and upstream interaction that Ubuntu's popularity could be leveraged to help with. It stands to reason that Ubuntu's wide popularity translates into an army of volunteer translators across a multitude of languages.

But those volunteer efforts are not flowing easily into upstream projects because launchpad is specifically designed to aggregate contributions and not to redistribute them back to upstream. If Canonical was actually interested in helping create a conduit between community effort and the upstream projects this problem would have been solved as part of launchpad's design...years ago. Canonical has created this problem by deliberately designing launchpad to try to be a central service to everybody's workflow. To make use of downstream translations, upstream projects have to pull from launchpad and thus rely specifically on Canonical. Canonical could have designed launchpad to work with upstream projects directly and push translations into upstream's processing.
What if launchpad had been originally developed with upstream coordination in mind? What if launchpad were open to contribution from day one? Would the Rosetta component have functioned like transifex does now?

If Canonical was an actual charitable organization, they'd have no compelling reason to keep launchpad closed and Ubuntu community volunteers like their translators would be free to build the tools and the workflows they need without having to worry about Canonical's service based business model.

Speaking of charitable organizations...isn't there a Ubuntu Foundation on the books...does that organization do anything day-to-day month-to-month or was it just a PR stunt?

-jef

Ubuntu debuts its Upstream Report

Posted Oct 1, 2008 22:55 UTC (Wed) by hppnq (guest, #14462) [Link] (3 responses)

Let's connect some dots....

Let's not. Let's ask for a release of the LWN site source code, so someone could say implement a reasonable comment filter.

Ubuntu debuts its Upstream Report

Posted Oct 1, 2008 23:00 UTC (Wed) by dlang (guest, #313) [Link] (2 responses)

there is a greasemonkey script that lets you do exactly that.

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 6:05 UTC (Thu) by madscientist (subscriber, #16861) [Link]

Not that I don't appreciate the effort that went into that GM script, but it is in no way a replacement.

Just as an example: I regularly use at least 3 different computers, with 3 different installations of FireFox, separate bookmarks, cookies, etc.

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 8:56 UTC (Thu) by hppnq (guest, #14462) [Link]

To be honest I do not always pass my own filter. ;-)

To this long time Red Hat user, the continuous Ubuntu bashing here is just becoming a bit boring, and frankly, it is looking pathetic. Talking a lot about what could be done distracts from the important stuff that is actually being done. People are of course free to determine whether something is important to them or not, and it is mainly the lack of respect for this important foundation of free software that annoys me.

For people who remember the discussions about the test bed Red Hat created with Fedora and the way Novell is dealing with Microsoft, the whining about Canonical or Ubuntu not contributing enough to the community is a very bad joke.

More on topic: to me it seems that the correct tooling is essential for getting the work done, so I welcome this very interesting contribution by Ubuntu.

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 2:27 UTC (Thu) by nevyn (guest, #33129) [Link] (3 responses)

Canonical isn't making money.

Not true.

They loose lots of money .. (on a pretty consistent basis so far)

Possibly true, although as a private company only they know.

So up until now Ubuntu is essentially a charity organisation that sends out CDs to everyone

So you agree, they are spending money ... just not on development.

and develops a _very_ popular distro.

I think we've established that they mostly market a distro. ... and as to how popular it is, well popcon vs. smolt stats. are pretty close. So let's just say "one of the popular" distros.

So you're argument (or "logic" as you say) is that it's fine for them not to participate, even though they are one of the popular distributions, because they are spending that money on PR/marketing ... but it wouldn't be fine for Red Hat or Novell to just stop paying developers to participate upstream and instead just spend that money on PR/marketing?

Or maybe you're just saying it's fine if noone grows the pie but just spends all their money on trying to grab as big a slice of the current pie as possible?

That's fine, I guess, you can have that opinion ... but given that I actually like Linux, and want to see it improve faster (and I'd kind of prefer having a job) personally I'll choose option B. Which is pointing out how Canonical are hurting the community and need to change. As did Greg, probably for the same reasons.

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 4:13 UTC (Thu) by dlang (guest, #313) [Link]

if you don't think that marketing a distro, especially to people who have trouble with existing distros isn't growing the pie then you are being blind.

it's not a zero-sum game, ubuntu's gain doesn't directly cause RedHat loss

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 4:39 UTC (Thu) by jspaleta (subscriber, #50639) [Link]

" Canonical isn't making money.
Not true.
They loose lots of money .. (on a pretty consistent basis so far)
Possibly true, although as a private company only they know."

As of May 22, 2008 Mark was quoted in an interview as saying that Canonical was "not close" to breaking even. I can't imagine a more qualified opinion on the matter than Shuttleworth.

http://www.guardian.co.uk/technology/2008/may/22/internet...

This interview is also interesting to note that because Mark talks about the future of software as "unlicensed" software. That is a bit of an odd way of describing FOSS if that was his intent. Everywhere else I see the phrase "unlicensed software" used it is used in the context of some sort of licensing violation. I'm pretty sure that Mark doesn't mean to imply that the future of software is going to be the use software in violation of the license that it is offered under.

-jef

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 8:21 UTC (Thu) by jschrod (subscriber, #1646) [Link]

> > and develops a _very_ popular distro.
> I think we've established that they mostly market a distro.

Who established that? you? Then you're part of a vocal group of people, but where others (me included) take objection to that opinion. Ubuntu develops a popular distribution by integrating different packages for a target audience of computer non-experts desktop users better than their competitors. This integration and smoothing work is *not* just marketing -- and it doesn't matter that you and others want to debase that important work that's part of our Linux eco-system.

For the record, I am not involved in Ubuntu and don't even use it. (I installed it once in a VM and recognized after a few weeks that it's not my style of distro.) But I've met many people who started to use Linux because of Ubuntu -- they tried RH and SUSE before, and Ubuntu was the first Linux distro that »did things right« for them. And just because of those newbies starting to use Linux, Ubuntu is a Good Thing(tm).

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 16:12 UTC (Thu) by dberkholz (guest, #23346) [Link] (1 responses)

why do you expect ubunto to submit a patch for every bug they report?

How does tracking a useful metric imply that it's expected to reach 100%? Keeping track of one additional piece of data could make a big step toward disproving the common thought that Ubuntu doesn't contribute enough upstream.

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 16:31 UTC (Thu) by dlang (guest, #313) [Link]

Ok, I misread the question.

I read it as an attack based on the fact that a ubuntu bug gets linked against an upstream bug without submitting a patch.

if it was intended to only be a request to document if such a patch was generated then it's a very different story.

however I would expect that the percentage of linked bugs where a patch is provided would be _very_ low. I think this is very reasonable, ubuntu (or any other distro) may produce a patch, but most of the time they shouldn't.

Ubuntu debuts its Upstream Report

Posted Oct 1, 2008 22:47 UTC (Wed) by hppnq (guest, #14462) [Link]

... but the focus is clearly on improving Ubuntu, as opposed to improving the software ecosystem that makes up the distribution.

Now are they still trying to improve that?! As opposed to making as much money as possible out of our nice communist-hippie-meets-Wall-Street Open Source charity, sorry, Linux distribution ecosystem?

I don't think it is true for Ubuntu, but what would be wrong with being "just a consumer" of free software?

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 6:57 UTC (Thu) by quozl (guest, #18798) [Link] (7 responses)

I'm upstream for pptp-linux and pptpd. I had cornered Mark at a linux.conf.au some years ago complaining that Ubuntu were fixing things and not passing anything back to me, and he said he'd look into it.

Your article here has encouraged me to have a quick look ... the Ubuntu bugs for pptp-linux are quite extensive, and I feel cheated, I didn't know there was such a rich body of input for me. I'll be busy for some time now. I only wish they had been passed upstream more slowly!

Hint for upstreams ... create an account on launchpad and ask it to deliver you bug reports. Do Ubuntu's work for them. Ignore the bugs that are obviously distribution specific. Run multiple bug trackers. Sigh.

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 9:32 UTC (Thu) by johill (subscriber, #25196) [Link] (1 responses)

... and soon any upstream project will be tracking a dozen different bug trackers. Have fun.

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 16:13 UTC (Thu) by dlang (guest, #313) [Link]

if you don't like integrating multiple bug trackers you have the following options.

not allow the distros to have their own bug trackers and force users to take everything directly to the upstream bug trackers (including distro specific bugs)

the current status-quo where you have a lot of independant bug trackers and no linkage between them.

it's far better to have the upstream bug trackers tied in with dozens of distro bug trackers.

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 13:49 UTC (Thu) by markshuttle (guest, #22379) [Link] (2 responses)

If you run Bugzilla or Trac, you might want to try the plugins that Canonical sponsored, which should let comments flow in both directions between Launchpad and your bug tracker. That will make coordinating existing reports that are appropriately linked much easier.

We haven't yet added the ability to pass bug reports to another bug tracker automatically, for fear of spamming upstream bug trackers. If you can find a community member who is particularly interested in pptp and Ubuntu then they could easily play that role, as there isn't a dedicated person maintaining pptp in Ubuntu.

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 18:24 UTC (Thu) by jsbarnes (guest, #4096) [Link] (1 responses)

I actually prefer having a triager from Canonical/Ubuntu deal with
forwarding information on from Launchpad by hand. IME Launchpad issues are
often filed directly by (sometimes nontechnical) users, which is great in
terms of accessibility and openness, but can create a lot of noise that
distracts from the underlying issue. I don't know how many times I've
looked at a Launchpad issue and see a user say something like, "Oh yeah I
see this problem too, only [I'm running totally different hardware and
software and seeing a different issue entirely]". :)

The downside is that responsiveness is bottlenecked on the triager, but
given that the packager often has to spin a debug or patched version of the
software to get new info anyway, I don't think that's really much of an
issue.

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 20:26 UTC (Thu) by bryce (guest, #16388) [Link]

I think the comment forwarding would be the most valuable thing here. I agree with Mark and Jesse on the cons of auto-upstreaming bugs.

Comment forwarding would remove the triager as the bottleneck for communication between the reporter and upstream. I make it a policy to always ask the user to register with the FDO bug tracker when I upstream their bug, but unfortunately they do so only about half the time. The other half, I'm in the position of cut-and-pasting questions and replies between upstream and the reporter.

With automatic bug upstreaming, in theory it sounds like a great idea, since the bug upstreaming process takes a considerable amount of time. But in practice it's far from a simple matter. Like Jesse points out, many ubuntu bug reporters aren't knowledgeable in what to include to make a useful report, or add noise to the bug report that distracts from the core issue. So an important step in upstreaming a bug is revising the description, condensing out irrelevant comments, ensuring that logs, backtraces, and so forth are present and relevant to the issue, etc. None of this is easy to do with a script...

However, I do think there are aspects that can be coded around to make portions of the process easier. To pick just one example, in Intrepid we're introducing Xorg-specific hooks for the apport bug reporting toolset, so to file a ubuntu bug against xorg, just run 'ubuntu-bug xorg'. It automatically retrieves and includes xorg.conf, Xorg.0.log, gdm logs, lspci output, and so on. Should save the reporter lots of time, and eliminates back-and-forth between the reporter and the triager. Stuff like the intel reg dumper tool or other special package-specific tools can be hooked in as well.

Ubuntu debuts its Upstream Report

Posted Oct 2, 2008 19:24 UTC (Thu) by tzafrir (subscriber, #11501) [Link]

I'm partially upstream and partially Debian maintainer for a certain package.

The Ubuntu maintainers for that package and similar packages are the "Masters of the Universe". Those great masters never seem to triage bugs and forward bugs upstream. Debian packages should (and mostly do) follow up on issues and update upstream. The Ubuntu Universe packages tend to have no real maintainer.

And it is not the job of upstream to triage Ubuntu bugs. Upstream is not qualified to guess that Ubuntu Beastie Barbie is actually ubuntu 5.999 and this naturally implies libc6 2.11, libken 1:5.21 and liberty 3.14159 . Yes, those would be trivial to find from http://packages.ubuntu.com, but does upstream know about that? Does upstream know that it is actually a good idea to ask about sources.list because the user might have tried to upgrade?

Upstream should not need to face the specific quirks of each distribution. This is what packagers / power users are for.

Ubuntu debuts its Upstream Report

Posted Oct 9, 2008 11:25 UTC (Thu) by azrael (guest, #53640) [Link]

"I'm upstream for pptp-linux and pptpd. I had cornered Mark at a linux.conf.au some years ago complaining that Ubuntu were fixing things and not passing anything back to me, and he said he'd look into it.

And you would expect Mark to check every single upstream project if they are receiving the patches from Ubuntu? Please...

Why didn't you contact with Ubuntu developers/maintainers directly? A single mail to ubuntu-devel-discuss@lists.ubuntu.com would be probably enough.

Ubuntu bug triage

Posted Oct 3, 2008 0:01 UTC (Fri) by giraffedata (guest, #1954) [Link]

I don't think what's being called "triage" here is triage. I think it's just sorting or dispatching or screening.

Triage is a specific kind of sorting where you sort problems into three categories (hence the "tri"): 1) problems you won't work on immediately because they can wait; 2) problems you won't work on immediately because you can't fix them; 3) problems you will work on immediately.

It comes from battlefield medicine.

"closed source" Launchpad

Posted Oct 9, 2008 11:38 UTC (Thu) by azrael (guest, #53640) [Link]

People seem to not understand the whole concept of open source licences.
Most of these licences (AGPL excluded) are in effect only if you distribute the software (in source or binary form). Canonical _doesn't distribute_ Launchpad. They just run it on their servers. So it doesn't matter if Launchpad is closed or open. It could be even GPL3 already. Canonical is just NOT distributing it, so they don't have to distribute the source. Same goes for Google, Sourceforge, etc.
So stop complaining because it's just plain jealousy. They have the right to not distribute their own creation.

If anyone asks: I wrote a master thesis on open source licenses (and passed the final exam).