Assesment [LWN.net]

Assesment

Posted Aug 5, 2009 16:41 UTC (Wed) by atoponce (guest, #57402)
Parent article: Shuttleworth: On cadence and collaboration

I agree generally with what Mark is trying to accomplish. It's clear he wants two things:

1. Single collaboration from all the distributions on a single version of softawre.
2. Ubuntu hatred removed from the Debian project.

Both should be applauded, but the details tell a different story.

First, operating systems package software, apply their branding, patches, etc, and ship it with the distribution. As a result, users shouldn't be going upstream for bugs, as much as they should be going to the distribution. THEN, the package maintainer should take the concerns upstream. After all, he took a specific package version, and should be able to communicate to the developer clearly.

Imagine if every distribution sourced a single upstream version. First, each operating system will package the software (differently, that is- RPM, TAR.GZ, DEB, etc) with their patches that work with their operating system. They'll apply their branding, and such. At this point, it's not the same as upstream. It's already different. So, where do the end users go for support? If they go upstream, it might just be a packaging issue, perhaps a conflict with another package. If they go to the package maintainer, then we're back to square one, where no one benefits from collaboration.

It's a bit short sighted, if you ask me. The concern really shouldn't be about collaborating packages together, but when building packages from upstream, not creating mini-forks, if you will. Keep it pristine. I should be able to take a DEB for Ubuntu, and install it on Debian. I should be able to take apart the DEB, and build an RPM that installs on Fedora, RHEL or even SUSE. Hell, I should be able to make a TAR.GZ, and install on Arch or Slackware.

No, the issue isn't distributions collaborating interdependantly, a noteworthy goal, but the distributions should be applying as little as possible to the packages to make it as close to a specific upstream version as possible. This is where FreeBSD and Arch shine. It's the upstream source, packaged in a tarball, and compiled|installed on the end user's machine.

Assesment

Posted Aug 5, 2009 17:45 UTC (Wed) by bfields (subscriber, #19510) [Link] (1 responses)

Imagine if every distribution sourced a single upstream version. First, each operating system will package the software (differently, that is- RPM, TAR.GZ, DEB, etc) with their patches that work with their operating system. They'll apply their branding, and such. At this point, it's not the same as upstream. It's already different. So, where do the end users go for support? If they go upstream, it might just be a packaging issue, perhaps a conflict with another package. If they go to the package maintainer, then we're back to square one, where no one benefits from collaboration.

I think you're trying to make a black-and-white distinction out of something that's actually a continuum. Making packages more similar to upstream may still improve the opportunities for collaboration between distributions even if it doesn't eliminate *every* distro-specific issue.

Sure, you probably can't eliminate distro-specific package maintainers entirely, but you may be able to simplify their job significantly: for example, if they find a bug in their package, and report it upstream, there may be a greater chance that their report matches a report from another distro, and hence that work done on one benefits both.

Assesment

Posted Aug 5, 2009 18:21 UTC (Wed) by tajyrink (subscriber, #2750) [Link]

Indeed, ideally there are no distro-specific patches, and part of every maintainers' job is to reduce the delta. I wouldn't say Fedora version of a typical package is really much different from Debian version. It's a bit different for huge ones like OO.o (with Sun and go-oo...), but still. Every difference from upstream means more maintaining burden for the distro, which is best avoided.

Assesment

Posted Aug 5, 2009 20:32 UTC (Wed) by dmaxwell (guest, #14010) [Link] (2 responses)

I look at Ubuntu, and the contributions they've made to GNU/Linux, and it's impressive. However, they've also diverged so much from upstream, that it's difficult to take the Ubuntu packages and install them on a Debian system, or otherwise. It's becoming wholly Ubuntu-dependent.

Largely true but I'll note a bright spot. The way Debian source packages are handled and installed aren't very different between the two. I won't pin an Ubuntu binary package into a Debian machine or vice versa but I often build source packages from one on the other. For instance, at one point Ubuntu had the most recent release of SpamAssassin but neither Volatile or Backports had it for Stable. So I rebuilt the Ubuntu source packages for it and installed that on my stable machine. It worked a treat. All the config and init files went where they should and it basically Just Worked. Sure I could convert and build say a Fedora package but the chances of it working correctly without a lot of surgery were not good.

This basically is the same process used at backports.org to bring newer stuff to Stable in a sane fashion.

Conversely, I'll sometimes see something tasty in Sid and build it for one of my Ubuntu desktops.

It isn't too awful to do usually:

1. Put a deb-src line from the "foreign distro" into sources.list. Note well "deb-src" ONLY.

2. apt-get build-dep package_of_interest.

The things you need to build the package come from your native distro and this is what will keep what you're building from pulling in uptizillion foreign libs and doing things like replacing your libc. If this fails then you'll need to apt-get build-dep additional-dependency and the apt-get source additional-dependency and build and install it first. I don't bother if this part starts getting ridiculous. Building something from say Jaunty on a Stable that is two years old increases the chance of excessive pain at this point. If your porting from a current Sid to Jaunty as of today then you'll probably have everything you need in your own binary depos.

3. apt-get source package_of_interest
4. cd package_of_interest_source_dir
5. fakeroot dpkg-buildpackage -b

If all goes well an installable port of the package will show up one directory upwards.

Assesment

Posted Aug 6, 2009 14:24 UTC (Thu) by tdwebste (guest, #18154) [Link]

I agree with everyone here, syncing up release dates doesn't do much to improve collaboration between debian and ubuntu. What is need is collaborative bug tracking and package source control.

The Ultimate Debian Database helps show ubuntu/debian package relationships.
http://udd.debian.org/
http://wiki.debian.org/UltimateDebianDatabase

Unfortunately Debian and Ubuntu use incompatible bts systems. Currently Ubuntu's launchpad based bug tracking system _bts_ is lacking accessible package version information. Bug tracking information tied to package version is essential for debian where packages go through many version iterations between releases.

This was part of the original design for Launchpad Bugs, but it never came to fruition. The very earliest bug still open on Launchpad Bugs asks for this:
https://bugs.launchpad.net/malone/+bug/424

Up to and including Hardy, ubuntu used apt-listbugs which referred to debian's bts with package version tracking. Even though this pulled bug information from the debian bts, it gave a reasonable indication of what packages contained significant bugs. apt-listbugs was withdrawn, because ubuntu package customization increasing has made the related debian bts irrelevant.

Topic branches and trees are ways by which package customization can be tracked.

In order to meet the Debian Collaboration Team's objective the launchpad bts must interface with the debian bts. Only this way developers benefit from the topic branches, trees of distributed package source control. To collabate bugs must be tracted across both debian and ubuntu and be accessable to both native debian and ubuntu developers.

Assesment

Posted Aug 6, 2009 21:57 UTC (Thu) by langagemachine (guest, #56890) [Link]

Sorry, did not follow you here (apt newbie inside) :
> The things you need to build the package come from your native distro and this is what will keep what you're building from pulling in uptizillion foreign libs and doing things like replacing your libc.

So, what will happen if the package depends on other libs, in a more recent version than installed on the system ? Will these be fetched from source, then locally installed ?

Assesment

Posted Aug 5, 2009 20:44 UTC (Wed) by jspaleta (subscriber, #50639) [Link] (8 responses)

"Let upstream determine the version that the distribution maintains."

So what does "maintain" mean when you complicate the packagescape with optional mini-repositories like Ubuntu specific PPAs? Aren't those PPA's "maintained?" They may not be official but does that matter to end users who are configuring their system to use PPAs? Doesn't the existence of PPAs break some of the fundamental assumptions here about the benefits of sticking with a specific upstream release. There are a lot of Ubuntu binaries built in launchpad PPAs, no one is talking about versioning-locking the PPA space are they? How many Ubuntu users configure at least one optional Ubuntu PPA for something in order get a newer version of some application? How widespread is the use of PPA binaries on Ubuntu LTS releases?

If Shuttleworth were really serious about the benefits of cross-distro version-locking, then I think he would need to rethink the policy on how PPAs are allowed to be used in the scope of just Ubuntu because they greatly undermine any sort of concept of an upstream preferred version which get priority attention.

-jef

Assesment

Posted Aug 5, 2009 21:25 UTC (Wed) by beuno (guest, #44010) [Link] (2 responses)

People will install random software on their computers no matter where it comes from. Be that getdeb.net, random debs or building from tarballs. This has absolutely nothing to do with the issue at hand. PPAs address (and improves) those use cases, as well as helping developers and cutting-edge users test out new versions to stabilize them.

The majority of users (way above 90%) will never use a PPA, or install newer versions of the software that's available by default or in the official repositories.

This means that whatever gets frozen in the archive is what the vast majority of the users experience, and distribution developers focus their time on those packages. This is what the efforts are geared towards.

PPAs have nothing to do here. You are plain out wrong and sensationalist.

Assesment

Posted Aug 5, 2009 22:19 UTC (Wed) by jspaleta (subscriber, #50639) [Link] (1 responses)

I am sensationalist..that is true. I very much enjoy sensations.. don't you?

90% of all users will never touch a PPA? Is that an emotional shoot from the hip estimate or is that a fact based analysis? I'd rather like to avoid emotional statistics if at all possible. So if you can back that up with some verifiable data analysis, please do so.

I do not question that PPAs serve a useful purpose. Having users out there making use of each and every version released by an upstream developer is a very good thing. One could argue that having as many users as possible testing the newest releases as they become available maximizes the benefit to upstream developers. This is at odds with the stated benefits of cross-distro syncing for "preferred" versions. If every release that upstream developers make needs testing..then every release needs to find its way into the hands of users for widespread testing and feedback. Having distributions stagger what they distribute is one way to see a continuum of release testing.

If all the major distros version lock you are more likely to get boom-bust testing cycles where a lot of bugs go unnoticed across multiple releases instead of a flow of bugs and fixes for each upstream release. The natural feedback loop of the release early release often model is at odds with the concept of preferred version-locking.

Perhaps the reality is the fact that version locking that some distributions feel compelled to do for stability reasons is the underlying problem and not the solution. Since upstream development for many projects moves at a fast clip, the multi-year promise by distributions to keep versioning static retards the natural feedback cycle of release early release often that upstream project development makes use of.

-jef

Assesment

Posted Aug 6, 2009 5:44 UTC (Thu) by dfarning (guest, #24102) [Link]

>I am sensationalist..that is true. I very much enjoy sensations.. don't you?

I guess that depends on ones goals. Do you want to be perceived as a developer who looks at the technical pros and cons of a decision or as a marketer gathering support for your product.

One role requires sensationalism the other does not.

david

Assesment

Posted Aug 7, 2009 14:34 UTC (Fri) by daniels (subscriber, #16193) [Link] (4 responses)

Let's rephrase: 'if Red Hat were serious about RHEL, they'd ban Livna, rpmfind, and people.fp.o from existence'.

In other words, what are you on about?

Assesment

Posted Aug 7, 2009 16:26 UTC (Fri) by jspaleta (subscriber, #50639) [Link] (3 responses)

That's just silly talk. Red Hat doesn't control these other repositories so there is no way Red Hat could be expected to speak for those organizations if Red Hat were to agree to cross-distro version locking.

Canonical most definitely control PPAs as they are part of the Launchpad service running on Canonical funded infrastructure. Canonical could turn off the Ubuntu specific PPAs tomorrow. And of course there are the Canonical's OEM specific partner repositories for Ubuntu pre-installs.
I just want it to be clear as to what Canonical is actually willing to agree to from their end as to how far version locking will go and how big an impact that will have on end-user.

Canonical could easily agree to some sort of cross-distro version locking on core components in main and then break the spirit of that agreement by making use of the PPA infrastructure they control and encouraging users to pull enhanced versions of core components from PPAs or OEM partner repositories they directly control.

But you are right, they don't need to use PPAs to break the spirit of any cross-distro version locking agreement. They could also ship multiple versions of core components in Ubuntu proper like they are planning on doing with the kernel in order to get Android emulation support out to people in Karmic. Would shipping an optional 2.6.29 kernel with Android enhancements be considered a breach of a cross-distro version-locking agreement if everyone agreed to shipping a 2.6.31 version?

-jef

Assesment

Posted Aug 7, 2009 17:42 UTC (Fri) by daniels (subscriber, #16193) [Link] (2 responses)

Sigh. I guess by that logic, OpenSUSE (sorry, Novell -- I forgot about the obsession with the controlling companies) are responsible for everything built with their build service, so they're definitely out. And Red Hat definitely control people.fedoraproject.org, so they're out as well.

As for the rest of the comment, it seems to be your standard 'someone said something about Ubuntu, let's slam them on ten thousand semi-related points and see how often I can use the word Canonical and bring up their financial backing of Ubuntu' spiel. Getting pretty dull now.

You seem to have extrapolated from 'let's get the main distributors to agree on which versions they'll ship as part of their primary releases' to 'anything put on a private package repository is part of the Ubuntu release and Canonical is responsible and this is why it's going to all fall apart'; at least, that's the most coherent summary I could come up with. Words honestly fail me. If you're actually genuinely confused about this and not just coming up with reasons to slam Ubuntu into the ground once again, you might want to check up on what a release actually comprises of. Hint: it's not random stuff on the web that happens to share the same base domain.

Anyway, by the same logic, Rawhide is not allowed to exist. Have fun selling that one.

-daniels, running Fedora on his primary laptop for the time being and Debian on his other machines, not affiliated with either Ubuntu or Canonical, etc, etc

Assesment

Posted Aug 7, 2009 18:33 UTC (Fri) by jspaleta (subscriber, #50639) [Link] (1 responses)

Novell isn't pushing for cross-distro syncing publicly. If they went on record supporting the idea...then yes I would want to know how they viewed the role of their build service in light of any sort of syncing agreement. Because their build service would be just as useful to break the spirit of a 2 year cadence agreement. But thanks for bringing it up. It points to how difficult such a cadence would be to maintain in practise. As Novell derives more revenue from Studio in the future, breaking the spirit of such a syncing agreement with customized Suse remix appliances could essentially nullify some of the stated benefits to upstreams because we'd still see a continuum of versioning in long term deployments. Especially if Novell chooses to support some of those remixes with enterprise level support.

But this isn't Novell's idea...its Shuttleworth's meme. If Canonical doesn't think that PPAs nor OEM repositories that they control would be considered part of the agreement, that should be said as early on as possible to prevent any later re-interpretation.

Let's be clear at the outset as to what the boundaries are for each distributor....no implicit assumptions. It would be far far worse if one of the other distros who decided to work with Shuttleworth on this cried foul about Canonical controlled addon repositories after the agreement was in place.

-jef

Assesment

Posted Aug 8, 2009 8:15 UTC (Sat) by daniels (subscriber, #16193) [Link]

If Canonical doesn't think that PPAs nor OEM repositories that they control would be considered part of the agreement, that should be said as early on as possible to prevent any later re-interpretation.

Why would they? He said part of the release. PPAs are not part of releases. No-one has ever even suggested they are, except for you, who apparently considers Rawhide and people.fedoraproject.org to be a part of RHEL.

A release is defined fairly strictly by virtue of what's in the repositories for that release, which is a finite and well-known set of packages. You're the only one on the planet who seems to think that just because something has the same domain name or is funded by the same people, it's also magically part of the release, which is provably false. This argument is getting incredibly dull.