LWN: Comments on "Getting multiarch dpkg into Debian"

Getting multiarch dpkg into Debian

ibukanov — Mon, 19 Mar 2012 21:38:42 +0000

https://bugzilla.redhat.com/show_bug.cgi?id=718404

Getting multiarch dpkg into Debian

jjs — Fri, 16 Mar 2012 01:53:17 +0000

Debian handles multiple versions just fine. In fact, during a transition (say gcc4.5 - gcc4.6), both packages are installed, along with their dependencies. Once the transition was done (no more dependencies on gcc4.5), that set of packages is automatically removed, unless you have marked them as manual install or pinned them. No weak technical foundation at all.

In this case, Debian is going beyond biarch to multi-arch. Not just have x86-32 and x86-64 (amd64) installed, but say have ARM, SPARC64, x86-32, and x86-64 simultaneously installed, to ease cross-development. That's hard, But Debian is working to get it right.

Getting multiarch dpkg into Debian

mathstuf — Tue, 13 Mar 2012 13:18:43 +0000

Mind linking to said bug?

Getting multiarch dpkg into Debian

mlankhorst — Tue, 13 Mar 2012 12:00:03 +0000

oh but then I look it up and it's already been a bug for the past 3 releases of fedora with nothing that has been done to fix it. llvm devel packages have the same issue, also not fixed.

Getting multiarch dpkg into Debian

mpr22 — Mon, 12 Mar 2012 13:24:37 +0000

It seems to me that if a -devel package has to be --force'd, you should examine why - then either fix the cause if it's at your end, or file a bug report if the cause is at the package maintainer's end.

Getting multiarch dpkg into Debian

man_ls — Mon, 12 Mar 2012 10:00:03 +0000

May I nominate this comment for distribution quote of the week?

Getting multiarch dpkg into Debian

man_ls — Sun, 11 Mar 2012 22:15:22 +0000

Yes, it is really interesting, and it gives a new perspective on the whole article: the version of dpkg in experimental may cause db corruption.

Getting multiarch dpkg into Debian

mv — Sat, 10 Mar 2012 17:31:03 +0000

This follow-up may be interesting. Besides announcing an upload of multiarch capable dpkg to unstable, Guillem points out some results from the code review.

Getting multiarch dpkg into Debian

rahulsundaram — Fri, 09 Mar 2012 20:49:00 +0000

Just so that you know, I maintain over 100 RPM packages and never had to use --force even though I routinely use -devel packages.

Getting multiarch dpkg into Debian

mlankhorst — Fri, 09 Mar 2012 20:19:02 +0000

Yes, the rpm packages don't need --force, unless you start actually using any -devel and then you have to use it all the time, which is annoying if you actually want to upgrade packages, since there is no yum --force..

Getting multiarch dpkg into Debian

sml — Fri, 09 Mar 2012 08:02:39 +0000

Fascinating article! This kind of analysis is why I subscribe to LWN.

To echo the first post, I think this shows the power of Debian's democratic structure. Approaches to fundamental change on the scale of multiarch are always going to be controversial and consensus difficult and slow.

It's unfortunate that Guillem's conscientious approach to dpkg maintainership (and lack of free time) clashed with release goals, but I guess this is the price of striving for release predictability. Maybe it's the "Ubuntu effect".

Getting multiarch dpkg into Debian

rahulsundaram — Thu, 08 Mar 2012 18:19:24 +0000

Post a correction. Don't ask people to do more research just to find out what you are talking about.

Getting multiarch dpkg into Debian

guillemj — Thu, 08 Mar 2012 17:59:56 +0000

I'm sorry to say, but parts of the article contain factual errors, others are just gross mischaracterizations. And while some of the events happened on private conversations, either mail or IRC; public mails on mailing lists, some bug reports or actual code on the git trees should be enough to get a more accurate idea of the situation.

Getting multiarch dpkg into Debian

rahulsundaram — Thu, 08 Mar 2012 02:27:56 +0000

Most users who use bi-arch use it to run 32 bit binaries on a x86_64 system which has been supported in RPM for ages already and works fine. Debian has addressed a larger design and in a more generic way and has taken longer to do it. Just a different design goal and choice which is understandable. No need to bring silly things --force (which I have never used in over a decade with RPM systems and there are really zero reasons to do it) and relocatable RPM packages( haven't come across any in the real world) into the discussion.

Getting multiarch dpkg into Debian

rgmoore — Thu, 08 Mar 2012 02:05:02 +0000

The package maintainer just wanted time to review the code to ensure it met the standards for a package that basically controls the stability of the entire distribution.

The underlying problem is that the standard procedure doesn't give any kind of deadline for the package maintainer to respond. That's reasonable because there really isn't a one-size-fits-all response interval, but it means there's a danger that something will be put on the back burner and never taken off because the maintainer always sees something else as a higher priority. That seems to have been what happened in this case, and it wasn't until the technical committee gave him a firm deadline that he bumped the priority enough to do the review.

The big question is whether there needs to be some general procedure for dealing with this kind of problem, or whether ad hoc solutions will work. This is a rare case where the package is on the critical release path for the whole project, so the technical committee really had to step in. But what happens when it's on a less important project that isn't going to hold up the whole release? Can a maintainer put off changes indefinitely because he wants to review everything and doesn't have time to do it? Does there need to be some kind of general process to generate a definitive answer for changes that have been delayed?

Getting multiarch dpkg into Debian

smoogen — Thu, 08 Mar 2012 01:49:19 +0000

From a Fedora/RPM person.. I don't think you are correct on this. THere is actually some magic that was added in RPM to allow for multiple files being owned by the same package and then using directory packaging to make sure needed files don't conflict.

RPM packaging chose a way to do this via policy that I believe dpkg could ahve done.. but it would not have met other goals of theirs to make sure it worked on N+1 architectures. So lets not try to start a "my packaging system is better than yours" because those never convince anyone of anything.

Getting multiarch dpkg into Debian

Cyberax — Wed, 07 Mar 2012 22:53:18 +0000

>Yes, just installing some random arch's libraries and executables and then having the whole thing magically run would be nice... if you are into that. If the software is open source, a native (and thus presumably much more performant) package is at the end of your fingertips, so this point is moot for most of the user population anyway.
Nope. Quite a significant part of 'regular' Linux population uses bi-arch support every day. And it's going to get even more complex with the new x32 architecture.

And it'll mostly 'just work' in Debian.

Getting multiarch dpkg into Debian

Tobu — Wed, 07 Mar 2012 20:58:46 +0000

So far we have three design choices: package states keyed by a name, by a (name, arch) tuple, and by a (name, arch, version) tuple.

Keying by name is simple and robust: packages with different names must agree not to own the same file names.

Keying by version is tricky and bug-prone: if versions are to be parallel installable, different versions of a package either need to not own the same files, or to keep identical contents in files that didn't move. This doesn't seem compatible with shipping changelogs or common resources.

Keying by arch while requiring that the versions stay locked, on the other hand, is safe: the files were built from the same source, so files that are shared across architectures will easily be identical; architecture-dependent files will need to use paths that embed the architecture name.

Getting multiarch dpkg into Debian

vonbrand — Wed, 07 Mar 2012 20:44:16 +0000

Haven't used --force for untold ages here... forcing the installation of something without the proper dependencies will brew trouble.

Yes, the RPM format allows relocatable packages; in practice making something relocatable is a major undertaking, for very little use (so almost nobody hops through all the required hoops). Small wonder.

Yes, just installing some random arch's libraries and executables and then having the whole thing magically run would be nice... if you are into that. If the software is open source, a native (and thus presumably much more performant) package is at the end of your fingertips, so this point is moot for most of the user population anyway.

Getting multiarch dpkg into Debian

Cyberax — Wed, 07 Mar 2012 20:24:11 +0000

So that's probably why RPM packages often require using the --force hammer to make a package work.

Also, RPMs are supposed to be relocatable. Even though it doesn't work in practice since nobody tests it. Then there are questions of foreign architectures - how do I install an ARM version of libxml.so so I can use qemu to run my binary without full-system emulation?

Getting multiarch dpkg into Debian

walex — Wed, 07 Mar 2012 16:28:52 +0000

RPM and other packaging systems have not had this limitation EVER. Indeed RPM can also handle having library packages of different versions too, something that Debian mishandles by putting the version in the base package name too.

It is very sad that Debian is based on such a weak technical foundation, but it all goes to prove that putting in a lot of extra effort can make even poor technical choices work up to a point.

It is also sad that there is no popular community based distribution that can compete with Debian and has better technical foundations (and less bizarre policies).

Getting multiarch dpkg into Debian

pboddie — Wed, 07 Mar 2012 11:11:36 +0000

I think the only possibly negative thing this really indicates is that Guillem needs some more help (or needs to trust his helpers more)

I think that the general point here is that in projects people need to be able to ask for help, and people must be willing to offer help. Again, in general, good management is about bringing these things together, whereas bad management (as many of us are unfortunately all too aware) often involves someone thinking that their job is merely to tell you what to do, frequently not contributing to the activity in any way other than to indicate that things must be done more quickly, as opposed to helping you get your work done by giving you what you need.

Getting multiarch dpkg into Debian

stefanor — Wed, 07 Mar 2012 10:50:03 +0000

Practically, experimental is normally a staging area, rather than testing ideas. Guillem still had doubts about the design of multi-arch and, I presume, didn't want developers to adapt their packages to a multi-arch implementation that wouldn't be compatible with the final form.

Getting multiarch dpkg into Debian

josh — Wed, 07 Mar 2012 08:54:03 +0000

The "experimental" distribution exists for exactly this reason: testing out changes too dangerous to put in unstable (where they could otherwise potentially filter into testing and stable assuming no release-critical bugs). Objecting to an upload to unstable would seem very reasonable and appropriately cautious. However, uploading to experimental makes a lot of sense, as a way to let developers start testing out the functionality, reporting bugs, and figuring out how to make their packages work with it.

Getting multiarch dpkg into Debian

scientes — Wed, 07 Mar 2012 00:26:55 +0000

something like this occurs with ruby 1.9.3, which has the package name of ruby1.9.1, (there is also a ruby1.9 (.0) which while confusing, I agree with. http://packages.debian.org/sid/ruby1.9.1

Getting multiarch dpkg into Debian

rahvin — Tue, 06 Mar 2012 23:19:22 +0000

First MultiArch as stated in the article is a long term goal of the entire project and everyone involved. The package maintainer just wanted time to review the code to ensure it met the standards for a package that basically controls the stability of the entire distribution.

Second, managing a process and people involved in that process is HARD even in commercial environments where you can tell someone what they should work on. It's even harder in environments where you can't tell someone what to do because they are a volunteer.

The package maintainer was right to want stability in dpkg and the technical committee was also right to want to move the code forward as it's a critical part of MultiArch. So if they are both right the trick is to bring about a solution that recognizes that and works with the process, that's HARD. As the article says the tech committee issued an ultimatum so the package maintainer made time for the review and got the patch merged. The process worked and in particular worked on a project with true democratic control and a mostly volunteer workforce. To me that's a direct indication of the dedications of the Debian maintainers. I think the only possibly negative thing this really indicates is that Guillem needs some more help (or needs to trust his helpers more) but DPKG is one of the most critical pieces of the Debian toolchain so I understand his caution.

Getting multiarch dpkg into Debian

alvieboy — Tue, 06 Mar 2012 23:08:17 +0000

I'm sorry ?

These conflicts arise every time because they have to (and glad they do). There's no actual split between "development" and "release management". Both have same goal - to get the best-of-the-art delivered to the user. Different motivations, perhaps, but goal is the same. Different paths to accomplish same goal, but they tend to balance each other.

Equilibrium is the key word. Like in everything else in life.

Now, saying that "Conflicts like this should never happen"... They do, and I'm glad they do - that means people actually worry about. Both developers, and "release managers".

Alvie

Getting multiarch dpkg into Debian

bug1 — Tue, 06 Mar 2012 22:36:38 +0000

Conflicts like this should never happen.

Development decisions and release management decisions should be made by different people, each have different goals. Developers have long term goals, release management short/medium term goals. There is a conflict of interest.

Unfortunately debian has a long history of linking the two.

Getting multiarch dpkg into Debian

cmorgan — Tue, 06 Mar 2012 21:38:14 +0000

I guess that's how Debian works. I've had the same kind of issues with the libpcap version in Debian, http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=657900 I understand that strictly speaking one could argue that the abi hasn't actually changed so the version should be kept at 0.8 but I'm just asking for a symlink to make things look like they do on Fedora/OpenSUSE. I've even considered trying to improve libpcap so the abi would change and would force Debian to resync with what the libpcap developers consider to be the "current" version number.

Chris

Getting multiarch dpkg into Debian

alvieboy — Tue, 06 Mar 2012 21:24:05 +0000

Sounds indeed like Democracy to me.

I'm glad this finally moves on.

However, I can understand Guillem. I also develop and maintain a packaging system myself, and I'm not very open when it comes to change it, for the sake of stability. But sometimes we have to do it, and the first thing we need to do is to convince ourselves that those changes are indeed a necessary move. This is why I think Guillem did not accept them, he is not convinced they are necessary or that they lack the required quality (I think the former, but I might be wrong).

Still, he seems to have been forced to accept those changes. This will eventually cause a bigger problem, cause you cannot properly maintain something you don't agree with.

Hopefully Debian will manage to overcome this, and so Guillem. Let's hope they do, for the sake of their users (like myself).

Alvie