LWN: Comments on "Debian multiarch support"

Debian multiarch support

k8to — 2006-12-18T10:54:03+00:00

Wow. It's now six months after this proposal, and I see no movement.

Also, it's been over two years since this arch came on the market and the obvious need for multi-arch arose. Usually I'm pretty content with the slow pace of progress in Debian, in that it brings a lack of pain, but this particular issue is a point of discontent.

In order to get a bi-arch distribution with extremely broad package support, what, if any, are my options?

Debian multiarch support

dmaas — 2006-05-30T07:31:51+00:00

I am glad to see more attention brought to this issue.

I maintain a network of 32- and 64-bit x86 systems for my company. For various reasons it is highly desirable for all systems to run the same disk image (except the kernel, of course). We want to run a handful of computation-intensive software in 64-bit mode on the systems that support it, but everything else can be 32-bit.

I started with a plain 32-bit Debian system and manually added in a few pieces from 64-bit Fedora Core (mainly /lib64 and /usr/lib64) - just enough of the basic libraries to run the computation-intensive 64-bit packages.

It has worked out surprisingly well. I did notice one problem the article points out, which is conflicting binaries in /bin and /usr/bin. I had to install 64-bit versions of certain low-level tools like modprobe, strace, and ps to support the 64-bit systems. I did this by renaming the 32-bit versions to modprobe32, strace32, etc, appending similar suffixes to the 64-bit versions, and writing wrapper scripts which picked the right version to run on a particular machine. Thankfully there were only a handful of these cases.

The other nasty issue was glibc headers. The contents of /usr/include/asm and /usr/include/bits are platform-specific. I followed RedHat's model of installing both x86 and x86_64 headers with different names and picking the right one using small "stub" headers. I wish glibc would install its headers in a more platform-neutral manner.

(incidentally, some of you might want to read up on Plan 9, which solved this problem years ago :)

Debian multiarch support

msw — 2006-05-19T19:19:13+00:00

Red Hat did not invent lib64. It was in use in Irix well before it appeared in Linux (to my knowledge).

/usr/share

rfunk — 2006-05-19T16:44:19+00:00

Yes, /var is for files that change a lot during runtime. /usr only
changes when you upgrade the OS (or pieces of it).

This is all explained in the Filesystem Hierarchy Standard.
http://www.pathname.com/fhs/

/usr/share

jzbiciak — 2006-05-19T16:24:20+00:00

Well, it appears a bunch of otherwise arch-neutral stuff ends up in places like /var/lib currently. On my Ubuntu box, for instance, there's a bunch of Firefox files in /var/lib/mozilla-firefox, and they appear at first blush to be arch-neutral--they're all .rdf and .txt files pretty much. All the doc files, though, are in /usr/share/doc/mozilla-firefox/.

/usr/share

rfunk — 2006-05-19T16:07:18+00:00

Yes, that's pretty much what /usr/share is intended for, but the fact
that it was mentioned seems to imply a change of some sort.

/usr/share

jzbiciak — 2006-05-19T15:58:40+00:00

It's probably better to say "architecture independent files." When I hear the word "binary" referring to a file I usually think "executable." I think what they mean here are all the supporting files for an application, which may include image files, documentation files, and other fairly static things.

Re: i386 running amd64 binaries?

interalia — 2006-05-19T11:48:33+00:00

multiarch is mostly just a structural thing. Obviously, rearranging files and dynamic linkers on its own won't allow someone to run binaries for amd64 on i386, as you implied. But it lets you parallel-install them cleanly, which is great. multiarch not meant for simply having amd64/i386 co-exist, the idea is to have a consistent structure even if you want three or more architectures, something that the "/lib and /lib64" solution doesn't provide. The proponents contrast multiarch with the existing approach, which they call bi-arch.

As posited in previous comments, you might have a third /usr/lib/xx-yy-zz area for cross-compiling to ppc. Then you might run the compiled ppc programs using a native amd64 binary which is a ppc emulator. Perhaps even WINE could somehow become an arch for this purpose. If you read the threads on multiarch, people have come up with all sorts of ideas that can be done when you aren't limited to whether something is 32-bit or 64-bit.

Debian multiarch support

jwb — 2006-05-19T03:58:33+00:00

This is the sort of parochial redhatism that led to the creation of the Debian amd64 (formerly
pure64) architecture. /lib64 is deeply broken for many reasons, not the least of which is the fact
that 64 is the native word length on some platforms and many people don't want to have 32-bit
binaries or libraries on their systems at all.

i386 running amd64 binaries?

ncm — 2006-05-19T00:46:58+00:00

I'm finding it hard to believe that multiarch is all it takes to let you run amd64 binaries on an i386 host, as the first paragraph suggests. Has an amd64 instruction-level emulator been added since last I checked, or should it say, instead, that it lets you run i386-compiled binaries on your amd64 system?

/usr/share

rfunk — 2006-05-18T21:58:19+00:00

"The proposed location for all architecture-independent binaries is
under /usr/share."

I hope that refers to executables that truly run on *any* supported
architecture, generally only scripts. /usr/share would be a very bad
place for x86/x86-64 binaries that won't run on PPC.

Debian multiarch support

elanthis — 2006-05-18T15:10:33+00:00

The future LSB specs are likely going to change on that front. The /lib64 stuff is likely to become a symlink.

Simply put, there are many cases where you want more than just x86 and x86-64 on a system. Maybe you want PPC and PPC64. Or maybe you even want x86 and PPC. Another common one is x86 and IA64. In that scenario, the 32-bit libs don't go in /lib, they go in /emu/x86 (or something similar to that). Even though that's "standard" according to various ABI specs, it then makes it impossible to have a single x86 package that installs to both a native x86 machine, and x86-64 machine, and an IA32 machine. And a system that runs PPC natively but has x86 emulation features (for running those pesky x86 proprietary apps) is going to need yet another path.

Then, we don't just have issues of hardware architecture, but also the kernel and OS interface. Many OS's can run Linux binaries, for example, but you need to put them in chroots and such in most cases so that you can separate the native version of libraries with the Linux version of the libraries. Multiarch can solve this problem, too.

By putting all libraries in a standardized path like /lib/linux-gnu-x86, /lib/solaris-sun-sparc, /lib/linux-gnu-ia32, and so on, it becomes possible to have a single installed system that can run binaries from multiple OS's and multiple architectures, even in cases where the host CPU doesn't have some built-in compatibility system. (Think PPC binaries running on x86/x86-64 CPUs with Rosetta under OS X.)

So, while I'm very sure that x86-64 users are the big pushers for multiarch (I know that's what _I_ want it), there _are_ many other use cases for it that can't be solved cleanly without either it or massive chroots (and the headaches those bring, like trying to sync configuration).

Debian multiarch support

msw — 2006-05-18T14:14:34+00:00

I helped implement the multiarch support in RPM, and it's not just that simple. Often you need to install both 32-bit and 64-bit packages at the same time. Sometimes bits of those packges overlap. This (sort of) works in RPM because two packages are allowed to own the same file if the contents are identical. This lead to a ton of work to make the packages in Red Hat produce exactly the same contents (for the non-architecture-specific files) on both 32-bit and 64-bit systems. Sounds like fun? I assure you it wasn't.

We took that experience and designed a new package management system from the ground up. One of the primary design goals was elegant multi-arch and multi-lib support. The result is Conary, which does a pretty amazing job. We've easily installed 32-bit OpenOffice.org or Firefox on a 64-bit system - out of the box.

Debian multiarch support

wookey — 2006-05-18T08:07:47+00:00

Multiarch actually has wider use case than the obvious one of letting amd64 systems run both 64 and 32 bit binaries/libs. It also helps with cross-compilation issues, by having a standard place to install non-native libs (of multiple types) and a packagemanager that can cope with this. dpkg-cross currently provides this functiponality by re-writing paths in standard packages to make them cross-installable. Multiarch should make that aspect of dpkg-cross redundant.

I think it can also help with cross-building schemes which use emulation to run target binaries at build-time, but I need to think about it a bit more first.

So embedded/cross-building people should be keen to see multiarch happen, even if they don't care at all about amd64 64/32 mixed installations.

Debian multiarch support

kleptog — 2006-05-18T07:42:39+00:00

But where do the PPC64 libraries go?

It's all nice and good to have a solution for x86, but there are other architechtures that can support multiple ABI formats simultaneously, it would be nice to do it in a way that works for everyone.

Fortunatly, the dynamic linker is smart and solves 99% of the problem for us. We just need to make sure that packages don't try to install over the top of eachother.

Debian multiarch support

dmantione — 2006-05-18T06:36:48+00:00

This should be no issue for discussion: According to the x86-64 ABI
32-bit binaries go in /lib, 64-bit binaries go in /lib64:
http://www.x86-64.org/documentation/abi-0.96.pdf

The reason is simple: This is the only way you can install (RPM) packages
designed for i386 systems on your x86-64 systems. You can simply forget
what architecture the package has, it just works.