In his talk at FOSDEM (Free and
Open Source Software Developers' European Meeting) 2012 in Brussels, Wookey
(who is working for Linaro on Linux
for ARM and doesn't have a first
name) talked about what multiarch is and why it's important. Multiarch
is a general solution for installing libraries of more than one
architecture on the same system.
By "general" we mean more than just the lib and lib64 directories for 32 and 64-bit x86 libraries. Currently the Filesystem Hierarchy Standard (FHS) attempts to address the use of these libraries on the same system by requiring that /usr/lib be reserved for 32-bit libraries, while 64-bit libraries are located in /usr/lib64. This so-called "biarch" design was adopted by Red Hat and SUSE, but not by Debian and Ubuntu. A general solution should not only scale to other architectures, but it should also "remove all corresponding bodgery we have in Debian, such as ia32-libs and biarch packages," Wookey says. The Debian developers have been working on a multiarch solution for years and multiarch support is a release goal for the coming Debian 7 "Wheezy" release, expected in 2013.
The basic idea behind multiarch is to generalize the biarch design to arbitrary architectures, and the way it is done is actually quite simple, Wookey maintains: you put your libraries into architecture-specific paths. For instance /usr/lib/libfoo goes into /usr/lib/x86_64-linux-gnu/libfoo if your machine has an x86_64 architecture, into /usr/lib/i386-linux-gnu/libfoo for an i386 architecture, into /usr/lib/powerpc64-linux-gnu/libfoo for a ppc64 architecture, and /usr/lib/arm-linux-gnueabi/libfoo for an armel architecture.
The multiarch paths contain the GNU triplets used by GCC to describe
architectures. For instance, in x86_64-linux-gnu "x86_64"
the processor type, "linux" designates the kernel, and
"gnu" stands for the
user-space ABI. However, multiarch adopts the GNU triplets with some
adjustments. For instance, both the i486-linux-gnu and
i586-linux-gnu GNU triplets will be translated to the
/usr/lib/i386-linux-gnu/libfoo path because, according to Wookey,
a few minor instruction set differences do not add up to a different ABI
requiring its own triplet. The advantage of this partial
rethinking of the file system hierarchy is that all libraries have a
canonical path. There are no special cases for the locations of native,
cross-built, or emulated (with QEMU) libraries: they are all the same.
What can we do with it?
So what can we do with multiarch? As already mentioned, multiarch makes cross-compilation much simpler: it is no longer a special case and essentially you're getting it for free as a byproduct of multiarch. This is primarily because the library loader path is baked into every executable by the linker, and thanks to multiarch's canonical path based on the system's architecture, this path is the same whether the library is built or cross-built.
In the classical approach of cross-building for armel (with dpkg-cross),
the build-time library path is for instance
/usr/arm-linux-gnueabi/lib/libfoo, while the runtime library path
is just /usr/lib/libfoo. With the multiarch approach for
cross-compilation, the library path is
/usr/lib/arm-linux-gnueabi both at build time and at run time, so
"it's much harder for libtool to screw it up," Wookey said. Another advantage is that you can just run the build tools under QEMU via binfmt-misc for testing.
Multiarch also allows for a better support for binary-only software, which tends to be only available for 32-bit systems. Thanks to multiarch, you can more easily install 32-bit versions of a 32-bit proprietary program's dependencies on a 64-bit system. Wookey gave as examples the Flash plugin, Skype or Xilinx development tools. Multiarch also allows cheap emulated environments: you can emulate only the parts you need.
The slow genesis of multiarch
Wookey quotes Tollef Fog Heen, who said in 2005: "ia32-libs [is
now] the biggest source package in Debian." That is because
currently any 32-bit software that has to run on an amd64 (which is the
name Debian uses for x86_64) installation depends on the package ia32-libs,
which contains i386 versions of all of the libraries, so its source package currently weighs in as a 555 MiB tarball. Ia32-libs was always intended as a temporary solution for the i386/amd64 case, but unfortunately (as often happens with these things), developing the proper general replacement took a lot longer. There were talks about a solution at Debconf 4 and 5 (in 2004 and 2005, respectively), there was a multiarch meeting at FOSDEM 2006, and in June 2006 the first multiarch patches for dpkg were uploaded.
In May 2009, the apt and dpkg maintainers agreed on a package management specification for multiarch at the Ubuntu Developer Summit in Barcelona. To avoid further delays, they restricted the scope to multiarch libraries. In August 2010, the first proposal for multiarch directory names was drafted, and in February 2011 a dpkg multiarch implementation (sponsored by Linaro) landed in Ubuntu. A month later, the normalized GNU triplets were adopted for the multiarch directory names, and then the Ubuntu 11.04 release came with 83 libraries multiarched. Together with 14 multiarch libraries in a PPA (Personal Package Archive), this was already enough to cross-install the 32-bit Flash plugin on a 64-bit system.
Currently, the Ubuntu core is almost completely multiarch: at the time
of Wookey's talk, 110 out of the 112 source libraries in the Ubuntu 12.04
main repository were multiarched, as well as 175 out of the 176 binary
libraries. Obviously all libraries have to be made multiarch-ready, but
also most -dev packages need converting as well, using a similar directory naming scheme as for libraries. That makes it possible to co-install include files that differ between architectures. But on top of this, any tool that is aware of library paths had to be fixed, including libc, dpkg, apt, compilers, make, pkg-config, pmake, cmake, debhelper, lintian, libffi, OpenJDK's lib-jna, and dpkg-cross.
Wookey made it clear that the multiarch development is a classic example
of a significant distribution-wide change, which is generally very
difficult to do right. One of the factors in the success of the multiarch
development is that they used written specifications to record a shared
understanding. As can be seen from the project's history, another key
factor is that they split the work into bite-sized deliverables.
How does it work?
Normally, a package of the same name but a different architecture is not
co-installable. Multiarch-ready packages, though, are given an extra field
Multi-Arch in the package specification. This field has one of three possible values, depending on the type of package. A library has the value "same": it can be co-installed with the same package from another architecture, but it can only be used to satisfy the dependencies of a package within the same architecture. An executable has the value "foreign": it cannot be co-installed with the same package from another architecture, but it should be allowed to satisfy the dependencies for any architecture (of course, preference is given to a package for the native architecture if available). And a package that contains both libraries and executables has the value "allowed". An example of this is the python package. The depending packages specify how they use it.
The Debian wiki has some information about how package maintainers can
package to multiarch, as well as some general information about multiarch support in
Debian. Note that a package for a foreign architecture is only installable if all of its (recursive) dependencies are either marked as multiarch or do not have corresponding packages installed for the native architecture.
An interesting implementation detail is that co-installability doesn't mean that documents from a package get installed twice when you install two architectural versions of it: according to Wookey, dpkg has support for reference-counting of documentation files from co-installable packages that overlap. So an identical documentation file in a 32 and 64-bit x86 version of a library only gets installed once, and it doesn't get removed until both versions of the library are removed.
In practice, you can easily add a new architecture to your machine's
Debian or Ubuntu installation. For instance, when you have an amd64
installation and you want to install some i386 libraries, you can add the
latter architecture with a simple
"dpkg --add-architecture i386" command. Use
"dpkg --print-foreign-architectures" to get a list of the
foreign architectures you have added, and
"dpkg-architecture -qDEB_HOST_MULTIARCH" to see the multiarch
pathname for your system's native architecture. The entries in
/etc/apt/sources.list also get an extra arch field, for instance:
deb [arch=amd64,i386] http://archive.ubuntu.com/ubuntu precise main
After an "apt-get update
" to refresh the package list, you can just install an available multiarch-ready library by specifying the architecture after a colon, for instance "apt-get install libattr1-dev:i386
". This has been working in Ubuntu for nearly a year now, since 11.04.
Things multiarch (currently) doesn't do
Currently the multiarch solution is limited to libraries. This means that you can't install executables from more than one architecture in /bin or /usr/bin with multiarch. Co-installable executables could be useful (for instance to reuse a single network-mounted root partition on systems of multiple architectures with no modification), but it is deliberately left out of the initial implementation because it would complicate matters further than they already are. Other than a multiarch path for executables, such a system would require kernel support or boot-time symlinking. Before implementing this, the multiarch developers need a detailed specification as they have done with the implementation for libraries, Wookey warned.
Another interesting but currently not implemented feature is that you could "cross-grade" your machine from one architecture to another one. For instance, if you have installed a 32-bit x86 distribution on your 64-bit machine, you could convert it to a 64-bit distribution without having to reinstall it. This could be possible by first manually installing the 64-bit versions of dpkg and apt and then changing which architecture is used by default, after which you could reinstall all installed software, but from the 64-bit architecture. This should work the same way for a cross-grade from arm to armel and from armel to armhf.
Wookey ended his talk with the message that Debian and Ubuntu have now
done the hard work for multiarch and shown that it works. However, it could
be useful beyond Debian and its derivatives. The multiarch directory scheme
will be a target for FHS/LSB standardization in the future, but even if
that doesn't happen, it's a much more scalable solution than the current one.
(Those wanting more details can watch the
video of Wookey's talk posted by the conference.)
Comments (30 posted)
Now, if you're some kind of trouble-making, commie, tin foil hatted
conspiracist, you might even think that this might be Fedora 17 Alpha
This is both laughably naive and treasonous! Purge such thoughts from
your mind immediately. Now. I'm warning you. My friends at Google will
let me know if you haven't.
-- Adam Williamson
One of the Board's goals for the F16/F17 timeframe is: "It is
extraordinarily easy to join the Fedora Project."
I would argue that, right now, that it is Extraordinarily Difficult to get
anything DONE in the Fedora Project.
-- Robyn Bergeron
I believe we put far too much burden on
folks trying to get things done...
Sadly I think that because no one person can do everything
on their own, and there's this expectation that they should somehow
figure out how to make it happen, to have the vision, leadership,
organization, coding & design & writing skills when they can't possibly
have all of that.
Note how the open source community as a whole has this emphasis on 'rock
stars.' Well, yes, the people who get things done are 'rock stars'
because you have to be to get anything done! This is not a *good*
-- Máirín Duffy
Comments (none posted)
The Debian Project has announced the availability of its patent policy for
the Debian archive. "The Debian Project maintains a critical stance
towards software patents: we consider software patents to be a threat to
Free Software and an obstacle to the Debian mission of providing an
entirely Free operating system for everyone's use. We believe software
patents provide no advantage in promoting software innovation and we
encourage our upstream authors to object to software patents.
Full Story (comments: none)
For those interested in helping to test the next Mageia distribution
first Mageia 2 beta
is out. New features are listed on this page
include a switch to systemd, the latest desktop environments, a switch from
MySQL to MariaDB, and more.
Comments (6 posted)
of openSUSE 12.2
has begun with the release of the first milestone (alpha).
"We’re pleased to announce that Milestone 1 contains many minor
updates, like a new Firefox version but also major things like new artwork
and KDE 4.8.
" Testing and bug reports are welcome.
Comments (none posted)
Scientific Linux has released
SL 6.2. See the release
for details. Version 6.2 is also available
in several live versions, including a LiveMiniCD with IceWM, a LiveCD with
GNOME, and a LiveDVD with GNOME, KDE and IceWM.
Comments (2 posted)
The Ubuntu 10.04.4 LTS update has been released; the Kubuntu variant has
been updated as well. "This is the last planned maintenance release for the 10.04 LTS series.
Future security updates and bug fixes will be individually downloadable
from the Ubuntu archive in the same way as before, but no further
updates to installation media will be provided for 10.04 LTS.
Full Story (comments: none)
Canonical's Ubuntu for
offering has been announced. "Ubuntu for Android
provides a full desktop experience, including office software, web
browsing, email and media applications, on Android phones docked to a
screen and keyboard. Thanks to tight integration with the Android service
layer, the transition between the two environments is seamless, making it
easy to access the phone's services from the desktop when docked.
The target audience at this point looks to be handset manufacturers rather
than end users.
Comments (32 posted)
Newsletters and articles of interest
Comments (none posted)
On the Canonical design team blog, John Lea gives an overview of the Unity design process
and describes various ways that users can help out. "Once the problem that we are trying to solve is clearly defined, the next step is to assemble the previous thinking that has gone into the problem area. Understanding what has gone before and the current state of the art is the starting point from which new connections can be made, concepts built upon and extended, and new ideas created. Mailing lists, bug reports, and forums are scoured for pertinent information and products relevant to the problem space are examined. In addition to the collation of previous thinking, fresh research can also be conducted to generate new insights. This solid understanding of the existing problem space is a elemental ingredient of the design process.
Comments (107 posted)
Here is an HP blog posting
giving a high-level description of how the webOS project will be governed. It is an Apache-inspired bureaucracy with a number of subprojects, each of which will have its own "project management committee" watching over it. "PMC members are expected to act individually, making decisions in the best interests of the project, when acting on PMC or development lists. Each PMC is responsible for ensuring their project follows certain core requirements set by the board or other corporate officers of Open webOS. Examples include following legal, branding, and infrastructure related requirements, and ensuring their community operates in a manner similar to that outlined by the Apache Way.
Comments (6 posted)
The H looks
at the new and improved stuff
in the eighth update to Red Hat
Enterprise Linux 5. "A little under five years since it was first released, Red Hat has provided customers with the eighth "minor release" of Red Hat Enterprise Linux (RHEL) 5. Scalability improvements mean that rather than 128 cores, guests virtualised with KVM can now use up to 256 processor cores. Red Hat says it has improved support for clock and timer hardware in KVM guest systems. KVM guests will now boot more quickly and, thanks to updates to the real time clock (RTC) code, RHEL 6 guests running on RHEL 5 hosts will run faster. The Spice client now supports Red Hat Enterprise Virtualization (RHEV) 3.0 and RHEL 6.2 hosts, enabling the remote desktop protocol to be used on wide area networks.
Comments (none posted)
Following the release
of Red Hat Enterprise Linux 5.8, Mark J. Cox takes a quick look
the vulnerabilities and security updates since the release of v5.7. "So, for a default install, from release of 5.7 up to and including 5.8, we shipped 42 advisories to address 118 vulnerabilities. 4 advisories were rated critical, 13 were important, and the remaining 25 were moderate and low.
Comments (none posted)
Page editor: Rebecca Sobol
Next page: Development>>