Distributions

News and Editorials

Distribution-friendly projects Part 2

[Editor's note: This article, which looks at the interactions of software projects and distribution providers, is presented in three parts. Part 1 introduces the concepts found here, in part 2.]

Technical needs

Under the name technical needs we're going to see a series of requests that distributors often have to make to the original developers of the software they want to package. Not all these requests are made by all distributors. Some will care more about one particular aspect than another. Some might apply only on non-mainstream distributions, and some distributions might just want to take care of philosophical needs and leave the technical side entirely alone, even if similar distributions aren't exactly common.

Most of the technical needs described in this article are present in the policies set forth by Debian (written), Gentoo (mostly unwritten), and apply to other distributions as well. Some of these needs won't be encoded in any policy and are often not requested explicitly by the developers. Those are mostly details that make a distributor's life easier. These details may not be mandatory, but it's still worth considering them. The easier the life of the downstream maintainer is, the easier it is for the software to be packaged.

Also, it's important to note that when a distribution makes a request, it might not be alone. Other distributions might want to take advantage of the same change, but they didn't have time to request it, or simply preferred to wait before packaging the software until some issues were resolved. Don't just ignore the request because the distribution which contacted you already took care of the issue by patching your software. Acknowledge the request and apply the patch, it will make both your and their life easier on the long term.

Sane version information

Distributions often rely on the version information provided by the original software developers. This usually means that they don't expect huge changes between version $x.y.z$ to version $x.y.z+1$.

One very common scheme for versions is the major, minor, micro version, which in the example above would be respectively $x$, $y$ and $z$ (it's a common misconception that $y$ is the major version component).

The way this kind of scheme is usually applied relates to the compatibility of the programming interface (API and ABI). Changes in the software warrant increments of various version components depending on the amount of changes in the interfaces:

• adding zero or more interfaces, without changing or removing previous interfaces, or the behaviour expected from them - meaning the software is entirely compatible with the older version - usually only require an increment of micro version;
• changing or removing interfaces, usually deprecated - in such a way that older software might require to be adapted, but not rewritten - usually require an increment of minor version;
• changing the interface entirely - requiring users of the software to rewrite their code, or otherwise do major structural changes - usually require an increment of the major version.

Obviously, increasing one component will usually involve resetting to zero the version components on the right.

There might be other components, too. For instance if the source archive has to be regenerated without any code change (missing file, updated addresses for the maintainers or the homepages), rather than changing the version entirely, a suffix might just be added at the end of the version, making it, for instance $1.2.3a$ or $1.2.3c$. If just a security issues has been fixed, it could also be expressed by adding a nano component to the version, like $1.3.34.1$, to emphasize that there is no change other than the security fix.

The source archives for the software should be named after both the project and the version, resulting in names like foobar-1.3.4.tar.gz. Having different versions of the same software that don't have the same naming causes confusion.

It is quite important for the distributions that source archives not be changed without changing the name: distributions usually make sure that the checksum (usually MD5, but often nowadays SHA1) of the archive is the one they recorded, and changing the tarball without notice often leads to failed builds.

There is a similar issue with the naming of the directory inside the archive. Most distributions assume that the source is included inside a directory with the same name of the archive (minus the extension), but often enough the archive contains sources not organised in a directory, or a directory with the name of the project without version. Similarly, if possible the directory should also contain eventual suffixes, to avoid adding extra cases in their presence.

Distribution methods like Ruby Gems and Python Eggs mandate similar version schemes for their packages for the same reason Free Software distribution would prefer them: it makes it easier to compare versions, and know when something has to be updated.

Internal libraries

One common issue considered by both Debian and Gentoo policies relates to the use of internal copies of libraries. Sometimes the software needs some uncommon libraries to work properly. These libraries are unlikely to be found on users' systems, which would require them to download and install them separately. Such a task is not easy for new users. A few projects will keep an internal copy of the libraries they want to use for that reason, and will use that internal copy unconditionally.

Adding an internal copy of a library seems cheap to the original developers, and it's convenient for users to download and install a single package, however this causes a large number of problems to the distributors. The first problem is that they might have to patch the same bug several times. Let's all think of zlib as a practical example, a very common library implementing the classic deflate algorithm of compression. It's a very small library, that a lot of projects imported internally over the years. Not too long ago, a serious security issue was found in the code of zlib, and all the distributors had to patch it out as fast as they could. In a perfect world, patching zlib, and eventually rebuilding everything that linked to it would have sufficed. Unfortunately, we're not in a perfect world. More software was packaged with internal copies of the library, requiring each of those packages to be patched to make sure the issue was solved.

There are many other implications with using internal bundled copies of libraries, and most of them are critical for distributors. These problems increase their complexity when the internal copies of libraries are modified to suit better the use the application has for them. In those cases, even though the source might be advertised as being part of another library, they are actually different from that library, and their replacement might be impossible, or may cause further problems.

• The code is no longer shared between programs: not only the source code, which requires extra work to fix bugs and security issues, but also executable code and data. When shared libraries are used, the memory used by processes loading them is reduced, as they will share code and part of the data. This cannot be done when using static libraries or, worse, internal copies of libraries.
• Symbols may collide during the loading: modern Linux and Unix systems use the ELF format for programs and libraries. This format provides a so-called flat namespace for the symbols (data and functions) to be found. When using internal copies of other libraries in a library, the two definitions of the same symbol might collide, and just one of them can be used. If the interface used by the library changed subtly, it is possible that this will lead the program in an execution path that was not intended and is not safe.
• Distribution-specific changes need to be duplicated: as it will be discussed later on, sometimes distributions need to make changes to source code, to fix bugs (security related and not), or change paths of files for instance. Internal copies require downstream maintainers to repeat these changes multiple times.

For this reason, a good compromise between the needs of the original authors and the needs of the distributions is to treat internal copies of libraries as untouchable, thus disallowing any changes in its interface or behaviour. That way those users who get the package directly from upstream still have only one package to download and build. The distributions, who want to share code as much as possible, should have a way to ask the build system to use the system copy of that library. An easy way to implement that is to provide --with-system-libfoo options at the ./configure call (for autoconf for instance), or to give a WITH_SYSTEM_LIBFOO" handle at the make command line.

By allowing the distributions to use their own copies of libraries, the developers are still preserving the ability for the user not to install extra dependencies, but also giving the distributions the power they need, to avoid changing the original code, sometimes in a conflicting way. It is important for the upstream authors to not change the behaviour of bundled libraries, as the distributions will most likely want to use a shared system library instead. Modifications made to a bundled library will likely cause problems for users who use get the package from their distribution's repository where it has been built with a shared system library.

An easy choice for optional dependencies

Almost all distributions prefer having a choice about the optional dependencies of a package. Source-based distributions (like Gentoo and FreeBSD's ports system) offer the same (or more) choices as the original project. Gentoo's USE flags or FreeBSD's knobs offers the user options on which options will be enabled. Binary distributions (like Debian or RedHat) might want to choose options to ensure that the final binary package does not try to use dependencies that are not present in their official repositories.

Again, if a project does not provide an easy way to control whether some optional dependency is used, most distributions will either try to workaround that problem (by forcing cache discovery variables) or change the build system themselves to get the choice to disable or enable some dependency. This creates problems similar to the ones discussed above: different distributions might use slightly different changes, which may cause errors when merging them in, and they might make errors that introduce new bugs.

As above, it's just a matter of providing a switch in the build system (like a --disable-feature or --without-feature in autoconf, or a WITHOUT_FEATURE knob for make). If the software has a plug-in infrastructure, binary distributions might also just package the different plug-ins in different packages, allowing the user to choose which ones to install. Software without plug-in structures might require building different packages with different feature sets. For instance, if a software can use either OpenSSL or GnuTLS as implementation of SSL/TLS layers, then the distribution might create two packages, linking to one or the other. The user could then choose between the two.

When some optional dependencies are discovered by the build system, used if present and ignored if not, without a way to tell the software to not build the optional feature that uses a library that is present on the system, we're talking about an automagic dependency. Automagic dependency is a term used to indicate when a package, optionally using another, discovers its presence automatically, without allowing for the user (or the downstream maintainer) to ask not to use it. This kind of dependency is usually a problem just for source distributions, as they build the software on users' systems, which may or may not have the same configuration as the developer working on the build scripts. Binary distributions on the other hand build their code in controlled environments having only the stated dependencies installed. This might actually confuse one of their developers in thinking that a given dependency is mandatory, seeing it enabled in their local build, and not finding an option to disable it.

In general, automagic dependencies should be avoided; having a soft failure default is usually equivalent for the user passing by - you enable the dependency if found, disable it if not found, but still give a way to tell the build system to disable it even when found. This preserves the behaviour intended by the original developers, but also provides the control that (source) distributions want to have over what is built.

Control over how the software is built

Another problem shared both by binary and source distribution is having control on how the software is built. For binary distributions this usually means being able to impose options to the compiler, linker and other tools during the package build process, so they respect their standard options. For source distributions, this means allowing the user to choose the options to provide to the compiler, linker, assembler and other build tools, on a package-by-package basis.

This does not mean that the distributions want to force-feed extra optimisations into software that might be fragile. This seems to be the biggest concern of developers for not wanting to provide a way to change the options used at compile time.

Distributions might want to reduce the optimisations used, or they might just wish to enable (or disable) warnings to more easily spot eventual problems with their packages. Distributions might also want to build debug information, or remove debug messages, and so on. There are a huge amount of possible combinations.

When the distributions want to reduce optimisation, that might be because the need to create packages which work on lower architectures not compatible with these optimisations. Or they know that some of these optimisations are not going to work with their environment. They might know that their version of the compiler does not support the optimisation, or there could be other reasons. Usually, the distribution knows the best way to handle the package for their own environment.

This also leads to a compromise between upstream developers and downstream maintainers: the former should provide their own default options and optimisations, leaving a way to override these defaults as the distributions see fit. On the other hand, distributions should try their best to determine when eventual problems might be caused by their own choice of optimisations. Distributions should not expect upstream developers to fix problems that they have caused with their choice of optimisations. This way, it's usually possible to keep the relationship between upstream and downstream in good terms even when the set of optimisations used is totally different.

More times than not, the problem is not even of willingness of the developers to provide an override, but rather a problem of actually having such an override working. While most distribution developers can fix these problems with relative ease, original developers would probably want to facilitate the work of their distributors by checking their own releases so that setting very minimal options to the compiler will work as intended. A common mistake is hard-setting CFLAGS (or similar variables) in the configure.ac file for autoconf (which otherwise has proper support for user-chosen options).

While we're talking about compiler optimisations it's important to note that for some software, e.g. number crunching software (multimedia applications, cryptography tools, etc.) enabling extra optimisations is desirable. Even so, it should be possible to disable extensive optimisation. These optimisations are usually fragile, and only work in particular environments (compiler type and version, and architectures), so having a way for distributors to decide what they actually want to enable is a very real need.

But having a way to provide options to compiler (C and C++, respectively CFLAGS and CXXFLAGS) is not all that is needed: most modern distributions might want access to the options used by the linker (LDFLAGS) to change the kind of hash tables to be generated, or to enforce particular security measures. For custom-prepared build systems, it's a common mistake to ignore this need, or to support it in the wrong way. Linker options should go before the list of object files, which in turn should go before the list of libraries to link to. This is another common mistake that distributors can fix with relative ease, but it would be better taken care of by the original developers, as it would require repeating the same steps for (almost) all distributions. [This ends part 2 of this article. Stay tuned for part 3, which will cover the philosophical concerns and present some conclusions.]

Comments (3 posted)

New Releases

Fedora Rawhide 20080404 Snapshot Released

A development snapshot of Fedora Rawhide was released April 4, 2008. It's available via bit torrent. "Please us bugzilla to report any problems you find (after making sure that somebody else hasn't already reported the issues). The Beta release notes ( http://fedoraproject.org/en/f9-beta-relnotes ) still mostly apply."

Full Story (comments: none)

GNUSTEP CD 1.6 released

The GNUstep project has released version 1.6, with GNUstep development environment. "As a bonus you get some classic games like nethack, and quite a few network and system recovery and administrator tools. There is also a few 3D and audio programs on it. It's based on the 2.6.24.x Linux kernel, and on the Debian Linux distribution, created using the live-helper package. It is available for i386 and amd64."

Full Story (comments: none)

Mandriva Linux 2008 Spring released!

Mandriva Linux 2008.1 has been released. "lots of super shiny new features like Eee compatibility and easy synchronization and all sorts of good stuff."

Full Story (comments: none)

openSUSE-Education 1.0 RC2 for openSUSE 10.3 is Ready

The openSUSE-Education release for the openSUSE 10.3 release candidate is available. "The latest release includes new LTSP 5 packages, an update to Tuxpaint, added NetBeans 6.0.1, and a slew of other changes and improvements.." See this page for more information.

Full Story (comments: none)

Slackware 12.1 RC1 available

From the April 3rd Slackware-current changelog: "OK, we're going to call this Slackware 12.1-rc1, though there is still some more minor work to do. Please help test! And if we're missing anything major, please let me know at volkerdi[-at-]slackware.com. Thanks. :-)"

Full Story (comments: 1)

News Flash: download Ulteo Application System Beta!

The Ulteo Application System Beta1 is available for download. "The Ulteo Application System is a FREE installable version of Ulteo, that ships with hundreds applications and innovative features..."

Full Story (comments: none)

Distribution News

Debian GNU/Linux

Release Update: architecture status, release goal status, BSPs

Here's a Debian Lenny release update. Armel architecture looks good, Arm and hppa less so. Bug squashing parties are helping to keep the number of RC bugs down, but there are still too many left. gcc-4.3 is now the default on many archs. Click below for details on these and other release topics.

Full Story (comments: none)

Surveying the Debian community!

Paul Wise writes: "As a Debian developer I have on occasion felt a bit out of touch with doing things with Debian and out of touch with other users. I mentioned to some folks at DebConf7 that I felt I focused too much on working on Debian and not actually connected to what the benefit of working on Debian is." To get into touch with the Debian community, Paul has some questions for Debian users and new contributors.

Comments (none posted)

Let's resurrect Debian Weekly News

A call has gone out to help resurrect the Debian Weekly News. "If you like to become an editor or proof reader, please subscribe to the debian-publicity mailing list. But even if you consider your english to be not good enough you can still be of help! Of course it's impossible to be at any place and watch everything, so we need your help to report noteworthy things to the debian-publicity mailing list. That includes watching our mailing list as well as news sites, blogs and other mailing lists about Debian, Linux and IT in general."

Full Story (comments: none)

Second call for votes for the Debian Project Leader Elections 2008

The second call for votes has gone out for the 2008 Debian Project Leader Election. "Votes must be received by 23:59:59 UTC on Saturday, April 12th, 2008"

Full Story (comments: none)

Fedora

Fedora Board Recap 2008-APR-01

The Fedora board met on April 1, 2008. Click below for a brief look at the topics discussed. Notably, after ruling against codeina previously, there has been a reversal and the latest Fedora 8 version has been added to rawhide for Fedora 9.

Full Story (comments: none)

Fedora: Call for Stories

The Fedora Project has issued a call for stories. "We're looking for Fedora Stories -- a person who wants to talk about how Fedora has enabled them to do something interesting or exciting with the innovative technology we provide. We want to use these stories as part of the bigger Marketing Plan for Fedora."

Full Story (comments: none)

Mandriva Linux

Notice of discontinued support for Mandriva Linux 2007.0

Mandriva Linux 2007.0 will reach its scheduled end-of-life on April 13, 2008. "At the same time, Mandriva Linux 2007.1 will be moving into the "base" support phase and will no longer be receiving updates for desktop-related components; only core packages (such as the kernel, etc.) and other networked components (web server, database, etc.) will receive updates for an additional 6 months, until October 13, 2008."

Full Story (comments: none)

SUSE Linux and openSUSE

Advance notice of discontinuation of SUSE Linux 10.1

SUSE Linux 10.1 will soon reach its end of life. "Having provided security-relevant fixes for more than two years, vulnerabilities found in SUSE Linux 10.1 after May 15th 2008 will not be fixed any more for this product. We expect to release the last updates around May 30th 2008. Please do not confuse SUSE Linux 10.1 with the SUSE Linux Enterprise 10 family of products, these are different products and follow different policies."

Full Story (comments: 4)

Distribution Newsletters

Ubuntu Weekly Newsletter #85

The Ubuntu Weekly Newsletter for April 5, 2008 covers Hardy Final``Freeze approaching, New MOTU members, Ubuntu Live registration info, Hardy Release Party Flyers, Launchpad OpenID, Forum News, Matt Zimmerman Interview, New Ubuntu related websites, and much more.

Full Story (comments: none)

PCLinuxOS Magazine Issue 20

The April edition of PCLinuxOS Magazine is out, with the latest news about PCLinuxOS and other topics.

Comments (none posted)

OpenSUSE Weekly News/17

This week the openSUSE Weekly News covers openSUSE-Education 1.0 RC2 for openSUSE 10.3, Tips and Tricks: Quick host-to-host transfer, Stephan Binner: openSUSE's KDE 4.0.3 Packages, Greg Kroah-Hartman: Linux Driver Project Status Report as of April 2008, Reminder: openSUSE project meeting, Event: LugRadio Live USA 2008, and several other topics.

Comments (none posted)

Fedora Weekly News Issue 127

This week's Fedora Weekly News looks at announcements for "Rawhide 20080404 Snapshot Released", "Call for Stories", "Announcement list for Fedora Translation Community", "Purging old Fedora test releases", "Fedora Mirrors Wanted" and "Fedora Unity releases updated Fedora 8 Re-Spin", plus several other topics.

Full Story (comments: none)

DistroWatch Weekly, Issue 247

The DistroWatch Weekly for April 7, 2008 is out. "It was slow news week for distributions, but developers have been quite busy. There were lots of developmental releases last week, including a Slackware 12.1 release candidate. openSUSE and Mandriva announced discontinued support, Gentoo released a beta, and a Debian developer is trying to bring back the Debian Weekly News. I took a look at the new Dreamlinux 3.0 release and while it remained pretty and added some new features, I had mixed results."

Comments (none posted)

Newsletters and articles of interest

Unattended Fedora 8 Installation With NFS And Kickstart (HowtoForge)

HowtoForge installs Fedora 8 with kickstart and NFS. "This document describes how to set up an installation environment with kickstart and NFS. With the resulting system you will be able to run unattended Fedora 8 installations on the client systems in your LAN - additionally, you will save a lot of Internet bandwidth. The whole client configuration can be included into the kickstart file (especially the post-installation script) so you, the admin, will also save a vast amount of time."

Comments (none posted)

10 things to consider when choosing a Linux distribution (TechRepublic)

Over at TechRepublic, Jack Wallen lists ten items to think about before choosing a Linux distribution. His items cover things like whether the distribution is 100% free, security, intended use, community, and more. "Of course, times and opinions change. For nearly 10 years I [rode] the Red Hat/Fedora wagon. And then, after considerable thought, I jumped over to Ubuntu. Why? Because it fit my evolving needs. Many will argue that one Linux distribution is just like another — and I agree, on fundamentals. But when it comes down to everyday use, each distribution is different from the next. So why would you want to use Debian vs. Fedora or Ubuntu vs. Mandriva? Let’s dive into this and find out."

Comments (16 posted)

Interviews

Interview: Jeremy Katz on Fedora Live CDs (Red Hat Magazine)

Jonathan Roberts talks with Jeremy Katz about improvements to Fedora Live CDs. "Are there any other improvements to the Live CDs for Fedora 9, or do you have any that you'd like to get implemented for later Fedora releases? For Fedora 9, a lot of the effort has been in making livecd-creator have an API that is exported and can be used by other tools that want to build images. So, e.g., for LTSP image creation, they're now using the livecd-creator imgcreate module as the backend for building the LTSP client images."

Comments (1 posted)

People of openSUSE: Jakub Steiner

People of openSUSE introduce Jakub Steiner. "When did you join the openSUSE community and what made you do that? Very late. Since Novell acquired Ximian I have been working on the SLE Cleaning up the rough visual edges.products rather than openSUSE. But SLED isn't a good choice if you want bleeding edge software. So to be able to keep up with GIMP & Inkscape development, I started running openSUSE. I started doing branding work for it since 10.1 I think. Maybe here and there earlier."

Comments (none posted)

Gentoo Foundation trustee speaks with LinuxCrazy

Roy Bamford, the newly elected President of the Gentoo Foundation's trustees, spoke with David Abbott from LinuxCrazy. You can download the podcast, or read the transcript created by David Hubbard.

Comments (none posted)

Page editor: Rebecca Sobol
Next page: Development>>