The Grumpy Editor's guide to (some) development distributions

Your editor is not always known for the display of large amounts of good sense. Arguably, this poor judgment is best displayed by the habit of running leading-edge development distributions on systems which are needed for everyday work. Development distributions are where distributors create their next releases; they tend to contain packages fresh from their upstream sources which are not, always, quite ready for prime time. As a result of running these systems, episodes like the following are discouragingly commonplace:

• Two hours prior to leaving to catch a plane to yet another conference, your editor decided to do a "quick" update on the laptop system, which was running the Ubuntu development repository. What followed was two highly focused hours with a rescue CD bringing the system back to a point where it would boot and run a presentation.

• Your editor's desktop, running Fedora's Rawhide, started to exhibit an interesting interaction between Gnash and the X server, with the result that, at random times, random X clients would be abruptly disconnected. The old lesson about frequently saving one's work was well reinforced over the few days it took to track down the problem.

Given that this kind of occurrence is not particularly rare, one might well wonder why your editor continues to run development distributions on mission-critical systems. Natural cranial density may well play into this situation, of course. There are also clear advantages for somebody in your editor's line of work in seeing where distributions - and the applications they ship - are headed before they get the "stable" stamp applied to them. But that does not explain why so many others run these distributions; there is more to it than that.

When one runs a development distribution, one is immersed in the heart of the process that makes Linux happen. There are opportunities to help improve the system (by reporting bugs if nothing else) and to influence the directions that development takes. The only way to effectively test such a complex collection of software is to use it in real-world situations; this testing is a crucial part of the process as a whole. And the truth of the matter is that software which is under ongoing development is alive in ways that occasional, stabilized releases are not. It's sort of like having a small child in the house: development distributions will surprise you every day with some new ability, a new look, or something totally off the wall and humorous.

Of course, again like a child, these distributions will also bring surprises in the form of an occasional, unpleasant mess to clean up, usually at highly inopportune times. It's part of the deal.

LWN tends not to run a lot of distribution reviews, mostly because doing a review which goes beyond commenting on the installer requires using the distribution for real work over a substantial period of time. Your editor has, however, been using these two distributions for long enough to get a sense for the differences between them. So what follows is a review of Rawhide and Ubuntu (through the Edgy, Feisty, and Gutsy development periods - your editor has not, yet, had the courage to jump onto the Hardy train) as development distributions. Needless to say, your editor will not be making comments on the installation process.

Stability

So what does matter to users of development distributions? Clearly, an issue which will always be at the top of the list is stability. One expects to get bitten on occasion, but it would be nice if "on occasion" were as occasional as possible. Getting real metrics on stability of development distributions would not be easy to do, but your editor can say this: Ubuntu seems to break more often than Rawhide does, and in more severe ways. Your editor's Rawhide system has never required reinstallation from scratch - something which cannot be said of the laptop. The sad fact is that Ubuntu, for all its qualities, does have a bit of tendency to ship buggy software; the official releases, too, sometimes have surprises, especially with the more obscure packages. Don Marti's recent experience in this regard is not exceptional.

Tracking a development distribution is hard work; one must, to the extent possible, keep an eye on what is changing and try to figure out when the most auspicious times for an update might be. Rawhide publishes a single message daily with a list of updated packages and known dependency problems (example). Ubuntu, instead, sends out a separate message for every updated package (example). Either way, trying to follow everything that changes from one day to the next is an impossible task for all but the most dedicated reader - the volume is simply too high. So one ends up starting an update, looking at the list of packages to upgrade, and trying to decide whether, given the day's requirements, an update is a sensible thing to do or not. As noted above, your editor occasionally gets this decision wrong.

Coding coding coding Tho the desktops blowing Keeps on near exploding Rawhide Brains, wit and pleasure Debuggin' forever Wishing our code was not to blame All the things that we're missing Typecasts, NULLs and phishing Are waiting at the end of the day -- Alan Cox

The Fedora community has a most helpful resource in the form of the fedora-test mailing list. When something goes badly wrong, somebody will usually post some sort of anguished cry, allowing everybody who is further back in the pack of lemmings to decide to skip the update for that day. Thanks to this list, your editor was happily able to avoid the inadvertent breaking of the X window system which recently burned a number of Rawhide users. Even better, Rawhide developers will occasionally post warnings that a difficult transition is about to happen; for example, Rawhide users are well advised to take evasive action if they care about graphical displays over the next few weeks. Occasional bad poetry (see sidebar) is a special bonus.

As far as your editor can tell, the Ubuntu community lacks an analogous list; there is no public forum specifically for users of the development distribution.

Package management and updates

Another aspect of tracking a development distribution is package management. Rawhide's tool for this task is yum. Your editor has expressed his opinion on this tool before; it is not, shall we say, the yummiest part of the distribution. That said, yum has improved considerably in recent times. Some basic caching means that, at least some of the time, operations like "yum search" no longer allow the consumption of more than one cup of coffee before they complete. The under-publicized skip-broken plugin allows a system to be updated even if an obscure package is uninstallable because of a dependency problem (and Rawhide frequently has dependency problems). Your editor no longer remembers when it was last necessary to remove rpm's cache files by hand. Most of the time, the yum/rpm combination works well enough these days - at least for those of us who are not in a hurry and have a lot of memory.

Ubuntu, of course, uses apt and dpkg. These tools, in many ways, remain nicer than their rpm-based equivalents. Apt is faster and seems to be smarter about complicated updates - though one must still watch closely when doing a dist-upgrade (often necessary with development distributions) to ensure that apt does not stealthily remove crucial packages. Tools like dpkg-reconfigure are most nice to have when an update causes, say, one's mail configuration to take a strange turn. All told, Ubuntu's Debian-based package management tools still are nicer to work with, though the gap is narrowing.

One interesting difference is in the handling of configuration file updates. RPM-based distributions will, when faced with an updated configuration file, put the new version in the target directory with a .rpmnew extension. Except when they, instead, replace the file and leave the old one with a .rpmsave extension. Except when they use .rpmorig instead. Your editor's system currently has 106 .rpmnew files, 41 .rpmsave files, and 103 .rpmorig files under /etc; your editor has no clue about what changes might be indicated by any of them. A diligent user will, of course, read the update log and carefully look to see what has changed whenever one of these files is created. The rest of us just never notice until something breaks, at which point we go in to try to figure out, say, why printing no longer works.

Ubuntu, true to its Debian roots, halts the update process and asks the administrator what to do. Except, on occasion, where it just silently replaces some vital configuration file with a non-working version. Fortunately, that does not happen very often, but there is a much more frequent problem which afflicts all Debian-based distributions: it will often stop and ask what to do about configuration files which have never been changed by anybody. In cases where the file has never required intervention, it would sure be nice not to have to tell the system explicitly that the replacement does not require intervention either. But it must be said that it is better to err on the side of asking what to do than to silently install (or drop) configuration file updates.

In general, Debian-based systems have traditionally been seen as being easier to update - especially from remote repositories - than RPM-based systems. Your editor has not really found that to be the case here, though. The desktop Rawhide system started at Fedora Core 2 and has never seen an installation CD since. As mentioned above, the Ubuntu system has required a couple of reinstallations. Ubuntu has, unfortunately, earned a bit of a reputation for upgrade-related problems; somewhere in the process of civilizing Debian, that distribution's robust upgradability has suffered.

Of course, Rawhide is not perfect in this regard. Your editor still has not had the time to figure out why the PulseAudio transition has left sound in a problematic state, for example. Gracefully replacing core infrastructure on a running distribution is simply not an easy thing to do.

Package selection

With regard to the variety of packages available, Ubuntu used to have a huge lead. When the full set of universe repositories is used, this Debian-based distribution still offers more packages than Rawhide does. In the recent past, Rawhide was, to a great extent, limited to the set of packages offered by Fedora Core. The separate-but-unequal "Extras" repository filled in some of the gaps, but, for the development distribution, Core and Extras were often inconsistent with each other and a system using Extras packages tended to be hard to update. Now there is only one repository and life is better. Your editor has always made a point of having something Debian-based around just because almost any package of interest was only an apt-get away. But now it is rare that a desired tool cannot be found in Rawhide.

Whether one can get the current version of a given program from either of these repositories tends to be a hit-or-miss proposition. Both distributions freeze updates as their release dates approach, with the result that, near the end of the cycle, the packages on offer tend to lag behind what is available upstream. Ubuntu is rather more aggressive than Rawhide, though, in shoveling quite-new software versions in shortly before its release dates. It's worth noting that the Fedora project appears to be planning to change how Rawhide works starting with the Fedora 9 cycle; the stabilized distribution and Rawhide will fork much sooner, allowing Rawhide to blast on ahead during the later stages of the distribution cycle.

Both distributions, sometimes, seem to be a bit over-zealous in churning packages; your editor has never understood why large packages like dejavu-fonts or the OpenOffice.org language packages (or just OpenOffice.org) seem to require daily, bandwidth-sapping updates.

For people who want easy access to packages containing patent-encumbered or non-free software, Ubuntu will clearly work better. Repositories with that kind of stuff are configured from the outset and the pieces fit together nicely. Rawhide users must go to third-party repositories - a move which is likely to introduce dependency problems and make updates harder to do. Meanwhile, this aspect of Ubuntu is, for better or for worse, likely to make life easier for those who want to run a development distribution on a difficult laptop.

Conclusion

Your editor tries to make a point of changing distributions on occasion, just to have a good feel for what is out there. The two development distributions discussed here are far from the only ones out there. Others worth of consideration for any willing victim include:

• Debian sid, which has long been the definitive development distribution. Many developers have been running sid for many years and have never looked for anything else.

• Gentoo. The last time your editor attempted to install Gentoo was some years ago on a very slow system. A few days later, it was still building packages. There is, however, a large group of users (with faster machines) who swear by this distribution.

• Mandriva's Cooker appears to have an active and growing user community.

When looking at the major distributions and their associated development versions, one is struck by one significant absence: there is no development repository available for SUSE Linux. There is an occasional openSUSE development release, but no access to the current development repository. As a result, SUSE misses out on the sort of active user and testing community that these other distributions have. Update: some commenters have pointed out the openSUSE Factory distribution, which your editor obviously missed.

Your editor does not wish to choose a winner among the two development distributions here, much less try to compare them with the other available distributions. One can certainly have a fun, useful, occasionally thrilling ride with any of them. The best thing for anybody with the requisite interest, technical skills (especially in the disaster recovery area), and calm disposition is to pick one and jump into the game. There's no better way to get into the process that creates Linux.

---

(Log in to post comments)

I find Debian testing to be a clever mix of new-yet-sane packages, and for more blood one can
always pull in selected packages from unstable (sid). Showstopper bugs and un-installable
packages are exceedingly rare in testing because of the fact that a package has to last 10
days (usually) in unstable without major problems or broken dependencies to move into testing.

I have exactly the opposite experience. And before anyone brings it up, 
yeah, anecdotal evidence is no evidence at all. Two-three years ago, when 
I tracked debian-testing, I had so much package breakage that I gave it up 
entirely and started tracking sid. Kubuntu came along, and I found it 
easier in many ways... except for the _graphical_ package management. I 
find the combo apt-cache/apt-get are still more reliable than adept to 
update my system, so today I work with 4 kubuntu-gutsy machines: one at 
work, three at home.

I got hit by that bug too, but it has already been fixed in the xfwm4 on 21 september:
http://packages.debian.org/changelogs/pool/main/x/xfwm4/x...

Somehow the package didn't hit testing before libgtk did however, installing the xfwm4 package
from sid solves the problem.

I was a long time Slackware addict until I decided to upgrade my Athlon system by adding two
disk drives which overloaded the power suppy; it never did completely recover, and I upgraded
to a brand new Opteron system.  There being no 64 bit slackware (that is, officially), I went
with gentoo.  I am never sure if I am happy with it or not.  Several upgrades have left me
with a need for a rescue disk, such as deleting a lib used by LVM and making ls (/bin/ls!)
dependent on a /usr/lib library.  USB audio seemed to come and go with various upgrades, tho I
think it has been fairly stable the last few months.

What particularly interested me in my Grumpy Editor's writeup is how the only dependency
problems I have had have been some ebuild screwup which has resolved itself within a few days,
that I have never had a problem getting recent versions of any package, and that it has never
overwritten a config file.  I run the testing version, and have had more than a few runins
with arrogant developers on both the forums and mailing lists.  But the system has been
remarkably trouble free compared to the problems faced by my Grumpy Editor.  I keep on
swearing almost daily that I am fed up with the arrogant developers and some of the constantly
changing config options, but this review makes me think maybe things could be worse elsewhere.
It's been three years, I think, or is it two?

I was running gentoo on my opteron box for a few years, doing updates a couple of times a
week. one thing I really did like about it is that when it ran into an altered config file it
gave you a tool to pick one or the other, or do a diff between them, etc. in many other ways
the etc-update tool needs improvement, but it was very nice in comparison to the issues
described above.

I had two times over a couple of years when the updates gave me problems, however when I had a
power hit take out a hard drive I was unable to get a current installer to boot on the system
well enough to get going (this was a week after moving so the availablility of distro boot
CD's was limited). I ended up throwing ubuntu server on there, and so far (through a dozen or
so minor updates and one release update) I've had no problems.

I tend to stick with the releases, although I have several packages that I download and
compile myself instead of useing what the distros provide.

Speaking of Gentoo:
Gentoo is suitable for stabel systems as well as being on the edge of development. I use it on
critical servers as well as on developing systems.

Gentoo has "emerged": using the live  and install cd you can easily get up to a running system
without compiling. This is your starting point.

the package management is done by a tool called portage and this tool has some advantages: you
can decide implicitely or explicitly for wich packages you want "stable" or "development"
conditions by using package masks. You can even select the versions to use / not to use in
fine granularity. if you want to update a package, you can take a snapshot of the old version
and then upgrade, leaving you a backdoor if something goes wrong.

gentoo provides a tool to do check your installation for security related updates and perform
this automatically.

Another advantage if you are a frequently-upgrading person is that gentoo leaves your config
files untouched and provides tools to do the merge process. this saves some anoyances after
upgrading.

if the update works, you can build a package and update other similary configured computers
from this.

Gentoo has drawbacks, too:
Config options are poorly documented, you need to search the forum for information. There are
no fully functional shiny gui tools for doing updates.
Compiling from source needs some patience. To get the X desktop as functional as in ubuntu you
need to manually select apropriate software and configure it to work correctly - without
documentation.

But if you want to be on the edge, this may not be a showstopper for you.

I was a bit disappointed... From the title I was hoping to find out which distribution was the
best for developing software on :-)

But openSUSE has the current development available.

It is in the "Factory" branch, which reflects the current
development tree.

Read up on it here: http://en.opensuse.org/Factory

Ciao, Marcus

It seems like quite a few people - including one grumpy editor - missed 
one of the core changes when the distribution changed its name from SuSE 
to openSUSE: The development process is now open.

I disagree with the classification of Gentoo as a development 
distribution. Being source-based does not automatically make it a 
development distribution. You can choose if you want to use unstable 
packages or not. By default you will not be using unstable packages.

> I disagree with the classification of Gentoo as a development distribution. [...] By default
you will not be using unstable packages

Using only stable packages in Gentoo is not the same as using, say, a "stable" version of
Debian. A "stable" OS in most people's mind is when you install a distro, configure it, and
can forget about the OS itself except doing regular updates (security and otherwise important
bugs). It is not the same in Gentoo. Today you installed, configured it, everything works
fine. Tomorrow you do "emerge world" and - oops - libexpat now version 2.0.something instead
of yesterday's 1.95.something. Both are stable. But the library sonames are different, so now
you have to revdep-rebuild all the related packages (which are dozens, both system- and
desktop-level), and until the process successfully finishes (if it does), your box is
literally unusable. And that's before we talk about your own soft which depends on expat (and
need to be recompiled manually) or, worse yet, third-party binary packages where simply there
is no hope.

I used to run Debian testing and unstable, and Gentoo is something like these in respect to
stability and maintenance headaches; YMMV, of course. However, Gentoo provides an
unprecedented level of package customization (both in terms of version number and build-time
options), and that's why I like it - it's been on my work desktop and 2 servers for more than
3 years. It was also on my previous laptop, but for the new one I decided on Ubuntu, since for
the relatively small amount of time I use it heavily (mostly for trips abroad), the
maintenance time is disproportionally large.

Particularly when employing the Paludis package manager. http://paludis.pioto.org

The expat update (and the similarly timed curl update) were, in my opinion, completely
botched. The way it's supposed to work is that both versions of expat would be installed at
the same time (since they don't have any overlapping files, since the soname of the library
changed, which is the fundamental solution for dependancy problems), and the older one could
be discarded once you'd rebuilt all the packages that had been built against the old library.
Unfortunately, the package maintainer decided to discard the older library at once (since the
system isn't set up to discard it automatically when it is safe to do so), which made
everything that uses it break in the transition (and revdep-rebuild isn't quite clever enough
at package ordering to necessarily fix the broken packages you need to use to build other
broken packages first, although it tries).

So there is a mechanism in place to avoid this problem, and it's used for a lot of packages,
but was not used in this case, which is why there were problems.

> So there is a mechanism in place to avoid this problem

You're absolutely right, there was a way to handle this specific case gracefully. Still, such
a situation (soname change) is impossible in the first place in most "stable" distributions. I
could use another example, e.g., once a udev update added a non-standard user to one of
/etc/udev/rules.d/*, with a result that my ldap-pam based system hung for several minutes
during the boot; etc. In short, when versions of _all_ packages change from time to time, a
chance for a, let's say, surprise is obviously higher ;-) One should certainly have this in
mind when opting for Gentoo.

An soname change may be impossible in a particular release of a stable distribution, but there
have been rather a lot of soname changes in "Debian stable" over the years. Of course, this
only happens at a new release, but that tends to make release times especially tricky. This is
particularly true because stable distribution upgrades are a rare enough event that the
procedures are never sufficiently carefully tested to be a suitable operation to perform on a
stable system.

The Gentoo mechanism for the stable version is pretty much unique in that you can run a system
forever without ever doing anything that's either beyond end-of-life or is outside of the
normal update process, whereas other systems require an upgrade, outside of the normal
process, eventually. Of course, the actual Gentoo "stable" distribution doesn't necessarily
avoid causing occasional breakage, but that's an implementation issue rather than a design
issue.

It's not that we wanted to give you a world of pain. It was that we wanted to avoid lots of
pain that is involved in this specific case when two DIFFERENT ABI version of libexpat are on
a system. Library A depends on expat and library B depends on library A and expat - Library A
gets rebuilt and will use libexpat.so.1 version, while library B still uses libexpat.so.0.
User launches an application linking to Library B, both expat versions are mapped in and
providing symbols, things get mixed up between them, enter a world of chaos and random hard to
explain crashes caused by the specific case this library works and is used, which made
preserve_old_lib method usage not a good option as that leads to exactly that chaos crashy
world.

We do not have a mechanism in place to avoid that. Manually it was possible to workaround it
by making a libexpat.so.0 pointing at libexpat.so.1, because then the same library is used by
both library A and B, but in this instance this workaround it worked just for most cases due
to the ABI breakage being small.

And yes, I am one (out of about three or four) persons responsible for the decision on which
route we went with the expat upgrade and I stay by it.

> Today you installed, configured it, everything works fine. Tomorrow you do "emerge world"
and - oops - libexpat now version 2.0.something instead of yesterday's 1.95.something. Both
are stable. But the library sonames are different, so now you have to revdep-rebuild all the
related packages (which are dozens, both system- and desktop-level), and until the process
successfully finishes (if it does), your box is literally unusable.

There are some tricks you can do with CONFIG_PROTECT to force portage to save old versions of
library files.

> There are some tricks you can do with CONFIG_PROTECT to force portage to save old versions
of library files.

CONFIG_PROTECT is about protecting _config_ files...

> CONFIG_PROTECT is about protecting _config_ files...
That does not mean you can not (ab)use it for protecting library files.

Or without the double negative...
CONFIG_PROTECT does work to protect non-config files.

as well as the mailing list for ubuntu changes there is an rss feed
http://media.ubuntu-nl.org/rss/

a lot of testers hang out in http://ubuntuforums.org/forumdisplay.php?f=305 or #ubuntu+1 on
freenode

Indeed apt-listbugs has saved me more than once when upgrading my sid and testing systems. I
would recommend installing it to anyone using these suites. Of course it requires a bit of
training to quickly judge from the bug report titles which ones are truly important and need a
closer look.

Perhaps ArchLinux could be considered amongst "development" distributions. 
It seems to track the latest releases of packages, and maintains a testing 
branch (available to users) where updates are quarantined for a time.

My Arch highlights:

- Binary packages
- Pacman PKGBUILD build system - a package management system simple enough 
for the average joe to create their own packages.
- Always very fresh versions of software

www.archlinux.org

And the reality of things breaking from time to time. The worst so far 
for me (having used Arch for only a year or so) was the Xorg 1.4 update 
which totally borked my ati setup. I'm now running the closed ati drivers 
until I have time to try the open ones again.

That being said, I've had more lockups requiring hard reset in the last 
week of running the Ati closed drivers than I did all the time running 
Arch, Gentoo (years) and cvs/svn builds of KDE3. Closed sucks.

Derek

I use Gentoo for multiple things--stable on my home server, and lots of unmasked development
versions on my desktop.  People often think that the selling point of Gentoo is some sort of
optimization ("it's compiled just for your processor!!@!!!") but that's missing the
point--it's about choice and flexibility.

I run Gentoo on my server because it supported an installation to EVMS fairly easily.  I run
it on my desktop because it easily lets me choose some bleeding-edge packages which, since it
compiles from source, tends to let the bleeding-edge packages play well with the stable ones.

I've tried other distributions; every (and I do mean EVERY) debian-based distribution where I
tried mixing bleeding-edge packages/repositories with the stable repositories wound up getting
apt wedged horribly to the point where I couldn't easily recover.  I've NEVER had that problem
with Gentoo; I think it excels as a distribution--if you're willing to pay the cost of
compilation (it's certainly made me leery of heavyweight desktop environments--I now run wmii
or dwm!).

And speaking of configuration file management--etc-update is good, but dispatch-conf (with
some tweaking to use colordiff to show differences, emacs-diff to easily merge, and CVS to
track changes) is a fantastic way to manage your configuration files.

It's not for everyone, but Gentoo really is a fantastic distribution if you value flexibility
highly.

I agree. Gentoo's flexibility is far and away its best feature. The flexibility of selecting
which level of stability is desired for which packages is a great way to ensure that core
essential packages like glibc, xorg, etc remain stable, but less essential packages, desktops,
etc, can get the unstable treatment, all with a minimal amount of fuss.  Now the compilation
times can be a hassle, but distcc and a pair of dual core machines can take a big bite out of
that. In fact, for smaller packages, it almost seems faster than having apt grind away at the
hard drive on its db for a while before downloading the binary and then grinding away at the
drive some more after installing (this anecdotal perception may be from my experience with
Ubuntu that was on an older laptop with a slower hard drive).

After having trying Fedora and Ubuntu, I got back to Debian/sid (in fact I never erased the
partition). Even in unstable form, I find debian extremely resilient to bad packages, and
aptitude is the software I miss everywhere else.

Aptitude marks all dependencies of a package you install as "automatically installed", which
means if you uninstall the package, all those unused dependencies will be uninstalled too. Its
conflict resolution system is far more advanced too. All in all, aptitude makes apt-get (and
even more yum/rpm) look like a primitive toy in comparison.

Sure, Ubuntu has aptitude, but it can't really be used here: their update tool has some
knowledge needed to do some cleanup on each version update, so in a sense they perverted the
dependency system by adding parallel information somewhere else.

Try aptitude for a while, and you'll never get back to something else ...

I use aptitude for everything on Ubuntu, and am beginning to like it.  I'm not sure why 'it
can't be used on Ubuntu' - you can avoid update-manager entirely even for dist-upgrades if you
are confident you can fix breakages, but perhaps you could explain what you meant here.

Agreed, the functionality is leaps and bounds better than it used to be.

Unfortunately the user interface is still terrible.

I've been using aptitude on ubuntu since warty.  Have upgraded my machine from warty to hardy
(yes, laptop runs hardy) via aptitude since then.  While I've had issues following ubuntu
development, its usually quickly fixable by jumping into IRC for a few minutes as I'm usually
not the only to have the problem.

I've had some problems at distribution upgrade (from 6.10 to 7.04 IIRC) with aptitude, and
have been told that those upgrades *have* to be made with the upgrade_manager, because it does
more things than simply upgrade.
Effectively, on another laptop using the recommended tool the upgrade process did a big
cleaning of some packages and installed some more. Sorry, I can't remember which ones.

Rawhide is a development distribution. DejaVu is updated often in rawhide because:
- its upstream is very dynamic (many releases with lots of additions needing testing)
- the Fedora package innards have heavily changed over time (not that easy to plug in a
default font dynamically)
- it's highly user-visible, and discovering some of those changes broke the system default
font on some systems a month before release time is not acceptable
- I like it that way :p

It's not updated often in Rawhide because users "need" it updated often, it's updated often
because there is a lot of development on this package and rawhide is there to track this kind
of changes and detect regressions early. The dynamics of a development distro are not the same
as those of a stable version.

>Two hours prior to leaving to catch a plane to yet another conference, your editor decided to
do a "quick" update on the laptop system, which was running the Ubuntu development repository.
What followed was two highly focused hours with a rescue CD

And the moral of the story: do not make your test systems your regular systems. Install
multiple distros alongside each other on different partitions, or probably even better yet,
just use something like VMware.

> Except, on occasion, where it just silently replaces some vital
> configuration file with a non-working version. Fortunately, that does not
> happen very often, but there is a much more frequent problem which
> afflicts all Debian-based distributions: it will often stop and ask what
> to do about configuration files which have never been changed by anybody.

Both of these are usually due to packaging bugs.


> Ubuntu has, unfortunately, earned a bit of a reputation for
> upgrade-related problems; somewhere in the process of civilizing Debian,
> that distribution's robust upgradability has suffered. 

It's an old myth that Debian's quality is mainly due to dpkg and APT. It's
mainly due to packaging policies and practices.

I don't see why the editor doesn't multi-boot a stable and a development version of these
distros - most hard disks are enormous relative to distro sizes these days, and it's easy
enough to set this up along with a separate /home partition.  If the development distro
breaks, simply reboot into the stable version and fix the problems from there, with chroot if
needed.  This is particularly useful for laptops and where you want good performance when
recovering the development distro.

I really like Ubuntu but I do find its upgrades are not quite as problem-free as advertised,
particularly the dist-upgrades (e.g. update-manager breaks when used with apt-cacher).
Cloning the stable distro before doing an upgrade to a new stable version, or a development
version, is therefore not a bad idea.

"your Editor" may want to explore the wonderful world of virtualization. It works great for
things like development distributions.

>Apt is faster and seems to be smarter about complicated updates

Apt misses some dependencies sometimes - at least this has been my motivation to switch to
"smart".

>Your editor's system currently has 106 .rpmnew files, 41 .rpmsave files, and 103 .rpmorig
files under /etc; your editor has no clue about what changes might be indicated by any of
them.

.rpmnew: The author of the RPM package decided that the configuration file in question --
possibly modified by the user -- is better left as-is (%config(noreplace) in the spec file).
This is the case with e.g. smb.conf, where you certainly would not want the defaults to come
back. Instead, the defaults are put into smb.conf.rpmnew.

.rpmorig: The author of the RPM package decided that the configuration file in question is
safe to overwrite (%config in the specfile) -- but at least keeps a copy of the user's
changes. This is the case with /etc/wgetrc and all the obscure stuff in /etc/gnome you
normally never touch in your regular life.

.rpmsave: This happens when you remove the RPM package and the file in question is tagged
%config or %config(noreplace), and was modified by the user. In case someday you reinstall the
package, your old settings (e.g. smb.conf again) are retained until you manually delete them
or put them back into place.

It's all sane.

>Ubuntu, true to its Debian roots, halts the update process and asks the administrator what to
do. Except, on occasion, where it just silently replaces some vital configuration file with a
non-working version.

...which never happens with rpm (assuming the spec file was properly written). Also, rpm is
designed for non-interactivity, so we can't just ask people questions. I agree that you could
whack up something with isatty(), but let's better not.

>In general, Debian-based systems have traditionally been seen as being easier to update -
especially from remote repositories - than RPM-based systems.

RPM does not need the funny dpkg --purge thing, for one.

>With regard to the variety of packages available, Ubuntu used to have a huge lead. When the
full set of universe repositories is used, this Debian-based distribution still offers more
packages than Rawhide does. 

Sometimes, smaller is better. The ever-increasing number of packages per repository bloats the
metadata files and makes YourFavoritePackageManager slower in processing. Splitting up
repositories into categories (example: opensuse.org/repositories/) is a good idea IMHO.

> Apt is faster and seems to be smarter about complicated updates - though one must still
watch closely when doing a dist-upgrade (often necessary with development distributions) to
ensure that apt does not stealthily remove crucial packages.

'upgrade' will never remove packages, so one can do an 'upgrade' run first, followed by a
'dist-upgrade' invocation that (if it does anything at all) can be pondered more thoroughly.

FWIW, apt will not remove essential packages (those that are needed to install other
packages), but with "crucial" you obviously meant stuff you need for your work. These
packages, once identified, can probably be protected through some apt_preferences(5) magic.

> Ubuntu, true to its Debian roots, halts the update process and asks the administrator what
to do.

dpkg's --force-conf{old,new,def} options can prevent these questions. It will work very much
like rpm then.

I wonder why none of these distributions offers a three-way merge between old-distro-version
new-distro-version and user-modified-version. By wasting a bit of diskspace it's trivial to
do.

You need yum merge-conf!  This will address your issue 
with .rpmsave/.rpmnew.

One warning though - merge-conf doesn't work well for non-interative yum 
update (e.g., yum-cron).

The Fedora kernel SRPM contains a broken-out patch series.  The Ubuntu .deb source package for
the kernel contains only a single, merged patch to the kernel.org sources.