Do we need this crap on LWN? [LWN.net]

Do we need this crap on LWN?

Posted Feb 21, 2007 22:59 UTC (Wed) by markc (guest, #4419)
In reply to: Do we need this crap on LWN? by ofeeley
Parent article: ESR's goodbye note

But if the folks pushing RPM are considering improvements, after it's
development was almost abandoned for a few years, then it would be a major
plus for all of linuxdom if they wholesale dropped rpm altogether, adopted
the deb system, and then added what they feel might be missing to debs.
Then in one 1/2 year painful changeover period for rpm users (maybe, maybe
not) we'd have more than 75% of the linux distro universe using the same
package format and, also, Debians current huge and well organised
repositories would be INSTANTLY available to RH/Fedora users. That would
do more, in 6 months, to unify "linux" everywhere than all the talk and
LSBing we've had this century. Right now is such a golden opportunity for
RH/Fedora to make one of the most sensible changes, tht they have not even
contemplated, with the largest positive outcome for all concerned.

RH/Fedora and spinoffs would gain a packaging system that already has the
essential improvements they seek AND a huge swag of ready made packages
(minus the ongoing transitional RPM package format nightmare, possibly for
years). The dpkg/apt folks would gain a new swag of interested developers
and a new perspective on how to make debs even better than they are now.
In twelve months, at least three quarters of the linux universe would be
hugely better off. We would all have a major major league win and provide
an instant doubly powerful front to coerce both hardware and software
vendors to support linux (one pkg format to bind them all) and hedge
against any future M$ patent threats (they would no longer be able to pick
on just RH... well, it could help somewhat). Oh well, I can dream.

(Log in to post comments)

Do we need this crap on LWN?

Posted Feb 22, 2007 0:40 UTC (Thu) by ofeeley (subscriber, #36105) [Link]

"RH/Fedora and spinoffs would gain a packaging system that already has the
essential improvements they seek"

What are those improvements that are desperately sought by us? Without the slightest trace of smartassedness I don't know what I'm missing. I have used yum for quite a while now and am very happy with it (I haven't bothered to look at pirut or pup although I did use pup's predecessor up2date for years). I've had a few problems but not many and nothing that I couldn't fix relatively easily.

"AND a huge swag of ready made packages"

The fedora repositories already have a huge swag of ready-made packages, just from within the officially blessed Core and Extras (which have now merged). If you add in unofficial repositories then it looks like there /may/ be a slight advantage of around 10% in terms of numbers of deb format packages as previously discussed: http://lwn.net/Articles/222681/

"(minus the ongoing transitional RPM package format nightmare, possibly for
years)."

What is this transitional nightmare? In all seriousness I've never heard of it.

Do we need this crap on LWN?

Posted Feb 22, 2007 3:30 UTC (Thu) by markc (guest, #4419) [Link]

> What are those improvements that are
> desperately sought by us?

FWIW, I can only speak from personal experience. I inherited control of a
CentOS 3.6 vhost and was told on some Plesk forums it was not possible to
upgrade to CentOS 4.3 (me gobsmacked). I recently got a VPS with Debian
3.1 installed, within 2 weeks I upgraded it to Ubuntu Dapper then on to
Debian Etch (v4) then back across to Ubuntu Feisty (because Debian doesn't
have a nanoweb package) with zero manual intervention. I was impressed and
would not dare to attempt an analogous path between Redhat and Fedora /
whatever. Maybe experienced RPM based folks are not "desperate" but I was
to upgrade a CentOS 3.6 system so it could run a later Plesk control panel
and, in my worldview, it was not going to be possible.

> The fedora repositories already have a
> huge swag of ready-made packages...

Nyah nyah, my deb fanboy thing is bigger than your rpm fanboy thing. The
fundamental point was that if the RH/Fedora folks started using the deb
system then "we all" would have a much larger combined system than
separately. IMVHO if one or the other system was to be chosen as the base
to extend from then I would imagine most people would pick deb and drop
rpm. It would be interesting to see an open vote on such a decision.

> What is this transitional nightmare?

The current stability of the rpm system is because it has not been
upgraded or hacked on for a few years. Once the developers start coming
out with improved betas I can imagine there will once again be "upgrade
problems". Making a clean break to the deb system *may* be less painful
than ongoing rpm packager updates.

The underlying point is that there is an ever so minute opportunity to
have one single MAJOR packaging format for linux systems and *if* it was
vaguely possible for one of the 2 major players to drop their current
format and go with the other then that opportunity is/was *now*. The
combined resources of both systems would present, essentially, a single
unified linux package system for both commercial and open providers and
such a move would become the single most notable advancement for linux
this decade. One of the reasons this won't happen is because of rpm
fanboyism, as you have clearly demonstrated, so "we" shall remain a
fragmented group of fanboys.

Do we need this crap on LWN?

Posted Feb 22, 2007 5:42 UTC (Thu) by bojan (subscriber, #14302) [Link]

> I inherited control of a CentOS 3.6 vhost and was told on some Plesk forums it was not possible to upgrade to CentOS 4.3 (me gobsmacked).

There is a reason for not being able to upgrade a running system using "yum upgrade" - it's udev. It removes the old dev package during upgrade and your system dies.

Otherwise, it's a perfectly doable thing. You could even script it using kickstart. Done many upgrades of RHEL3 to RHEL4, completely unattended.

> One of the reasons this won't happen is because of rpm fanboyism, as you have clearly demonstrated, so "we" shall remain a fragmented group of fanboys.

See: http://kitenet.net/~joey/pkg-comp/

For instance, Debian packages cannot track file dependencies, something that distributions like Fedora rely on heavily. Actually, Fedora packaging policy is to prefer dependencies based on what the build process determines, rather than manual ones, which can be easily forgotten. Good or bad, that's the Fedora way...

Do we need this crap on LWN?

Posted Feb 22, 2007 6:42 UTC (Thu) by markc (guest, #4419) [Link]

> There is a reason for not being able to upgrade
> a running system using "yum upgrade" - it's udev.

So in the case of a CentOS 3.6 -> 4.3 upgrade it would not be advisable
for an inexperienced rpm user to upgrade a remote vhost.

> See: http://kitenet.net/~joey/pkg-comp/[1]
> For instance, Debian packages cannot track
> file dependencies, something that distributions
> like Fedora rely on heavily.

11. Some people consider file dependancies a gross misfeature.

Do we need this crap on LWN?

Posted Feb 22, 2007 7:52 UTC (Thu) by bojan (subscriber, #14302) [Link]

> So in the case of a CentOS 3.6 -> 4.3 upgrade it would not be advisable for an inexperienced rpm user to upgrade a remote vhost.

Oh, it's absolutely advisable. Just use the upgrade process (anaconda runs that). You just can't do it while the system is hot.

> 11. Some people consider file dependancies a gross misfeature.

And, quite obiously, some don't.

Do we need this crap on LWN?

Posted Feb 22, 2007 11:53 UTC (Thu) by markc (guest, #4419) [Link]

> Oh, it's absolutely advisable. Just use the
> upgrade process (anaconda runs that).

What is anaconda and would it be included in a CentOS 3.6 system ?

> You just can't do it while the system is hot.

There we go, the aspect of RPM management that is "desperately needed"[1]
and equally denied by those who know not. The fairly radical cross distro
upgrade procedure I describe earlier did not require a reboot and no
service was down for more that 30 seconds max. That, to me, is an
essential part of running SERVERS where uptimes rule and why I have always
used Debian based systems. I have one box that has been up for 464 days
and started with a Debian Sarge 2.6.12 devfs enabled kernel and now runs a
Ubuntu Dapper udev enabled system, on the same kernel, without a reboot...
the very excuse I was given for why upgrading from CentOS 3.6 to 4.3 was
not possible. It seems that unless RPM based folks actually use a Debian
based system they no not what they are missing out on.

[1] Refer to previous comments above.

Do we need this crap on LWN?

Posted Feb 22, 2007 15:51 UTC (Thu) by pizza (subscriber, #46) [Link]

> What is anaconda and would it be included in a CentOS 3.6 system ?

Stick the CD in, it boots up. That's anaconda.

> here we go, the aspect of RPM management that is "desperately needed"[1]
and equally denied by those who know not.

You're confusing "The RPM tools" with "The Debian Packaging Guidelines as implemented using the dpkg tools". Get your comparison straight.

> I have one box that has been up for 464 days and started with a Debian Sarge 2.6.12 devfs enabled kernel and now runs a Ubuntu Dapper udev enabled system, on the same kernel, without a reboot... the very excuse I was given for why upgrading from CentOS 3.6 to 4.3 was not possible. It seems that unless RPM based folks actually use a Debian based system they no not what they are missing out on.

"RPM-based folks" are apparently missing out on a lot of remote exploits and possible filesystem corruption!

I reboot my machines every 90 days (if not sooner due to power outages or whatnot) solely to install new kernels and run disk integrity checks.

I don't give two hoots about SERVER uptime; to me SERVICE uptime is all-important, and I achieve that with redundancy and performing test upgrades on non-critical systems in advance.

Do we need this crap on LWN?

Posted Feb 22, 2007 16:41 UTC (Thu) by markc (guest, #4419) [Link]

> Stick the CD in, it boots up. That's anaconda.

Great, but not on a dedicated host 10,000 miles away.

> You're confusing "The RPM tools" with "The Debian
> Packaging Guidelines as implemented using the dpkg
> tools". Get your comparison straight.

I think I'm comparing the use of any number of (unfamiliar to me) rpm
tools with apt-get where the later seems to have the edge... which is the
point that ESR seems to have finally stumbled upon in the head article.

> "RPM-based folks" are apparently missing out on a
> lot of remote exploits and possible filesystem corruption!

Some would argue it's not appropriate to have such file system checking
built in to a package management system.

http://packages.debian.org/cgi-bin/search_packages.pl?key...

> I don't give two hoots about SERVER uptime; to me
> SERVICE uptime is all-important, and I achieve
> that with redundancy and performing test upgrades
> on non-critical systems in advance.

Dare I suggest that if you used a Debian based distro you could go live
and stay live with less redundancy and pre-testing. My example of swapping
between 4 deb based distros on an el-cheapo commodity $15/month VPS
without losing any downtime was intentional. No redundancy or testing
needed, just good software, especially on the barest metal.

Do we need this crap on LWN?

Posted Feb 22, 2007 17:18 UTC (Thu) by pizza (subscriber, #46) [Link]

>> "RPM-based folks" are apparently missing out on a
>> lot of remote exploits and possible filesystem corruption!

>Some would argue it's not appropriate to have such file system checking
>built in to a package management system.

No; I was referring to the fact that you can't fsck a live filesystem, and that it's an unfortunate fact of life that filesystems are at best only as reliable as the disks they run on -- errors silently happen.

For me, periodic reboots serve two purposes -- Perform filesystem consistency checks, and install updated kernels (I am quite glad the 2.6.x.y series exists!)

It's more commonly known as scheduled preventative maintainence, and through the use of multiple physical systems, I avoid service downtime. (Even with virtualization, the host system needs occasional TLC too!)

> Dare I suggest that if you used a Debian based distro you could go live and stay live with less redundancy and pre-testing.

Been there, done that, been burnt by it. As I've alluded to many times in my posts here, it's the policies that you follow that ultimately matter. I discovered the hard way that using Debian didn't make things any simpler for me -- It just had a different set of quirks to watch out for.

Granted, this was a few years back, and Debian has addressed quite a few of the problems I had at the time... but as I've also said, if I don't see any tangible benefit to making a change, why bother?

Do we need this crap on LWN?

Posted Feb 22, 2007 20:01 UTC (Thu) by bojan (subscriber, #14302) [Link]

> Great, but not on a dedicated host 10,000 miles away.

You missed the unattended upgrade bit above. The one with kickstart.

Let me repeat again. This particular limitation (of not being able to upgrade CentOS 3 to CentOS 4 while running) has nothing to do with RPM. It has to do with the fact that CentOS 4 (i.e. RHEL4) uses udev and CentOS 3 (i.e. RHEL3) uses a dev package. When dev gets removed by upgrading to udev, your system is hosed (you have nothing left in /dev).

Otherwise, I've done plenty of hot upgrades by doing "yum upgrade" on many different versions of Fedora.

> which is the point that ESR seems to have finally stumbled upon in the head article

He stumbled upon his on inability to manage his own system and/or ask others how to do it.

Do we need this crap on LWN?

Posted Feb 22, 2007 20:15 UTC (Thu) by loening (subscriber, #174) [Link]

Speaking of remote upgrades with RPM. I personally have done
RH8->RH9->FC1->FC2->FC3->FC4->FC5->FC6

On a workstation that sits 300 miles away from me that I haven't seen in years.

dev => udev

Posted Feb 25, 2007 14:54 UTC (Sun) by smurf (subscriber, #17840) [Link]

So please enlighten us poor souls who have *no*idea* why you'd need to "clean out" /dev when moving to an udev-based system. Why not leave the old package-supplying-/dev-nodes installed?

You can mount a tempfs onto /mnt/whatever, create the bare essential device nodes there, move-mount that to /dev, and start up udev. (In fact, this is how a udev-based system boots in the first place.)

Total downtime for programs attempting to open /dev/null (or whatever): Zero.

Afterwards, if you do want to remove the old /dev-supplying package, well ... RPM has a --justdb option.

dev => udev

Posted Feb 26, 2007 4:27 UTC (Mon) by bojan (subscriber, #14302) [Link]

> why you'd need to "clean out" /dev when moving to an udev-based system

Well, if you run "yum upgrade", it is a fact that that's is what's going to happen: upgrade to udev will remove dev, which will remove all entries in /dev.

As you explained, it is not necessarily what needs to happen - but it does by default. So, unless you're prepared for some creative surgery (as per you explanation), you will get screwed.

Coming back the original question of supported upgrades from CentOS 3 to 4 (or RHEL 3 to 4), it is obviuos why this is not a recommeded thing to do - people don't normally want to recommend a complicated and risky process that would in the end require a reboot anyway (new kernel). RHEL packages were designed to be upgraded using the upgrade process, not "yum upgrade" (remember: RHEL 3 and 4 don't even ship yum, it's a CentOS addition).

You can find some more info here (search for udev):

http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manu...

Note how you have to be on 2.6 kernel before starting this. CentOS 3 (i.e. RHEL 3) is based on 2.4 kernel.

Instead, one cuts a small kickstart file, starts an upgrade and after some minutes has a fully upgraded system with far less risk of stuffing things up.

Do we need this crap on LWN?

Posted Feb 22, 2007 13:18 UTC (Thu) by ofeeley (subscriber, #36105) [Link]

"I recently got a VPS with Debian
3.1 installed, within 2 weeks I upgraded it to Ubuntu Dapper then on to
Debian Etch (v4) then back across to Ubuntu Feisty (because Debian doesn't
have a nanoweb package) with zero manual intervention."

If (as is probable from you calling it a VPS) it's a SWsoft/OpenVZ/Virtuozzo based VPS then the problem that you describe will definitely occur _if_ the hosting provider hasn't made the right templates available.

The only way that major upgrades across kernel versions can work with that sort of virtual server is if the kernel (and related infrastructure like the /dev/* stuff, glibc etc) that are pulled down when you do an update are matched by changes in the hypervisor.

The lack of upgradability of one CentOS version to another is more an issue of what your VPS hosting provider has made available than any limitation of rpm.

I don't know if the same limitations apply to other virtualisation techniques (Xen, kvm etc). I have experience of exactly the problem you're describing and it irritated me, but I blame the hosting company ;)

I can only assume that either you had pinned the kernel in your migration to/from various Ubuntu versions, or the hosting provider had templates available for all those versions.

Thanks for the rest of your clear and detailed answer.

Do we need this crap on LWN?

Posted Feb 22, 2007 13:35 UTC (Thu) by ofeeley (subscriber, #36105) [Link]

"One of the reasons this won't happen is because of rpm
fanboyism, as you have clearly demonstrated, so "we" shall remain a
fragmented group of fanboys."

It's a pity you ruined an interesting post with a personal attack. I don't think that asking for particular information as to the specific advantages of a system that I'm largely unfamiliar with, and correcting your mistaken impression that there are few rpm packages is fanboyism. I agree that a single packaging format would be useful in terms of reducing apparently duplicated effort.

Do we need this crap on LWN?

Posted Feb 22, 2007 14:13 UTC (Thu) by markc (guest, #4419) [Link]

> It's a pity you ruined an interesting post with a
> personal attack.

Thanks for pointing that out, I'll try and keep a more impersonal
perspective in the future. I don't often get involved in threads like
this. To clarify the kernels involved in the personal experiences I
quoted...

>> FWIW, I can only speak from personal experience.
>> I inherited control of a CentOS 3.6 vhost and was
>> told on some Plesk forums it was not possible to
>> upgrade to CentOS 4.3 (me gobsmacked).

This was a dedicated host with a kernel that probably came default with a
CentOS 3.6 install.

>> I recently got a VPS with Debian 3.1 installed,
>> within 2 weeks I upgraded it to Ubuntu Dapper then
>> on to Debian Etch (v4) then back across to Ubuntu
>> Feisty (because Debian doesn't have a nanoweb
>> package) with zero manual intervention.

This was indeed a recent experience on a Virtuozo VPS with a 2.6.9 kernel.

>> I have one box that has been up for 464 days
>> and started with a Debian Sarge 2.6.12 devfs
>> enabled kernel and now runs a Ubuntu Dapper
>> udev enabled system, on the same kernel,
>> without a reboot.

This one, I think, is a custom rolled kernel with devfs compiled in and I
happened to include CONFIG_SYSFS=y. Ah, I see, if I rebooted on this same
kernel then I may have problems, however, the kernel and grub settings
installed by Ubuntu Dapper would bring up a newer 2.6.15 kernel.