Do we need this crap on LWN? [LWN.net]

Do we need this crap on LWN?

Posted Feb 21, 2007 17:11 UTC (Wed) by ofeeley (subscriber, #36105)
In reply to: Do we need this crap on LWN? by evgeny
Parent article: ESR's goodbye note

Well, there seem to be a lot of changes already happening around RPM after a period of relative stasis: http://wiki.rpm.org/Docs/RpmOrgFAQ

So an alternative to dumping the whole thing is to fix whatever needs to be fixed and it seems like there's a serious effort going on to do that.

It's worth pointing out that for all the hysteria spread about RPM (and dependency resolvers/updaters built on top of it -- such as YUM), it mostly just works if you're just using it to stay current with updates of packages _within_ each release. The issue of doing live upgrades from one version of Fedora to another is something that it would be nice to work out http://fedoraproject.org/wiki/YumUpgradeFaq, but it's doable as long as you're willing to troubleshoot.

(Log in to post comments)

Do we need this crap on LWN?

Posted Feb 21, 2007 17:19 UTC (Wed) by evgeny (subscriber, #774) [Link]

> Well, there seem to be a lot of changes already happening around RPM after a period of relative stasis: http://wiki.rpm.org/Docs/RpmOrgFAQ

This FAQ doesn't answer the principal question: what key features are going to be in RPM that don't currently exist in deb/apt-get?? or cannot be added with a lesser efforts to deb??

Do we need this crap on LWN?

Posted Feb 21, 2007 18:11 UTC (Wed) by TwoTimeGrime (guest, #11688) [Link]

I'm a long-time Debian user but one feature in RPM that's not in the deb system is the ability to check the integrity of an installed package. With RPM I remember I could run with an option and it will see if all the packages files still were installed and would check the MD5SUM of each file against the package database. It was an easy way to check if files from a package were missing or corrupted. It helped me when once on a RH 7.x machine I accidentally ran "rm -rf /usr". I was able to find out what files were missing and then reinstall all of the relevant packages.

Despite research and asking other debian sysadmins I could not find any options in dpkg to handle this.

integrity checking

Posted Feb 21, 2007 18:28 UTC (Wed) by rfunk (subscriber, #4054) [Link]

apt-get install debsums

integrity checking

Posted Feb 21, 2007 19:38 UTC (Wed) by sandy_pond (guest, #9734) [Link]

I don't think this has much to do with RPM vs debs as much as this link:

http://in.sys-con.com/read/278818.htm

"One of the co-founders of the open source movement has become the member of the Freespire Leadership Board. Freespire is a community-driven, Linux-based operating system that gives users the option of combining free open source software with certain proprietary codecs, drivers and applications. Raymond, well-known as both a theorist and an advocate for the open source movement, joins twelve other Freespire Leadership Board members, composed of thinkers, business people, evangelists, and key members of the Linux community."

integrity checking

Posted Feb 21, 2007 20:52 UTC (Wed) by muwlgr (guest, #35359) [Link]

I use debsums and I would like to note that .md5sums file is optional for .deb packages, unlike .rpms where I think md5sums are mandatory and could always be verified by rpm -V.

integrity checking

Posted Feb 22, 2007 0:25 UTC (Thu) by TwoTimeGrime (guest, #11688) [Link]

> apt-get install debsums

Thanks, but that highlights another think I dislike about debian. There are too many commands for package management. Now I have to remember dpkg, dpkg-query, and debsums along with all of their different options to manage my packages. At least with RPM systems there's a single command and options to remember which is just "rpm". Also, why isn't debsums' functionality built into the package management system by default? And why isn't debsums installed by default.

integrity checking

Posted Feb 22, 2007 3:24 UTC (Thu) by drag (subscriber, #31333) [Link]

Well you don't have to remember anything.

Use wajig tool or the jig interface.

It's a python package management system that does it right. It basicly takes the functionality provided by all these little tools and utilities and makes a nice little user-friendly interface for it.

For example to backport a package from testing to stable you'd go like this:
edit /etc/apt/sources.list and add the deb-src line for the repository.

wajig update
wajig upgrade (to make sure everything is up to date_
wajig build-depend foo
wajig build foo
wajig install ./foo-blah-1.0.0.deb

For instance I did that with OpenAFS server since the newer version in Testing was much better then what was aviable by default in stable. Took a whole 10 minutes to download, compile, then install the package.

Also I could of easily then taken the package and installed it on a bunch of machines.

The python bash-completion stuff works for it in testing. So if you go:
wajig <tab><tab>
it will list all the commands it supports.

Or you can go:
wajig list-commands

Also it has intellegent handling of agruements with trying to do best effort match to your command to a certain extent.
wajig listcommands
wajig List-Commands

will do the same thing.

Also you can do search through package names using tab.

And not only is the thing written in python and is convient to use.. it is QUICK. Very quick. Because the UI is in python were speed is not critical, but on the heavy calculating stuff it's all done by the little C programs that it's a front-end for.

That's the nice thing about Debian.

It's tools are small and contained. Just fast little utilities. Why have everything in some huge monolythic application? It just makes it buggy and difficult to use.

BTW this is the functionality you can get with wajig + debian package tools.

A lot of people prefer not to use front ends and use the tools directly, but I am lazy like that.

addcdrom -- Add a CD-ROM to the list of available sources of packages
auto-alts -- Mark the alternative to be auto set (using set priorities)
auto-clean -- Remove superseded deb files from the download cache
auto-download -- Do an update followed by a download of all updated packages
auto-install -- Perform an install without asking questions (non-interactive)
available -- List versions of packages available for installation
bug -- Check reported bugs in package using the Debian Bug Tracker
build -- Retrieve/unpack sources and build .deb for the named packages
build-depend -- Retrieve packages required to build listed packages
changelog -- Retrieve latest changelog for the package
clean -- Remove all deb files from the download cache
commands -- List all the JIG commands and one line descriptions for each
daily-upgrade -- Perform an update then a dist-upgrade
dependents -- List of packages which depend/recommend/suggest the package
describe -- One line description of packages (-v and -vv for more detail)
describe-new -- One line description of new packages
detail -- Provide a detailed description of package (describe -vv)
detail-new -- Provide a detailed description of new packages (describe -vv)
dist-upgrade -- Upgrade to new distribution (installed and new rqd packages)
docs -- Equivalent to help with -verbose=2
download -- Download package files ready for an install
file-download -- Download packages listed in file ready for an install
file-install -- Install packages listed in a file
file-remove -- Remove packages listed in a file
find-file -- Search for a file within installed packages
find-pkg -- Search for an unofficial Debian package at apt-get.org
fix-configure -- Perform dpkg --configure -a (to fix interrupted configure)
fix-install -- Perform apt-get -f install (to fix broken dependencies)
fix-missing -- Perform apt-get --fix-missing upgrade
force -- Install packages and ignore file overwrites and depends
help -- Print documentation (detail depends on --verbose)
hold -- Place listed packages on hold so they are not upgraded
init -- Initialise or reset the JIG archive files
install -- Install (or upgrade) one or more packages or .deb files
installr -- Install package and associated recommended packages
installrs -- Install package and recommended and suggested packages
installs -- Install package and associated suggested packages
install/dist -- Install packages from specified distribution
integrity -- Check the integrity of installed packages (through checksums)
large -- List size of all large (>10MB) installed packages
last-update -- Identify when an update was last performed
list -- List the status and description of installed packages
list-all -- List a one line description of given or all packages
list-alts -- List the objects that can have alternatives configured
list-cache -- List the contents of the download cache
list-commands -- List all the JIG commands and one line descriptions for each
list-daemons -- List the daemons that JIG can start/stop/restart
list-files -- List the files that are supplied by the named package
list-hold -- List those packages on hold
list-installed -- List packages (with optional argument substring) installed
list-log -- List the contents of the install/remove log file (filtered)
list-names -- List all known packages or those containing supplied string
list-orphans -- List libraries not required by any installed package
list-scripts -- List the control scripts of the package of deb file
list-section -- List packages that belong to a specific section
list-section -- List the sections that are available
list-status -- Same as list but only prints first two columns, not truncated
list-wide -- Same as list but avoids truncating package names
local-dist-upgrade -- Dist-upgrade using packages already downloaded
local-upgrade -- Upgrade using packages already downloaded, but not any others
madison -- Runs the madison command of apt-cache.
move -- Move packages in the download cache to a local Debian mirror
new -- List packages that became available since last update
news -- Obtain the latest news about the package
new-upgrades -- List packages newly available for upgrading
non-free -- List installed packages that do not meet the DFSG
orphans -- List libraries not required by any installed package
package -- Generate a .deb file for an installed package
policy -- From preferences file show priorities/policy (available)
purge -- Remove one or more packages and configuration files
purge-depend -- Purge package and those it depend on and not required by others
purge-orphans -- Purge orphaned libraries (not required by installed packages)
readme -- Display the package's README file from /usr/share/doc
recursive -- Download package and any packages it depends on
recommended -- Install package and associated recommended packages
reconfigure -- Reconfigure the named installed packages or run gkdebconf
reinstall -- Reinstall each of the named packages
reload -- Reload daemon configs, e.g., gdm, apache (see list-daemons)
remove -- Remove one or more packages (see also purge)
remove-depend -- Remove package and its dependees not required by others
remove-orphans -- Remove orphaned libraries (not required by installed packages)
repackage -- Generate a .deb file for an installed package
reset -- Initialise or reset the JIG archive files
restart -- Stop then start a daemon, e.g., gdm, apache (see list-daemons)
rpm2deb -- Convert a RedHat .rpm file to a Debian .deb file
rpminstall -- Install a RedHat .rpm package
rpmtodeb -- Convert a RedHat .rpm file to a Debian .deb file
search -- Search for packages containing listed words
search-apt -- Find local Debian archives suitable for sources.list
setup -- Configure the sources.list file which locates Debian archives
show -- Provide a detailed description of package [same as detail]
showdistupgrade -- Trace the steps that a dist-upgrade would perform
showinstall -- Trace the steps that an install would perform
showremove -- Trace the steps that a remove would perform
showupgrade -- Trace the steps that an upgrade would perform
size -- Print out the size (in K) of all, or listed, installed packages
sizes -- Print out the size (in K) of all, or listed, installed packages
snapshot -- Generates list of package=version for all installed packages
source -- Retrieve and unpack sources for the named packages
start -- Start a daemon, e.g., gdm, apache (see list-daemons)
status -- Show the version and available version of packages
status-match -- Show the version and available version of matching packages
status-search -- Show the version and available version of matching packages
stop -- Stop a daemon, e.g., gdm, apache (see list-daemons)
suggested -- Install package and associated suggested packages
tasksel -- Run the Gnome task selector to install groups of packages
toupgrade -- List packages with newer versions available for upgrading
unhold -- Remove listed packages from hold so they are again upgraded
unofficial -- Search for an unofficial Debian package at apt-get.org
update -- Update the list of down-loadable packages
update-alts -- Update default alternative for things like x-window-manager
update-pci-ids -- Updates the local list of PCI ids from the internet master list
update-usb-ids -- Updates the local list of USB ids from the internet master list
upgrade -- Upgrade all of the installed packages or just those listed
versions -- List version and distribution of (all) packages.
whatis -- A synonym for describe
whichpkg -- Find the package that supplies the given command or file

""Also, why isn't debsums' functionality built into the package management system by default? And why isn't debsums installed by default.""

Probably because there is little need for it to be installed by default.

With Debian Etch signed packages are going to be used by default. This will ensure that they are not tampered with and are not corrupt. This is the most important reason why you'd need checksums...

There are lots of little things that make your life easier.

Like for instance 'deborphans'. Which uninstalls unused dependancies that may have been installed by a package you ended up removing later.

Or for example: localepurge.

Localepurge can be used to remove locales and localized man pages. It can save many hundreds of megs of disk space, maybe even a gig or two on a machine with a lot of stuff installed. Once you run it then it will automaticly be ran after installing or upgrading packages.

This stuff seriously kicks-ass. Most of the time the functionality of Yum is fine.

But Yum's slowness can be a huge pain. At my work I use a old 600mhz machine. A update using yum got interrupted halfway through becuase I ran out of disk space on / (my mistake).

I was there fixing it from 1 am to 5 am in the morning and the majority of the time was spent on waiting on yum to painfully go through it's motions. It was very very painfull to watch. Even with the -C switch it was painfull.

If it was Debian, and not CentOS:
apt-get -f install

would of fixed it in about 3 minutes.

integrity checking

Posted Feb 22, 2007 5:44 UTC (Thu) by k8to (subscriber, #15413) [Link]

wajig is pretty fantastic. I wish it had become the standard instead of aptitude, and as a result was a bit more maintained.

There are some pretty simple glaring bugs with wajig that have gone unfixed for a lonnnng time. wajig show package fails if it is no longer downloadable but currently installed. apt-cache shows it fine. Huh?

Yeah I should submit a patch but sorry the other 15-odd patches I've submitted about other debian tools have come first.

apt in CentOS (was: integrity checking)

Posted Mar 1, 2007 4:24 UTC (Thu) by topher (subscriber, #2223) [Link]

Just a note, if you enable the CentOS Extras repository on a CentOS 4.x box, there is an apt package available. You can enable it, use yum to install it, and then use apt to manage your packages from then on.

Definitely worth it, IMO. ;-)

Do we need this crap on LWN?

Posted Feb 21, 2007 18:14 UTC (Wed) by pizza (subscriber, #46) [Link]

Okay, here's a key feature dpkg+apt is lacking that RPM has had for some time now:

* Sane multi-lib support.

Do we need this crap on LWN?

Posted Feb 21, 2007 23:07 UTC (Wed) by proski (subscriber, #104) [Link]

Not quite. "yum install foo" will install both foo.x86_64 and foo.i386 and all dependencies for both. I expect it to install foo.x86_64 only. Apart from that, it's mostly OK.

Do we need this crap on LWN?

Posted Feb 22, 2007 8:08 UTC (Thu) by bojan (subscriber, #14302) [Link]

So, run "yum install foo.x86_64".

Do we need this crap on LWN?

Posted Feb 21, 2007 18:37 UTC (Wed) by foo (guest, #1117) [Link]

I prefer debs, too, but one thing I miss
from my RPM days is the ability to query
which package owns a particular file.

E.g.:

# rpm -qf /etc/diameter/sunping.xml
ethereal-0.10.3-15.15

which package owns a file

Posted Feb 21, 2007 18:50 UTC (Wed) by rfunk (subscriber, #4054) [Link]

dpkg -S /etc/diameter/sunping.xml

which package owns a file

Posted Feb 22, 2007 15:00 UTC (Thu) by foo (guest, #1117) [Link]

Great, thanks!

which package owns a file

Posted Feb 27, 2007 23:00 UTC (Tue) by cdmiller (subscriber, #2813) [Link]

Yeah but whats up with this?

dpkg -S /etc/passwd
dpkg: /etc/passwd not found.

Whereas:
rpm -qf /etc/passwd
setup-2.7.3-1mdv2007.0

which package owns a file

Posted Feb 27, 2007 23:15 UTC (Tue) by rfunk (subscriber, #4054) [Link]

That's a generated file rather than one installed from a package. (The base-passwd
package generates it, but I realize your point is about how to get that information.)

The easiest way I know to get that sort of information is to grep the files
in /var/lib/dpkg/info/.

Do we need this crap on LWN?

Posted Feb 21, 2007 19:16 UTC (Wed) by fmarier (subscriber, #19894) [Link]

To search in your installed packages:
dpkg -S /usr/lib/libxml2.so.2

To search in all packages (installed or not), first install the "apt-file" package, then run:
apt-file update
apt-file search /usr/lib/libxml2.so.2

Do we need this crap on LWN?

Posted Feb 21, 2007 19:48 UTC (Wed) by vondo (guest, #256) [Link]

Since we've got deb experts reading this... The thing I miss from rpm days that I don't know how to do in deb/apt is:

rpm -qa --last

Give me a list of all the packages installed and order them by the last time they were updated.

dpkg magic

Posted Feb 21, 2007 20:55 UTC (Wed) by rfunk (subscriber, #4054) [Link]

I don't think a capability like that is built in, but try this:
ls -ltr /var/lib/dpkg/info/*.list | sed -e 's,^.*/$[^/]*$\.list,\1,'

dpkg magic

Posted Feb 21, 2007 22:06 UTC (Wed) by jonabbey (subscriber, #2736) [Link]

Ah, but what would Eric say about such an unfriendly, regressive, inward-focused approach to software design?

Okay, yeah, sorry, I've got nothing.

Do we need this crap on LWN?

Posted Feb 22, 2007 5:46 UTC (Thu) by k8to (subscriber, #15413) [Link]

Until _quite_ recently this was completely impossible. Really. No records were kept at all.

Now that dpkg keeps a log, someone could at least write such a thing, although I don't think one exists yet.

dpkg log

Posted Feb 22, 2007 14:41 UTC (Thu) by rfunk (subscriber, #4054) [Link]

the ls/sed hack I posted above works on Debian sarge, which (although current stable)
isn't exactly "quite recent". No need for the dpkg log. And as far as I can see, recent
dpkg only keeps a log if you explicitly tell it to anyway. (Though I could be missing
something more recent.)

dpkg log

Posted Feb 22, 2007 18:21 UTC (Thu) by k8to (subscriber, #15413) [Link]

Buh, I didn't look at the criteria closely. I've often needed a log and found the 'most recent update' information was too lossy.

Do we need this crap on LWN?

Posted Feb 22, 2007 16:49 UTC (Thu) by akumria (subscriber, #7773) [Link]

Something like:

eve:[~]% sudo tail /var/log/dpkg.log
2007-02-22 06:33:35 status half-configured ttf-farsiweb 0.4.dfsg-6
2007-02-22 06:33:36 status unpacked git 4.3.20-10
2007-02-22 06:33:36 status half-configured git 4.3.20-10
2007-02-22 06:33:37 status installed git 4.3.20-10
2007-02-22 06:51:26 status half-configured dhcp3-server 3.0.4-13
2007-02-22 06:51:29 status half-configured ttf-farsiweb 0.4.dfsg-6
2007-02-22 06:52:35 status half-configured dhcp3-server 3.0.4-13
2007-02-22 06:52:37 status installed dhcp3-server 3.0.4-13
2007-02-22 06:52:37 status half-configured ttf-farsiweb 0.4.dfsg-6

perhaps?

Do we need this crap on LWN?

Posted Feb 21, 2007 22:50 UTC (Wed) by mrons (subscriber, #1751) [Link]

How about:

yum localinstall somerandom.rpm

Do we need this crap on LWN?

Posted Feb 22, 2007 0:55 UTC (Thu) by mrons (subscriber, #1751) [Link]

> yum localinstall somerandom.rpm

Just to expand on what this does, yum looks at what dependencies somerandom.rpm has and installs those with somerandom.rpm

somerandom.rpm does not have to belong to some {un}official repository.

So it's like "rpm -i" except that repositories are consulting to resolve dependencies.

Can I do something like "apt-get somerandom.deb" and have dependencies resolved for me?

Do we need this crap on LWN?

Posted Feb 22, 2007 1:51 UTC (Thu) by spotter (subscriber, #12199) [Link]

not directly.

if you

dpkg -i some.deb

that will try to install it, but it wont configure due to missing dependencies.

and

apt-get install -f

will then fix it as it will resolve the missing dependencies and configure everything.

Do we need this crap on LWN?

Posted Feb 22, 2007 5:47 UTC (Thu) by k8to (subscriber, #15413) [Link]

I use wajig as my do-everything debian package tool. It supports 'wajig install <packagename>' as well as 'wajig install ./a_package-3-3_i386.deb'

Do we need this crap on LWN?

Posted Feb 22, 2007 0:01 UTC (Thu) by mmarq (guest, #2332) [Link]

"" This FAQ doesn't answer the principal question: what key features are going to be in RPM that don't currently exist in deb/apt-get?? or cannot be added with a lesser efforts to deb?? ""

hmm ... better would be what misses in both, and or what can be improved in both.

well i remenber GoboLinux... its so damn easy that is hard to belive
http://www.linux.com/article.pl?sid=07/02/09/1817227

But of course, many "commercial" apps already big on other environments, and that Linux would be needing to get a strong foot inside the workstation/desktop arena, arent going be be distributed in source code :(

But then again solutions appear like if by magic...
http://installjammer.com/index.php?option=com_wrapper&...

So, imho, what Linux need is a crossplatform installer capable of dealing with RPM and deb, and also capable of installing from source code... if only InstallJammer could implement that source code feature!... it would be perfect.

Do we need this crap on LWN?

Posted Feb 21, 2007 22:59 UTC (Wed) by markc (guest, #4419) [Link]

But if the folks pushing RPM are considering improvements, after it's
development was almost abandoned for a few years, then it would be a major
plus for all of linuxdom if they wholesale dropped rpm altogether, adopted
the deb system, and then added what they feel might be missing to debs.
Then in one 1/2 year painful changeover period for rpm users (maybe, maybe
not) we'd have more than 75% of the linux distro universe using the same
package format and, also, Debians current huge and well organised
repositories would be INSTANTLY available to RH/Fedora users. That would
do more, in 6 months, to unify "linux" everywhere than all the talk and
LSBing we've had this century. Right now is such a golden opportunity for
RH/Fedora to make one of the most sensible changes, tht they have not even
contemplated, with the largest positive outcome for all concerned.

RH/Fedora and spinoffs would gain a packaging system that already has the
essential improvements they seek AND a huge swag of ready made packages
(minus the ongoing transitional RPM package format nightmare, possibly for
years). The dpkg/apt folks would gain a new swag of interested developers
and a new perspective on how to make debs even better than they are now.
In twelve months, at least three quarters of the linux universe would be
hugely better off. We would all have a major major league win and provide
an instant doubly powerful front to coerce both hardware and software
vendors to support linux (one pkg format to bind them all) and hedge
against any future M$ patent threats (they would no longer be able to pick
on just RH... well, it could help somewhat). Oh well, I can dream.

Do we need this crap on LWN?

Posted Feb 22, 2007 0:40 UTC (Thu) by ofeeley (subscriber, #36105) [Link]

"RH/Fedora and spinoffs would gain a packaging system that already has the
essential improvements they seek"

What are those improvements that are desperately sought by us? Without the slightest trace of smartassedness I don't know what I'm missing. I have used yum for quite a while now and am very happy with it (I haven't bothered to look at pirut or pup although I did use pup's predecessor up2date for years). I've had a few problems but not many and nothing that I couldn't fix relatively easily.

"AND a huge swag of ready made packages"

The fedora repositories already have a huge swag of ready-made packages, just from within the officially blessed Core and Extras (which have now merged). If you add in unofficial repositories then it looks like there /may/ be a slight advantage of around 10% in terms of numbers of deb format packages as previously discussed: http://lwn.net/Articles/222681/

"(minus the ongoing transitional RPM package format nightmare, possibly for
years)."

What is this transitional nightmare? In all seriousness I've never heard of it.

Do we need this crap on LWN?

Posted Feb 22, 2007 3:30 UTC (Thu) by markc (guest, #4419) [Link]

> What are those improvements that are
> desperately sought by us?

FWIW, I can only speak from personal experience. I inherited control of a
CentOS 3.6 vhost and was told on some Plesk forums it was not possible to
upgrade to CentOS 4.3 (me gobsmacked). I recently got a VPS with Debian
3.1 installed, within 2 weeks I upgraded it to Ubuntu Dapper then on to
Debian Etch (v4) then back across to Ubuntu Feisty (because Debian doesn't
have a nanoweb package) with zero manual intervention. I was impressed and
would not dare to attempt an analogous path between Redhat and Fedora /
whatever. Maybe experienced RPM based folks are not "desperate" but I was
to upgrade a CentOS 3.6 system so it could run a later Plesk control panel
and, in my worldview, it was not going to be possible.

> The fedora repositories already have a
> huge swag of ready-made packages...

Nyah nyah, my deb fanboy thing is bigger than your rpm fanboy thing. The
fundamental point was that if the RH/Fedora folks started using the deb
system then "we all" would have a much larger combined system than
separately. IMVHO if one or the other system was to be chosen as the base
to extend from then I would imagine most people would pick deb and drop
rpm. It would be interesting to see an open vote on such a decision.

> What is this transitional nightmare?

The current stability of the rpm system is because it has not been
upgraded or hacked on for a few years. Once the developers start coming
out with improved betas I can imagine there will once again be "upgrade
problems". Making a clean break to the deb system *may* be less painful
than ongoing rpm packager updates.

The underlying point is that there is an ever so minute opportunity to
have one single MAJOR packaging format for linux systems and *if* it was
vaguely possible for one of the 2 major players to drop their current
format and go with the other then that opportunity is/was *now*. The
combined resources of both systems would present, essentially, a single
unified linux package system for both commercial and open providers and
such a move would become the single most notable advancement for linux
this decade. One of the reasons this won't happen is because of rpm
fanboyism, as you have clearly demonstrated, so "we" shall remain a
fragmented group of fanboys.

Do we need this crap on LWN?

Posted Feb 22, 2007 5:42 UTC (Thu) by bojan (subscriber, #14302) [Link]

> I inherited control of a CentOS 3.6 vhost and was told on some Plesk forums it was not possible to upgrade to CentOS 4.3 (me gobsmacked).

There is a reason for not being able to upgrade a running system using "yum upgrade" - it's udev. It removes the old dev package during upgrade and your system dies.

Otherwise, it's a perfectly doable thing. You could even script it using kickstart. Done many upgrades of RHEL3 to RHEL4, completely unattended.

> One of the reasons this won't happen is because of rpm fanboyism, as you have clearly demonstrated, so "we" shall remain a fragmented group of fanboys.

See: http://kitenet.net/~joey/pkg-comp/

For instance, Debian packages cannot track file dependencies, something that distributions like Fedora rely on heavily. Actually, Fedora packaging policy is to prefer dependencies based on what the build process determines, rather than manual ones, which can be easily forgotten. Good or bad, that's the Fedora way...

Do we need this crap on LWN?

Posted Feb 22, 2007 6:42 UTC (Thu) by markc (guest, #4419) [Link]

> There is a reason for not being able to upgrade
> a running system using "yum upgrade" - it's udev.

So in the case of a CentOS 3.6 -> 4.3 upgrade it would not be advisable
for an inexperienced rpm user to upgrade a remote vhost.

> See: http://kitenet.net/~joey/pkg-comp/[1]
> For instance, Debian packages cannot track
> file dependencies, something that distributions
> like Fedora rely on heavily.

11. Some people consider file dependancies a gross misfeature.

Do we need this crap on LWN?

Posted Feb 22, 2007 7:52 UTC (Thu) by bojan (subscriber, #14302) [Link]

> So in the case of a CentOS 3.6 -> 4.3 upgrade it would not be advisable for an inexperienced rpm user to upgrade a remote vhost.

Oh, it's absolutely advisable. Just use the upgrade process (anaconda runs that). You just can't do it while the system is hot.

> 11. Some people consider file dependancies a gross misfeature.

And, quite obiously, some don't.

Do we need this crap on LWN?

Posted Feb 22, 2007 11:53 UTC (Thu) by markc (guest, #4419) [Link]

> Oh, it's absolutely advisable. Just use the
> upgrade process (anaconda runs that).

What is anaconda and would it be included in a CentOS 3.6 system ?

> You just can't do it while the system is hot.

There we go, the aspect of RPM management that is "desperately needed"[1]
and equally denied by those who know not. The fairly radical cross distro
upgrade procedure I describe earlier did not require a reboot and no
service was down for more that 30 seconds max. That, to me, is an
essential part of running SERVERS where uptimes rule and why I have always
used Debian based systems. I have one box that has been up for 464 days
and started with a Debian Sarge 2.6.12 devfs enabled kernel and now runs a
Ubuntu Dapper udev enabled system, on the same kernel, without a reboot...
the very excuse I was given for why upgrading from CentOS 3.6 to 4.3 was
not possible. It seems that unless RPM based folks actually use a Debian
based system they no not what they are missing out on.

[1] Refer to previous comments above.

Do we need this crap on LWN?

Posted Feb 22, 2007 15:51 UTC (Thu) by pizza (subscriber, #46) [Link]

> What is anaconda and would it be included in a CentOS 3.6 system ?

Stick the CD in, it boots up. That's anaconda.

> here we go, the aspect of RPM management that is "desperately needed"[1]
and equally denied by those who know not.

You're confusing "The RPM tools" with "The Debian Packaging Guidelines as implemented using the dpkg tools". Get your comparison straight.

> I have one box that has been up for 464 days and started with a Debian Sarge 2.6.12 devfs enabled kernel and now runs a Ubuntu Dapper udev enabled system, on the same kernel, without a reboot... the very excuse I was given for why upgrading from CentOS 3.6 to 4.3 was not possible. It seems that unless RPM based folks actually use a Debian based system they no not what they are missing out on.

"RPM-based folks" are apparently missing out on a lot of remote exploits and possible filesystem corruption!

I reboot my machines every 90 days (if not sooner due to power outages or whatnot) solely to install new kernels and run disk integrity checks.

I don't give two hoots about SERVER uptime; to me SERVICE uptime is all-important, and I achieve that with redundancy and performing test upgrades on non-critical systems in advance.

Do we need this crap on LWN?

Posted Feb 22, 2007 16:41 UTC (Thu) by markc (guest, #4419) [Link]

> Stick the CD in, it boots up. That's anaconda.

Great, but not on a dedicated host 10,000 miles away.

> You're confusing "The RPM tools" with "The Debian
> Packaging Guidelines as implemented using the dpkg
> tools". Get your comparison straight.

I think I'm comparing the use of any number of (unfamiliar to me) rpm
tools with apt-get where the later seems to have the edge... which is the
point that ESR seems to have finally stumbled upon in the head article.

> "RPM-based folks" are apparently missing out on a
> lot of remote exploits and possible filesystem corruption!

Some would argue it's not appropriate to have such file system checking
built in to a package management system.

http://packages.debian.org/cgi-bin/search_packages.pl?key...

> I don't give two hoots about SERVER uptime; to me
> SERVICE uptime is all-important, and I achieve
> that with redundancy and performing test upgrades
> on non-critical systems in advance.

Dare I suggest that if you used a Debian based distro you could go live
and stay live with less redundancy and pre-testing. My example of swapping
between 4 deb based distros on an el-cheapo commodity $15/month VPS
without losing any downtime was intentional. No redundancy or testing
needed, just good software, especially on the barest metal.

Do we need this crap on LWN?

Posted Feb 22, 2007 17:18 UTC (Thu) by pizza (subscriber, #46) [Link]

>> "RPM-based folks" are apparently missing out on a
>> lot of remote exploits and possible filesystem corruption!

>Some would argue it's not appropriate to have such file system checking
>built in to a package management system.

No; I was referring to the fact that you can't fsck a live filesystem, and that it's an unfortunate fact of life that filesystems are at best only as reliable as the disks they run on -- errors silently happen.

For me, periodic reboots serve two purposes -- Perform filesystem consistency checks, and install updated kernels (I am quite glad the 2.6.x.y series exists!)

It's more commonly known as scheduled preventative maintainence, and through the use of multiple physical systems, I avoid service downtime. (Even with virtualization, the host system needs occasional TLC too!)

> Dare I suggest that if you used a Debian based distro you could go live and stay live with less redundancy and pre-testing.

Been there, done that, been burnt by it. As I've alluded to many times in my posts here, it's the policies that you follow that ultimately matter. I discovered the hard way that using Debian didn't make things any simpler for me -- It just had a different set of quirks to watch out for.

Granted, this was a few years back, and Debian has addressed quite a few of the problems I had at the time... but as I've also said, if I don't see any tangible benefit to making a change, why bother?

Do we need this crap on LWN?

Posted Feb 22, 2007 20:01 UTC (Thu) by bojan (subscriber, #14302) [Link]

> Great, but not on a dedicated host 10,000 miles away.

You missed the unattended upgrade bit above. The one with kickstart.

Let me repeat again. This particular limitation (of not being able to upgrade CentOS 3 to CentOS 4 while running) has nothing to do with RPM. It has to do with the fact that CentOS 4 (i.e. RHEL4) uses udev and CentOS 3 (i.e. RHEL3) uses a dev package. When dev gets removed by upgrading to udev, your system is hosed (you have nothing left in /dev).

Otherwise, I've done plenty of hot upgrades by doing "yum upgrade" on many different versions of Fedora.

> which is the point that ESR seems to have finally stumbled upon in the head article

He stumbled upon his on inability to manage his own system and/or ask others how to do it.

Do we need this crap on LWN?

Posted Feb 22, 2007 20:15 UTC (Thu) by loening (subscriber, #174) [Link]

Speaking of remote upgrades with RPM. I personally have done
RH8->RH9->FC1->FC2->FC3->FC4->FC5->FC6

On a workstation that sits 300 miles away from me that I haven't seen in years.

dev => udev

Posted Feb 25, 2007 14:54 UTC (Sun) by smurf (subscriber, #17840) [Link]

So please enlighten us poor souls who have *no*idea* why you'd need to "clean out" /dev when moving to an udev-based system. Why not leave the old package-supplying-/dev-nodes installed?

You can mount a tempfs onto /mnt/whatever, create the bare essential device nodes there, move-mount that to /dev, and start up udev. (In fact, this is how a udev-based system boots in the first place.)

Total downtime for programs attempting to open /dev/null (or whatever): Zero.

Afterwards, if you do want to remove the old /dev-supplying package, well ... RPM has a --justdb option.

dev => udev

Posted Feb 26, 2007 4:27 UTC (Mon) by bojan (subscriber, #14302) [Link]

> why you'd need to "clean out" /dev when moving to an udev-based system

Well, if you run "yum upgrade", it is a fact that that's is what's going to happen: upgrade to udev will remove dev, which will remove all entries in /dev.

As you explained, it is not necessarily what needs to happen - but it does by default. So, unless you're prepared for some creative surgery (as per you explanation), you will get screwed.

Coming back the original question of supported upgrades from CentOS 3 to 4 (or RHEL 3 to 4), it is obviuos why this is not a recommeded thing to do - people don't normally want to recommend a complicated and risky process that would in the end require a reboot anyway (new kernel). RHEL packages were designed to be upgraded using the upgrade process, not "yum upgrade" (remember: RHEL 3 and 4 don't even ship yum, it's a CentOS addition).

You can find some more info here (search for udev):

http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manu...

Note how you have to be on 2.6 kernel before starting this. CentOS 3 (i.e. RHEL 3) is based on 2.4 kernel.

Instead, one cuts a small kickstart file, starts an upgrade and after some minutes has a fully upgraded system with far less risk of stuffing things up.

Do we need this crap on LWN?

Posted Feb 22, 2007 13:18 UTC (Thu) by ofeeley (subscriber, #36105) [Link]

"I recently got a VPS with Debian
3.1 installed, within 2 weeks I upgraded it to Ubuntu Dapper then on to
Debian Etch (v4) then back across to Ubuntu Feisty (because Debian doesn't
have a nanoweb package) with zero manual intervention."

If (as is probable from you calling it a VPS) it's a SWsoft/OpenVZ/Virtuozzo based VPS then the problem that you describe will definitely occur _if_ the hosting provider hasn't made the right templates available.

The only way that major upgrades across kernel versions can work with that sort of virtual server is if the kernel (and related infrastructure like the /dev/* stuff, glibc etc) that are pulled down when you do an update are matched by changes in the hypervisor.

The lack of upgradability of one CentOS version to another is more an issue of what your VPS hosting provider has made available than any limitation of rpm.

I don't know if the same limitations apply to other virtualisation techniques (Xen, kvm etc). I have experience of exactly the problem you're describing and it irritated me, but I blame the hosting company ;)

I can only assume that either you had pinned the kernel in your migration to/from various Ubuntu versions, or the hosting provider had templates available for all those versions.

Thanks for the rest of your clear and detailed answer.

Do we need this crap on LWN?

Posted Feb 22, 2007 13:35 UTC (Thu) by ofeeley (subscriber, #36105) [Link]

"One of the reasons this won't happen is because of rpm
fanboyism, as you have clearly demonstrated, so "we" shall remain a
fragmented group of fanboys."

It's a pity you ruined an interesting post with a personal attack. I don't think that asking for particular information as to the specific advantages of a system that I'm largely unfamiliar with, and correcting your mistaken impression that there are few rpm packages is fanboyism. I agree that a single packaging format would be useful in terms of reducing apparently duplicated effort.

Do we need this crap on LWN?

Posted Feb 22, 2007 14:13 UTC (Thu) by markc (guest, #4419) [Link]

> It's a pity you ruined an interesting post with a
> personal attack.

Thanks for pointing that out, I'll try and keep a more impersonal
perspective in the future. I don't often get involved in threads like
this. To clarify the kernels involved in the personal experiences I
quoted...

>> FWIW, I can only speak from personal experience.
>> I inherited control of a CentOS 3.6 vhost and was
>> told on some Plesk forums it was not possible to
>> upgrade to CentOS 4.3 (me gobsmacked).

This was a dedicated host with a kernel that probably came default with a
CentOS 3.6 install.

>> I recently got a VPS with Debian 3.1 installed,
>> within 2 weeks I upgraded it to Ubuntu Dapper then
>> on to Debian Etch (v4) then back across to Ubuntu
>> Feisty (because Debian doesn't have a nanoweb
>> package) with zero manual intervention.

This was indeed a recent experience on a Virtuozo VPS with a 2.6.9 kernel.

>> I have one box that has been up for 464 days
>> and started with a Debian Sarge 2.6.12 devfs
>> enabled kernel and now runs a Ubuntu Dapper
>> udev enabled system, on the same kernel,
>> without a reboot.

This one, I think, is a custom rolled kernel with devfs compiled in and I
happened to include CONFIG_SYSFS=y. Ah, I see, if I rebooted on this same
kernel then I may have problems, however, the kernel and grub settings
installed by Ubuntu Dapper would bring up a newer 2.6.15 kernel.