RPM 5.0 released [LWN.net]

The press release announcing the availability of RPM 5.0 has gone out. This is the Jeff Johnson RPM fork - not the version used by most Linux distributions at this time. "The relaunch of the RPM project in spring 2007 and today's following availability of RPM 5 marks a major milestone for the previously rather Linux-centric RPM. RPM now finally evolved into a fully cross-platform and reusable software packaging tool."

to post comments

Formats supported

Posted Jan 5, 2008 18:10 UTC (Sat) by epa (subscriber, #39769) [Link] (17 responses)

The RPM packages, in addition to the default Gzip and optional Bzip2 compression, now support also LZMA compression.

This sounds like a very handy feature given the long amount of time my PC spends downloading the latest 0.0.0.0.1 version update of OpenOffice from a Fedora mirror. lrzip would be even better. (Better still some way for smart, yum etc. to download only the differences between one package and the next, but I don't know if that really needs any involvement from rpm, apart from supporting packages built without compression or with gzip -0.)

Finally, support for the old RPMv3 (LSB) package format was removed to cleanup and simplify the code base.

I know a lot of people don't have much respect for the LSB, either as an idea or an implementation, but surely this is a retrograde step? No distribution could remain LSB-compliant unless they included a separate, old version of RPM just to install LSB packages or converted them with utilities like alien. This is already the situation faced by Debian, Slackware, Ubuntu and others, so perhaps it's 'fairer' that all distributions have to treat LSB packages as different from their native packaging format. RPMv3 would then be a distribution-neutral package format.

However it's not clear that Linux distributors are going to adopt the new rpm 5 (especially not Fedora and Red Hat), so it could all be academic.

Formats supported

Posted Jan 5, 2008 18:22 UTC (Sat) by sbergman27 (guest, #10767) [Link] (10 responses)

> Better still some way for smart, yum etc. to download only the differences between one
package and the next,

You're wanting "Presto!"  It's in Fedora 8.  But the binary diff repos are not ready yet.
Maybe in time for F9?

I know that Suse was using some sort of binary diffs years ago, though I don't recall being
impressed by their speed.

Formats supported

Posted Jan 5, 2008 20:25 UTC (Sat) by jengelh (subscriber, #33263) [Link] (4 responses)

There are .patch.rpms, which contain full copies of all files that changed.
Then there are .delta.rpms, which are deltas - and yes, I have to agree, applying deltarpms is
not too fast. But that is a general thing about deltaing, not limited to rpms. As soon as
transferring the full .rpm is faster than delting it up, deltarpms are useless unless you have
a strict pay-per-byte pipe.

Formats supported

Posted Jan 6, 2008 7:36 UTC (Sun) by Richard_J_Neill (subscriber, #23093) [Link] (3 responses)

Surely you don't have to maintain the original .rpm file to use a delta-rpm.  The original
.rpm could be re-constructed on the fly from the packages that it installed.

Formats supported

Posted Jan 6, 2008 18:56 UTC (Sun) by MattPerry (guest, #46341) [Link] (2 responses)

Unlikely as RPMs contain metadata and pre- and post-installation scripts that are not kept on
the filesystem after installation.

Formats supported

Posted Jan 6, 2008 19:01 UTC (Sun) by jengelh (subscriber, #33263) [Link]

Scripts are kept. Otherwise, how would you run %preun and %postun? So yes, deltarpm magically
takes both the rpmdb and the delta file and produces a full (non-delta, perhaps .patch) rpm
out of it. And that takes a whee while.

Formats supported

Posted Jan 7, 2008 15:21 UTC (Mon) by roblucid (guest, #48964) [Link]

Actually OpenSuSE claims to do exactly that.  The Online update takes a 
delta rpm, and constructs the new files from the old, plus what's in the 
rpm delta file.

It generally works to!

Formats supported

Posted Jan 7, 2008 10:20 UTC (Mon) by epa (subscriber, #39769) [Link] (4 responses)

Hmm, surely a binary diff can just be generated on demand -
diff?version_1=bash-1.rpm;version_2=bash-2.rpm - and there is no need for lots of tiny diff
files to be pushed out to mirrors.  Because the diffs will be much smaller, indeed there may
not be a need to mirror them out.

(I'm still wondering why distros don't use bittorrent for updates, the idea of mirror sites
seems so clunky nowadays.)

Impossible

Posted Jan 7, 2008 23:15 UTC (Mon) by khim (subscriber, #9252) [Link] (2 responses)

Genration of DIFF is quite slow and requires some software not usually present on ftp-mirror sites. Thus it makes sense to precreate diff files and push them to mirrors - or not use them at all...

Impossible

Posted Jan 9, 2008 14:12 UTC (Wed) by epa (subscriber, #39769) [Link] (1 responses)

My point is, don't bother with the mirror sites, just have a single site generating the diffs
on demand.

>Impossible<

Posted Jan 9, 2008 14:43 UTC (Wed) by gvy (guest, #11981) [Link]

Useless w/o caching/propagating them.

Hey, go try xdelta1/xdelta3 for yourself and then imagine doing that "on demand".

Formats supported

Posted Jan 12, 2008 1:31 UTC (Sat) by mp (subscriber, #5615) [Link]

> I'm still wondering why distros don't use bittorrent for updates,
> the idea of mirror sites seems so clunky nowadays.

Actually, they try to: http://wiki.debian.org/DebTorrent

Formats supported

Posted Jan 5, 2008 18:57 UTC (Sat) by sbergman27 (guest, #10767) [Link] (1 responses)

"""
However it's not clear that Linux distributors are going to adopt the new rpm 5
"""

The last thing we need is to become dependent, again, upon a project headed up by a
uncooperative developer.  The XFree86->Xorg move has clearly been beneficial.  Standardizing
on the Jeff Johnson fork of RPM would be unwise, and a step backwards, in my opinion.

people vs companies

Posted Jan 9, 2008 14:47 UTC (Wed) by gvy (guest, #11981) [Link]

Just in case it might seem too beneficial, XF86 [speaking of e.g. 3.3.6, 4.3.0] was *much*
more robust per se than X.org -- which is currently in top3 insane upstreams for me.  At least
it did work, and they didn't manage to break *intel* drivers after specs were long published!

Re Jeff, I'm afraid I'd also stay away from Red Hat.  While having a lot of respect for them,
I don't consider them a vendor-neutral... er... vendor.

Formats supported

Posted Jan 5, 2008 19:23 UTC (Sat) by tzafrir (subscriber, #11501) [Link]

rpm5.org now has an extra mailing list: rpm-lsb . Right now the archives show just one
message:

http://rpm5.org/community/rpm-lsb/0000.html

Jeff Johnson writes there that he wants to document the format of rpmv4 (which is the one
actually used today) well enough so it could be used in the LSB specifications. He hopes "to
start in mid-January, and have a completed document by the end-of-March or so."

And he's not completely alone. While Mandriva has decided to use the original rpm recently,
there are still a number of distros that use rpm from rpm5.org .
See the following thread:

http://rpm5.org/community/rpm-devel/index.html#1824

Disclaimer: I don't follow closely rpm.org or rpm5.org .

Formats supported

Posted Jan 6, 2008 5:17 UTC (Sun) by zooko (guest, #2589) [Link] (1 responses)

By the way, I recently learned that 7z compresses noticeably tighter than bzip2, and even than
lrzip (for the data sets that I was interested in).  7z also has the advantage over lrzip (and
over rzip) that it can operate in "streaming" mode, and it has the advantage over all of them
(except probably gzip) of being faster/taking less CPU time.

http://allmydata.org/pipermail/tahoe-dev/2007-December/00...

Formats supported

Posted Jan 10, 2008 13:52 UTC (Thu) by kirkengaard (guest, #15022) [Link]

This discussion has been had many other places, and so far, LZMA (aka 7z) isn't unixy enough
to behave like we all expect gzip and bzip2 to behave.  Outputs and subarchiving, not to
mention magic numbers and crc, seem to be troublesome still.  It does what it does very well,
sure, but it doesn't do what gzip and bzip2 do well.

The other aspect is that of "sufficient improvement to outweigh transition costs".  Once it
behaves itself such that it can be a drop-in replacement for gzip or bzip2, the algorithm
speed and CPU efficiency will be no-brainer reasons to switch.  No workflow changes, IOW.

As your referenced email states, it's all about the tradeoffs and the conditions of use.  Look
up HPA on kernel binary compression formats.

Formats supported

Posted Jan 8, 2008 19:56 UTC (Tue) by n3npq (guest, #40075) [Link]

Whether removing support for RPMv3 header+payload signatures from rpm-5.0
is a retrograde step or not depends on whether you think that the additional signature
on header+payload adds security or not.

However, I do point you at the following links for additional information
regarding LSB "standard" packaging:
    http://rpm5.org/community/rpm-lsb/0001.html
    https://lists.linux-foundation.org/pipermail/packaging/20...

The original report of problems with LSB packaging failing to pass LSB's own test suite are at
    http://www.linux.org.ru/view-message.jsp?
msgid=2391597&lastmod=1199814655335&page=2

(apologies for the URL line wrap)

hth

RPM 5.0 released

Posted Jan 5, 2008 20:48 UTC (Sat) by Tet (subscriber, #5433) [Link] (13 responses)

Based on my previous interaction with Jeff Johnson, I think I'll be giving this a miss. I'll stick with distributions using a package manager written by someone I trust to get it right. And I'm afraid that's just not Jeff...

RPM 5.0 released

Posted Jan 6, 2008 0:42 UTC (Sun) by tzafrir (subscriber, #11501) [Link] (12 responses)

Interesting. If this was such an annoying problem, how come nobody stepped in to offer a
patch?

In fact comments by people more familiar with RPM tended to support Jeff's opinion. e.g:
https://bugzilla.redhat.com/show_bug.cgi?id=119185#c56

Jeff later fixed it in his forked version. But more importantly, in rpm5 he planned to switch
to sqlite to get rid of berkeley-db related issues.

RPM 5.0 released

Posted Jan 6, 2008 2:44 UTC (Sun) by gdt (subscriber, #6284) [Link] (4 responses)

Bugzilla contains bugs from non-paying customers. Paying customers, such as myself, also reported this bug and Jeff's response was no more useful.

Let's review: customers were paying top dollar for support (more per-item than to Sun or Cisco, and they support the hardware too), to a company which claimed that it was excellent at support, and this was a software project which the company created and presumably controlled. The onus for a fix was on Red Hat, and they failed to deliver in any timely fashion. And insulted their customers, and failed to take ownership of the issue, claiming nothing was wrong when there so obviously was.

The RPM database inconsistency is the low point of Red Hat's support efforts. That experience made sure we developed and maintain a contingency plan for migration to another Linux vendor. Our experience was pretty much directly responsible for moving our e-mail system from Red Hat Linux to Windows. It was difficult to argue that Linux was more reliable and Linux vendors more responsive than Microsoft under circumstances where this obviously was not so.

RPM 5.0 released

Posted Jan 6, 2008 9:00 UTC (Sun) by tzafrir (subscriber, #11501) [Link]

That is about RedHat, not about JBJ.

The amazing thing is this was such a painful issue and yet nobody came up with a patch.
Including others that are more familiar with rpm.

And how do the two current rpm versions fare with respect to database inconsistency?

RPM 5.0 released

Posted Jan 6, 2008 9:43 UTC (Sun) by petegn (guest, #847) [Link]

do i detect the smell of wet nappies ? ....

RPM 5.0 released

Posted Jan 6, 2008 18:06 UTC (Sun) by einstein (subscriber, #2052) [Link]

> Our experience was pretty much directly responsible for moving our e-mail system from Red
Hat Linux to Windows. 

Ouch, microsoft? cutting off your nose to spite your face, eh? 

Note that there are other vendors. We moved from redhat to suse/novell and have been very
happy with the results.

RPM 5.0 released

Posted Jan 6, 2008 21:02 UTC (Sun) by michich (subscriber, #17902) [Link]

Wasn't Jeff fired from Red Hat exactly because of his attitude to bug 
reporters?

RPM 5.0 released

Posted Jan 6, 2008 11:36 UTC (Sun) by Los__D (guest, #15263) [Link]

"Jeff later fixed it in his forked version. But more importantly, in rpm5 he planned to switch
to sqlite to get rid of berkeley-db related issues."

I really can't see how this bug is bdb related, it was related to the idiotic "best-effort"
algorithm, combined with missing checks.

RPM 5.0 released

Posted Jan 6, 2008 12:54 UTC (Sun) by cortana (subscriber, #24596) [Link]

Was it actaully fixed at all? I thought that rpm was changed to detect the filesystem being
mounted read-only, but that would not fix other problems, such as where the filesystem was
full, or one of the files being unpacked wanted to overwrite an immutable file, etc.

RPM 5.0 released

Posted Jan 6, 2008 19:03 UTC (Sun) by MattPerry (guest, #46341) [Link] (1 responses)

This is a good read for background info: http://lwn.net/Articles/196523/

Fedora is supposed to be taking charge of RPM:
http://linux.slashdot.org/article.pl?sid=06/12/15/040258

RPM 5.0 released

Posted Jan 7, 2008 0:39 UTC (Mon) by skvidal (guest, #3094) [Link]

Red Hat/fedora has taken over rpm.org. Panu Matilainen works on it as his primary job at red
hat. There are some other contributors from suse and some fedora maintainers working on it,
too. Ralf Corsepius provided a lot of work recently.

It's hosted externally to red hat, though.

RPM 5.0 released

Posted Jan 6, 2008 22:17 UTC (Sun) by clump (subscriber, #27801) [Link] (2 responses)

Offering a patch would have been a good thing, but it also seems quite likely that a patch wouldn't have been applied. This matter was absolutely about Jeff Johnson as he never acknowledged there was an issue with an rpm database incorrectly representing what was installed.

Forgive me, the comment you refer to agrees with the bug reporter:

Concrerning the real problem at hand, it would be very nice that when rpm detects it has read only filesystems that are not in the %_netsharedpath macro, it should fail early before running any package through the package state machine, because it can (there are issues, but bottom line it is very very possible).

My employer has tens of thousands of Redhat machines. Reading the bug report made me nervous, as I am responsible for a couple thousand of those machines. Johnson's behavior lets the mind wonder about other known, unfixed bugs.

I congratulate "Tethys" for pressing the issue, and I'd hope anyone considering Johnson's RPM fork would read the full bug report.

RPM 5.0 released

Posted Jan 7, 2008 19:53 UTC (Mon) by spot (guest, #15640) [Link] (1 responses)

It is worth noting that Jeff doesn't work for Red Hat anymore, and that the version of rpm
currently maintained in RHEL and Fedora is currently maintained by other people.

While it is sometimes difficult to separate a company from its employees, try to remember that
Red Hat does care about its customers, and works hard to resolve issues as timely as possible,
even when those issues are internal.

RPM 5.0 released

Posted Jan 8, 2008 19:13 UTC (Tue) by n3npq (guest, #40075) [Link]

Please permit me to confirm that, indeed, I am no longer employed by Red Hat, which
is the case for almost 3 years now.

Disclaimer: I do still have much invested in RHT, so please buy RHEL, as much as possible,
for your 10000+ machines now that you know that RHEL has no chance of being tainted
by any efforts of mine. Presumably spot@redhat.com should supply a similar disclaimer?

And for those who might be more interested in the rpm-5.0 release than, say, my place of
employment, or bugzilla #119185 or other hysterical 4+ year old baggage from when
I was <jbj@redhat.com> doing my job, I point you at the rationale for OpenPKG's choice
to provide vendor neutral infrastructure for the RPM project (I'll leave it to reader's to
decide their own meaning for "official") at

    http://trainofthoughts.org/blog/2008/01/06/rpm5-vs-rpm/

Enjoy!