User: Password:
|
|
Subscribe / Log in / New account

DNF and Yum in Fedora

Benefits for LWN subscribers

The primary benefit from subscribing to LWN is helping to keep us publishing, but, beyond that, subscribers get immediate access to all site content and access to a number of extra site features. Please sign up today!

By Jake Edge
January 15, 2014

The 0.4.10 release of the DNF package manager—slated to eventually replace Yum—was the proximate cause for a sizable fedora-devel mailing list thread, but it would seem a bit premature to get too concerned about the switch. The switch is targeted for Fedora 22, which is unlikely to be delivered before 2015. But the discussion does provide an opportunity to look in on DNF to see where things stand since we last looked at the package manager in June 2012.

Yum is, of course, the current package manager for Fedora (and RHEL). It started out as the Yellowdog Updater (YUP for Yellow Dog Linux), but was rewritten (and renamed) to "Yellowdog Updater, Modified" or Yum in the early 2000s. Yum manages packages in the RPM format, so it became the package manager of choice for several RPM-based distributions. But Yum has fairly slow performance and high memory usage; code "cruft" has increased over time as well. According to Martin Langhoff, much of that is due to backward compatibility requirements that Yum has—a rewrite could dispense with many of those.

There is another reason that Yum is being targeted for replacement: it uses its own, iterative dependency-resolution mechanism. More recent (and better performing) schemes for doing dependency resolution exist, and one, in the form of the satisfiability solving library libsolv, has been adopted by several other projects (including, of course, libsolv's origin: openSUSE's zypper package manager).

DNF does not use libsolv directly, but via the hawkey library. The move to hawkey has a number of benefits, including providing an API for languages other than Python. The existing Yum dependency-resolution code can be called from elsewhere, but only easily from Python. Programs like PackageKit that are written in C or other languages must either run Yum to do that work, or come up with their own dependency resolution. So, PackageKit, the Anaconda installer, and other programs that need to determine the dependencies for packages can use hawkey (and thus libsolv) for that.

Unlike Yum, DNF will explicitly define its API. The API for Yum has grown in an ad hoc fashion over the years, so revamping Yum will allow removing some parts of the API moving forward.

Eventually, DNF (which, contrary to many jokes, doesn't stand for anything) will be renamed to Yum, even though it is not a drop-in replacement. There are commands and features of Yum that will not be supported by DNF. Therein lies some of the unhappiness seen in the mailing list thread.

DNF complaints

Harald Reindl noted an announcement from DNF project lead Aleš Kozumplík that invited Fedora users to start trying out DNF in place of Yum. Reindl is concerned that various important Yum features are getting left by the wayside and that DNF will replace Yum without being ready. In another post, he likened it to the GRUB2 and systemd transitions, which were done before those components were ready, he said.

In particular, Reindl believes that the kernel package should be treated specially, as it is in Yum, rather than treated like any other package, as DNF does. A "yum erase kernel" command will not erase (aka remove) the package for the currently running kernel, while the DNF equivalent (replace "yum" with "dnf") will remove all kernel packages, including the currently running kernel. That will likely leave the system in an unbootable state, which is probably not what the user intended.

Kozumplík does not seem worried about the problem, though. He added an entry in a page describing differences between DNF and Yum indicating that there are no plans to treat the kernel package differently. He closed a bug filed about the problem as well. That didn't sit well with Reindl, but others were not particularly upset. Chris Adams explained that the Yum behavior was actually rather surprising to him:

I never knew that "yum erase kernel" wouldn't actually remove all packages named "kernel", because I would never say "erase the kernel" and expect something to not erase the kernel. It could be correct to erase all installed kernel packages, for example on a VM where booting is external to the VM. Or, maybe you want to re-install the kernel package to fix something such as a file getting deleted accidentally (which "yum reinstall" is documented to not handle).

But, as Rahul Sundaram pointed out, it is not just the kernel package that needs protection. The Yum protected_packages setting is ignored by DNF, but Sundaram thinks that may need to be reconsidered:

I mean, how often would one run dnf remove glibc on purpose and the significant amount of accidental runs of yum that caused serious problems resulted in yum developers adding some protection against removing key packages. dnf changing this expected behavior is [a] problem and clearly this is a design decision which I think needs to [be] revisited.

Matthew Miller noted that protected_packages was originally put into Yum as a plugin, "but then it proved itself in real use and became built-in". So suggesting that the feature could be added to DNF via plugin, as the DNF documentation does, brings it all full circle—except that there is no protected_packages-like plugin for DNF, at least yet.

Another issue is the keepcache option, which has been dropped from DNF. It would keep a copy of downloaded RPM files in the cache, rather than deleting them after installation. The DNF documentation indicates that there are "no real use cases", but several in the thread disagreed. Kevin Kofler said that he changes the Fedora Yum default (keepcache=0) on every machine he installs. A bug filed on the problem lists additional use cases. Kozumplík's response indicated that keepcache may return.

While some of the missing features are clearly important (and Sundaram has collected a list of problems he believes need to be addressed before any switch could be made), there is no huge rush. Fedora 21 is slated for August at the earliest, which would push Fedora 22 well into 2015—February or beyond. That is part of what caused the irritation displayed by some of the thread participants; it is just too early (at least in their minds) to be alarmed about the state of DNF. If it were three months into the Fedora 22 cycle, one would guess that things would be different.

Barring any serious bugs being found, DNF would seem to be in pretty good shape overall, as Sundaram's list (which includes all of the features that Reindl and others were complaining about) shows. Before it replaces Yum, there will still be opportunities to slow down that process and delay it into Fedora 23 or beyond. But a year is a long time and one suspects that the DNF team will have dealt with any problems by then. Filing bugs and trying to ensure they get addressed is quite reasonable at this point—in fact quite helpful to the team—but it is rather premature to get up in arms about the transition.


(Log in to post comments)

DNF and Yum in Fedora

Posted Jan 16, 2014 10:51 UTC (Thu) by NRArnot (subscriber, #3033) [Link]

It absolutely needs a mechanism to prevent accidental removal of the kernel or other protected_packages! One obvious route to this disaster would be to mis-type kde* as ke* ....

Why does anybody think that it's undesirable for the package manager to balk and warn you if you try to do something known to be dangerous? All that's needed is a --force flag to override the safety catch.

DNF and Yum in Fedora

Posted Jan 17, 2014 2:45 UTC (Fri) by HelloWorld (guest, #56129) [Link]

> It absolutely needs a mechanism to prevent accidental removal of the kernel or other protected_packages! One obvious route to this disaster would be to mis-type kde* as ke* ....
There is a mechanism for that. dnf asks every time you uninstall a package and you can say yes or no. I don't think more than that is needed.

DNF and Yum in Fedora

Posted Jan 17, 2014 3:15 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]

Well the problem is that there are other tools like for instance yum-complete-transaction or package-cleanup --cleandupes that can run amok and cause an excessively large list of packages after say a power recycle in the wrong moment and it is easy to miss a critical package in that list and say yes. The amount of such problems completely disappeared after yum added some protection for critical packages by default and I would not think we would want to go back on this.

DNF and Yum in Fedora

Posted Jan 17, 2014 11:51 UTC (Fri) by jezuch (subscriber, #52988) [Link]

> One obvious route to this disaster would be to mis-type kde* as ke* ....

If you're doing things like that, then you have some weird habits. It's not hard to cause a disaster this way and after two or three of them you start to rely on autocompletion to make absolutely sure that you didn't mistype anything.

Well, at least I did :)

(I also double-check any command that has wildcards in it. ANY command.)

> Why does anybody think that it's undesirable for the package manager to balk and warn you if you try to do something known to be dangerous?

"Balk and warn" is not the same as "do not allow". The Debian package manager does warn if you try to remove a package from a "core" section (don't remember the exact naming) but it won't prevent you from that if you know what you are doing (and express this clearly enough). It will also warn you if you try to remove a running kernel, but that's done in the maintainer scripts in the package itself, not in the package manager.

DNF and Yum in Fedora

Posted Jan 17, 2014 12:38 UTC (Fri) by hummassa (subscriber, #307) [Link]

Yeah, it's a case of Huffman encoding: the most uncommon (asking for removal of kernel and other protected packages) should be longer than the most common (doing that by mistake). Simple as that.

Why write a new package manager anyway?

Posted Jan 16, 2014 12:16 UTC (Thu) by niner (subscriber, #26151) [Link]

I don't understand, why a new package manager is being written at all. If Fedora wants a libsolv based package manager, why not simply replace Yum with zypper? Zypper has years of development and real world testing and supports everything a distribution needs. It has to since it's been used by openSUSE for years. Why give Fedora users a new, untested, incomplete package manager when a reliable and featurefull alternative already exists? Sounds like NIH to me.

Why write a new package manager anyway?

Posted Jan 16, 2014 12:54 UTC (Thu) by johannbg (subscriber, #65743) [Link]

Right we need to stop the fragmentation and bring the packaging format and package management systems into a whole.

It's along time since this stopped being funny...

APT - aptitude - dselect - YUM - APT-RPM - poldek - up2date - urpmi - ZYpp -slapt-get - slackpkg - zendo - netpkg - swaret - appbrowser - Conary - Equo - pkgutils - pacman - PETget - PISI - Portage - Smart Package Manager - Steam - Tazpkg - Upkg - dpkg - synaptic - rpm - zif, dnf - PackageKit -Gnome Software - Ubuntu Software Center

And probably bunch of other stuff I forgot.

Why write a new package manager anyway?

Posted Jan 17, 2014 16:16 UTC (Fri) by etienne (guest, #25256) [Link]

Maybe writing down the list of thing that a good package manager should do, making a library, and one (or more) graphical user interface for user convenience is the way to go:
most package manager lack something...

Even the list of thing to do is complex to describe:
- capture the full list of dependencies automatically, at compile time (by looking which include files are used, and which library are linked-in, and what versions) - depending on the configuration selected for the package itself.
- install all dependent packages when the user install an application
- remember which packages have been installed automatically and remove them automatically when they are no more needed
- when a package is uninstalled, magically decide if the dependencies which have been upgraded by the install should be downgraded
- detect which package has never been used (newest access time of any files in the package) when a user ask for it
- detect/disallow two package using the same (configuration) file, even if this file is the obvious place to change that parameter
- manage any "plugin" which have to be upgraded after a main application has been upgraded
- manage interpreted languages: upgrade python/java/... libraries without creating a package version containing the whole thing
- display the dependency graphically so that a user can understand where a problem comes from (dselect is not that appealing)
- display some information about a package without having to install, and classify packages in a way a user can find what he needs

I have probably missed some requirements.

Why write a new package manager anyway?

Posted Jan 17, 2014 21:36 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

> when a package is uninstalled, magically decide if the dependencies which have been upgraded by the install should be downgraded

Do you have an example of this? It doesn't seem obvious to me.

Why write a new package manager anyway?

Posted Jan 20, 2014 10:29 UTC (Mon) by etienne (guest, #25256) [Link]

>> when a package is uninstalled, magically decide
> Do you have an example of this? It doesn't seem obvious to me.

If the user uninstall a package because he does not need it, then the dependent libraries shall be kept (it is better to use up-to-date software), but if a user uninstall a package because there is a regression - then the package manager shall guess if the regression is in the package itself or in one of the dependant libraries pulled by the main package, that is the automagic part.

Why write a new package manager anyway?

Posted Jan 20, 2014 12:42 UTC (Mon) by mpr22 (subscriber, #60784) [Link]

Making that work halfway reliably sounds halting-complete.

Why write a new package manager anyway?

Posted Jan 20, 2014 18:50 UTC (Mon) by vonbrand (guest, #4458) [Link]

Worse. In the halting problem you know what behaviour to look for; here you haven't got the faintest clue (Is it slower? Now crashes with <random file>? The screen shows strange colors?).

Why write a new package manager anyway?

Posted Jan 20, 2014 23:50 UTC (Mon) by mathstuf (subscriber, #69389) [Link]

Maybe different subcommands "remove" and "revert" could help here? This really actually sounds like "yum history undo" to me… Good luck making a good UI for it and explaining to the user what is happening and the difference :D .

Why write a new package manager anyway?

Posted Jan 21, 2014 17:45 UTC (Tue) by nix (subscriber, #2304) [Link]

Quite. This is a telepathy-complete problem. (In some cases I suspect it may be precognition-complete.)

Why write a new package manager anyway?

Posted Jan 23, 2014 22:59 UTC (Thu) by etrusco (subscriber, #4227) [Link]

Smart AKA SmartPM was fantastic at the time it was released. too bad it wasn't picked up by any distro offically, not even Conectiva, its funding/founding distro. Of course it couldn't compete with Ubuntu's Software Center on the looks, but it was lean, really fast and... smart ;)
I used to test it from time in Ubuntu and Debian, but it's been a few years since I last used it. Time to play :)

Why write a new package manager anyway?

Posted Jan 24, 2014 8:43 UTC (Fri) by BlueLightning (subscriber, #38978) [Link]

FWIW we are using Smart within the Yocto Project; we're also making improvements to it, although the upstream project isn't very active at the moment - something we'd like to help fix, if possible.

Why write a new package manager anyway?

Posted Jan 16, 2014 17:19 UTC (Thu) by smoogen (subscriber, #97) [Link]

zypper was looked at and tried but it relies on various things that are specific to SuSE and not used in Fedora (and vice versa stuff that Fedora uses that SuSE doesn't). Patching it out would fork it effectively and changing Fedora to match SuSE would be a large flag day.

Zypper solves the problems that OpenSuSE developers and users are looking to be solved. Hopefully yum/dnf look at the problems that Fedora developers and user are looking to be solved. Sometimes they overlap (hawkeye) and sometimes they don't.

Why write a new package manager anyway?

Posted Jan 16, 2014 17:29 UTC (Thu) by jhoblitt (subscriber, #77733) [Link]

It sounds like your claiming that package management is fundamentally different between Fedora & SuSe...

Could you explain how package management on those distros have mutually incompatible requirements?

Why write a new package manager anyway?

Posted Jan 16, 2014 18:42 UTC (Thu) by dowdle (subscriber, #659) [Link]

Another thing to consider... to the best of my knowledge, dnf is a fork of yum... so it is yum as a separate project... in which they are doing a few things... namely switching resolver libraries... and spending more time on cleaning up the code and getting rid of as much cruft as possible. Some of that cruft are features they don't think are really used/needed. As everyone knows... as code ages it does gain a bit of cruft. :)

So dnf to me is really just a streamlining of yum and trying to make it better rather than being a completely different package manager.

Why write a new package manager anyway?

Posted Jan 16, 2014 19:15 UTC (Thu) by johannbg (subscriber, #65743) [Link]

What matters here is unification and if that means all distribution need to sacrifice somewhere in the stack and if that leads to us having to kick out yum/dnf in Fedora then so be it.

Red Hat just has to swallow that pill.

Why write a new package manager anyway?

Posted Jan 16, 2014 19:50 UTC (Thu) by rahulsundaram (subscriber, #21946) [Link]

Dowdle is right that DNF is just a yum fork and not a new package manager but some of that unification is happening already since Panu apparently is looking into merging dependency solving logic (from libsolv) into RPM directly. Red Hat switched from home grown up2date to community developed yum before and could switch again if it is really warranted but it is likely that they are just going to continue to improve yum (via the dnf project as of now).

However if you really want to talk about unification, I don't see Red Hat switching to DEB format or Debian switching to RPM format anytime soon but either of these formats are just tarballs with some additional metadata.

The interesting problems are the large amount of differences in other layers higher up (packaging guidelines, choice of components, home grown solutions like Debian menu etc) and if you unify them all, you will have a single Linux distribution. Not going to happen though but the differences are getting somewhat nullified via other deployment models (language specific ones are prominent now) and also emerging sandbox (https://docs.google.com/document/d/1QTgxakyUVFMkvr-xFY2Xg...) and container based models(ex: Docker).

GNOME Software's answer appears to be abstract it out and have plugins support more than just the native packaging format of the distributions. It is a changing landscape and we will see what such efforts evolve into.

Why write a new package manager anyway?

Posted Jan 16, 2014 21:23 UTC (Thu) by johannbg (subscriber, #65743) [Link]

In anycase there is nothing preventing Fedora to switch oh wait yes there is...

"if you unify them all, you will have a single Linux distribution."

For quite sometime now ( couple of years ) the plumbing layer has been slowly moving to the direction of becoming whole which is an move which I see significantly strengthen the GNU/Linux ecosystem but I disagree that we wind up with a single Linux distribution since I philosophize and say what deviates distributions now is not so much the bits but more their philosophy and the communities surrounding each philosophy and those will always exist which also is good.

I agree it's going to be interesting to see if an to what extent GNOME Software changes the landscape for distributions shipping Gnome and distributions in general and I would not be surprised if it became it's own distribution in that process as well.

Why write a new package manager anyway?

Posted Jan 18, 2014 10:09 UTC (Sat) by rahulsundaram (subscriber, #21946) [Link]

" In anycase there is nothing preventing Fedora to switch oh wait yes there is..."

Finish your sentence if you want others to follow your train of thought.

" For quite sometime now ( couple of years ) the plumbing layer has been slowly moving to the direction of becoming whole which is an move which I see significantly strengthen the GNU/Linux ecosystem"

The plumbing layer is consolidating however your idea of consolidating packaging formats is certainly not happening. There is now ever more than before.

" I agree it's going to be interesting to see if an to what extent GNOME Software changes the landscape for distributions shipping Gnome and distributions in general and I would not be surprised if it became it's own distribution in that process as well."

It would have a software center that would act as a distribution point but there is no real reason it would morph into a Linux distribution on its own. I don't see the logical reason or the commercial incentive to do that.

Why write a new package manager anyway?

Posted Jan 16, 2014 17:40 UTC (Thu) by niner (subscriber, #26151) [Link]

I tried finding some information on this topic on the net, but other than https://fedoraproject.org/wiki/Features/DNF#Why_not_zif.2... I've come up empty. And the linked page gives no real answer either. It lists the reasons as:
* dnf looks for a middle ground between a sane API and some backwards compatibility.
* dnf and hawkey are first steps toward using the same resolver accross the entire stack.
* Libsolv is a well tested and proven code base, currently the most sophisticated and optimized dependency solving implementation.

But zypper uses libsolv (indeed, libsolv was developed for zypper), so the third point would be fulfilled by default. Since zypper uses libsolv, hawkey can still be used to provide the same resolver for the entire stack fulfilling the second point.

This leaves just the "middle ground between a sane API and some backwards compatibility". As hawkey could still be used the "sane API" argument does not count as a reason leaving only "some backwards compatibility". I'm quite sure that writing a wrapper around zypper that provides backwards compatibility with yum would be an easy task.

This leaves just the openSUSE specific things in zypper you mentioned that I can't find anything about. Could you please elaborate?

Why write a new package manager anyway?

Posted Jan 16, 2014 18:16 UTC (Thu) by smoogen (subscriber, #97) [Link]

I am trying to find the version of rpm that SuSE uses and going through the various spec files to see. The obvious items to me would be tie-ins to YaST and the Recommends logic which Fedora doesn't use.

In the end, I am going off from what the various people who have tried to get zypper into Fedora in the past. They ran into issues that are just basic design choices between the OS's.

Why write a new package manager anyway?

Posted Jan 16, 2014 18:26 UTC (Thu) by johannbg (subscriber, #65743) [Link]

Well it would immediately be better if all rpm based distribution could use the exact same tools.

Why write a new package manager anyway?

Posted Jan 16, 2014 21:06 UTC (Thu) by RedDwarf (guest, #89472) [Link]

I don't know about Fedora. But as an openSUSE packager, the only differences I can think of are openSUSE "workarounds" that should be fixed anyway.

a) "Clean"
Uninstall unused packages. You know, you install pacA, which requires pacB. The you uninstall pacA and pacB stays installed.
To do this properly you need to know if a package was installed automatically, or directly per the user instruction.
libzypp uses an history file (O(n), with n being the number of RPM/zypp operations you have made since you installed the system).
This information should be stored in the RPM database itself, so if you install a package without using libzypp you still have the info.

b) Soft dependencies
"Recommends", "Suggests", etc.
Why upstream RPM still doesn't include them is a big mystery to me.

c) Poorly documented dependency types
AND dependencies, which are included in the repository metadata but not in the RPMs (WTF!) because RPM doesn't support AND dependencies.
Dependencies on installed hardware (think drivers), system language, etc... basically extra features that you can ignore.

d) Something similar to file triggers
It's not even really used. But libzypp supports something to run a script at the end of the whole transaction. This shouldn't be needed, RPM should support file triggers.

openSUSE's RPM is here: https://build.opensuse.org/package/show/openSUSE:Factory/rpm
Yes, a lot of patches, but they don't really change things so much.

Why write a new package manager anyway?

Posted Jan 16, 2014 22:19 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

> Uninstall unused packages

Yum stores this info in the yumdb. Since RPM doesn't actually fetch, it doesn't have a reason why it is doing installs and removals. For yum, there's the yum-remove-leaves package (and yum history undo $transaction). My understanding is that dnf acts with remove-leaves behavior by default.

> "Recommends", "Suggests", etc.

Ask RPM devs. Something about it being hairy to implement? Or the semantics associated with it.

> AND dependencies

While nice, this isn't included anywhere. Instead, you Require a virtual package (like web-server) which is Provide'd by httpd, nginx, etc. But yes, dependencies on things outside of the RPM world would be nice, but is also likely a Hard Problem (what if I remove my joystick; should RPM refuse to do anything until I uninstall a joystick-requiring package?).

> Something similar to file triggers

This would be very nice. No more need to look up which scriptlet my package may need.

Why write a new package manager anyway?

Posted Jan 17, 2014 2:52 UTC (Fri) by HelloWorld (guest, #56129) [Link]

What are AND dependencies?

Why write a new package manager anyway?

Posted Jan 17, 2014 3:22 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]

There are a bunch of non upstream patches carried by the openSUSE rpm package that implements features that they are only apparently using exclusively. The best description of them I know of is at

https://www.youtube.com/watch?v=FNwNF19oFqM

Unfortunately the slides are not accessible anymore but if anyone has a non video reference, feel free to post it.

DNF and Yum in Fedora

Posted Jan 17, 2014 7:23 UTC (Fri) by xorbe (guest, #3165) [Link]

DNF is an odd name for a package manager. It stands for "Did Not Finish" in the athletic world.

DNF and Yum in Fedora

Posted Jan 17, 2014 10:43 UTC (Fri) by rahulsundaram (subscriber, #21946) [Link]

DNF and Yum in Fedora

Posted Jan 18, 2014 1:19 UTC (Sat) by salimma (subscriber, #34460) [Link]

Do Not Fork :p

DNF and Yum in Fedora

Posted Jan 26, 2014 23:14 UTC (Sun) by gdt (subscriber, #6284) [Link]

An issue with removing the kernel is that Fedora won't then cleanly reinstall a kernel if you notice this and then try to install a kernel prior to rebooting. Instead it trashes the grub configuration. This is a minor issue for yum, but obviously gains in importance if using dnf.


Copyright © 2014, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds