Selective upgrading of packages

Posted Jul 16, 2012 16:46 UTC (Mon) by epa (subscriber, #39769)
Parent article: Left by Rawhide

Fedora's six-month cycle keeps it fairly current but sometimes you want the very latest version of something or other. If it's already in Rawhide you can download the source package and rebuild it - after downloading, rebuilding, and installing all the packages it depends on, recursively. This kind of 'dependency hell' for binary packages went away when tools like apt and yum became common. But rebuilding from source is still a largely manual operation in the rpm world. If tools had better support for selectively upgrading certain packages (either as binaries, or rebuilding from source) then you could largely get the best of both worlds by running ordinary Fedora releases and occasionally sucking down an update from Rawhide. Things would still break, of course, but perhaps not as much as running Rawhide for everything.

I believe apt and dpkg offer something like this where you can run Debian stable and pick certain packages from unstable?

Selective upgrading of packages

Posted Jul 16, 2012 17:01 UTC (Mon) by drag (guest, #31333) [Link] (8 responses)

Apt offers pinning support. That will allow you to mix and match packages based weights you assign them. It's a useful feature. You can even 'weight' a group of packages in such a manner that it allows you to 'roll back' packages to earlier releases and such things.

But in terms of 'stable' using apt-pinning to pull in packages from Unstable or testing is not useful. The problem you run into is that the way Debian does dependencies is that when they upgrade some lower-level package they re-compile everything then have the new packages depend on the new lower-level package. This is probably not necessary as long as the developers of the low level packages are not huge dicks about breaking ABIs, but it does avoid the need for Debian to care when libraries don't bother to stay compatible with themselves.

The effect of this is that when you pull in packages from Unstable you will be forced to upgrade huge swaths of your OS.

If you want to mix and match packages the best approach I found is to use backports.debian.org and/or backwards port the packages yourself using deb-src files. Despite what the Gentoo folks may say Debian does make handling/building/install source-based packages fairly easy. Using that approaches I have always been successful.

Although I expect that forcing dpkg to install and ignoring dependency tracking will probably work fairly well in many cases.

Selective upgrading of packages

Posted Jul 16, 2012 17:26 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

That's why Debian also has Backports repository with new packages built against the previous distribution. It's a really nice thing if you need just a few packages on top of the stable distribution.

And actually using the Unstable distribution in Debian is much less scary than running Rawhide.

Selective upgrading of packages

Posted Jul 16, 2012 18:11 UTC (Mon) by drag (guest, #31333) [Link] (2 responses)

The downside is that if you are interested in the 'latest and greatest' then Debian Unstable can actually be behind the regular Fedora releases. Not always and not with everything, but it's quite common.

Selective upgrading of packages

Posted Jul 16, 2012 21:20 UTC (Mon) by awesomeman (guest, #85116) [Link] (1 responses)

This happens around freezes of testing, because it is preferred to be able to run updates to testing through unstable, which means that core packages can't be upgraded during that period, or this would greatly complicate releasing.

Selective upgrading of packages

Posted Jul 16, 2012 22:54 UTC (Mon) by drag (guest, #31333) [Link]

Well when you have a Fedora release it will generally be out ahead of Debian unstable for a while. After a couple months it would catch up, but then eventually Fedora has another release.

I use Debian Unstable and Fedora on my desktops.To me they seem roughly equivalent in terms of 'rawness' and goals even though they take different approaches.

To find a equivelant for Rawhide you'd have to look at mixing Debian Unstable with Experimental.

Selective upgrading of packages

Posted Jul 16, 2012 20:55 UTC (Mon) by epa (subscriber, #39769) [Link]

Thanks for the info. If it is common for mass rebuilds to happen for library updates then there is a need for pinning packages at the source-package level. That would reduce the churn from updating one package to a new version.

Selective upgrading of packages

Posted Jul 16, 2012 20:58 UTC (Mon) by foom (subscriber, #14868) [Link] (1 responses)

I find selectively pulling packages from unstable quite useful, and usable, at least on a server. Generally, it will want to upgrade a few core packages like libc/libstdc++, but I don't find that to be a problem. It doesn't actually cause a need to upgrade vast swaths of the OS, only the few core packages, and I trust that those likely still work. :)

It's possible that the interdependence of desktop packages might be greater and make it infeasible to usefully do for a non-server package without upgrading almost the entire OS, I haven't really tried that.

Selective upgrading of packages

Posted Jul 16, 2012 23:04 UTC (Mon) by drag (guest, #31333) [Link]

It depends.

If the package doesn't want lots of dependencies then I'll pull down straight from Unstable. If it's something that lots of other packages depend on I usually won't do it and will source code compile.

This is usually how it goes for me when I install Debian stable and find out the software I want to run wants newer versions of something-or-other then Debian provides. In ranking from preferable to not:

1. Check backports.debian.org
2. See if something can be pulled from testing without pulling in a lot of dependencies.
3. Use apt-source and related items to compile packages.
4. upgrade to testing or unstable.

Selective upgrading of packages

Posted Jul 17, 2012 10:57 UTC (Tue) by pkern (subscriber, #32883) [Link]

The problem you run into is that the way Debian does dependencies is that when they upgrade some lower-level package they re-compile everything then have the new packages depend on the new lower-level package. This is probably not necessary as long as the developers of the low level packages are not huge dicks about breaking ABIs, but it does avoid the need for Debian to care when libraries don't bother to stay compatible with themselves.

That's pretty uncommon, especially now that we have symbol files at least for C libraries. The "recompile everything" normally only happens when the ABI is broken, which causes the binary package name to change. As we're pretty anal about ABIs it's not uncommon that we point upstream to breakage.

It is true, however, that some library say that if you compile against it, you need at least that same version of the library to use it. That's one way one can take without symbol files. This means that you do not need to manually check for ABI additions. But we don't do mass rebuilds for those new versions, it's just that packages happen to link against them when they are built and then inherit those dependencies.

Selective upgrading of packages

Posted Jul 16, 2012 17:26 UTC (Mon) by dwmw2 (subscriber, #2063) [Link] (3 responses)

You can use yum to upgrade specific binary packages from Rawhide. For example yum --releasever=rawhide update openconnect should pull in the specific package and any of its dependencies. And if its list of dependencies comprises the whole world including glibc, you get to say 'no' and rebuild from source instead.

Note that this doesn't always work. Some GNOME projects are deliberately shipping with broken dependencies so the Rawhide packages aren't marked as requiring new libraries even though they do.

Selective upgrading of packages

Posted Jul 17, 2012 13:05 UTC (Tue) by Yenya (subscriber, #52846) [Link] (2 responses)

There is another problem in rawhide: it is not even usable for "let's see whether the latest-greatest version of this package fixes my problem": after reproducing my problem with rawhide packages and reporting the bug, I have been told "it is already fixed in Koji". And indeed, there has been several days old package in Koji, which contained the fix.

So as a side note, the whole mirroring of gigabytes of rawhide is pretty useless even for testing, because rawhide, from the tester's point of view, is already outdated.

Selective upgrading of packages

Posted Jul 17, 2012 20:54 UTC (Tue) by nim-nim (subscriber, #34454) [Link]

If you're serious about reporting bugs (not just running the latest and greatest) you should point yum directly to koji repos. That way you can test packages without he rawhide lag

Though the infra price should be paid in useful bug reports

Selective upgrading of packages

Posted Jul 18, 2012 6:21 UTC (Wed) by michich (guest, #17902) [Link]

It's unusual that a several days old build was in Koji without reaching Rawhide. I thought Rawhide gets updated with the latest Koji builds every day.

By your reasoning you'd have to conclude that testing of anything is always useless, because the developer could respond with "I already fixed the bug in my local git tree" or "I already thought about this bug and have a fix planned in my head". The fix propagation delay can simply never reach zero.