Weekly Edition
Return to the Distributions page
| |||||||||||||
New features in APT-RPM
1. Introduction
APT-RPM is a port of Debian's APT tool to RPM based distributions (Conectiva Linux, Fedora Linux, SuSE Linux, ALT-Linux, etc), written and maintained by Conectiva. APT is an advanced package management utility front-end which allows one to easily perform package installation, upgrading and removal. Dependencies are automatically handled, so if one tries to install a package that requires others to be installed, it will download all needed packages and install them. Recently, an intensive amount of development has been happening under the hood in the APT-RPM world, but unfortunately, most of the features are only perceived by a small number of people that follow the development closely. This article is an attempt to introduce the reader to some of the latest features available today in the RPM port of APT. 2. Dealing with local packages One of the recently introduced features, which was on the top of the TODO list for a long time, is the capability of dealing with random local packages using APT's ability to handle dependencies. With this feature, installing a local package with dependency handling is as easy as installing a file in any remote repository. Here is an example, assuming that the file is in the current working path: % apt-get install rpmver-2.0-13498cl.i386.rpm Reading Package Lists... Done Building Dependency Tree... Done Selecting rpmver to represent rpmver-2.0-13498cl.i386.rpm The following NEW packages will be installed: rpmver 0 upgraded, 1 newly installed, 0 removed and 8 not upgraded. Need to get 0B/6404B of archives. After unpacking 5552B of additional disk space will be used. Committing changes... Preparing... ########################################### [100%] 1:rpmver ########################################### [100%] Done. This is valid for operations dealing with source packages as well. The following example shows an operation that checks every build-time dependency of the given source package, asks for confirmation, fetches, and installs them locally. % apt-get build-dep apt-listchanges-1.49-11104cl.src.rpm Reading Package Lists... Done Building Dependency Tree... Done The following NEW packages will be installed: rpmver 0 upgraded, 1 newly installed, 0 removed and 8 not upgraded. Need to get 6404B of archives. Need to get 6404B of archives. After unpacking 5552B of additional disk space will be used. Do you want to continue? [Y/n] Get:1 ftp://mapi8.distro.conectiva latest/conectiva/all rpmver 2.0-13498cl [6404B] Fetched 6404B in 8s (798B/s) Committing changes... Preparing... ########################################### [100%] 1:rpmver ########################################### [100%] Done. 3. Local repositories Dealing with package files directly is just one way to use APT-RPM. Another easy way is to set up a local repository. Traditionally, local repositories are always available, but they require maintenance of meta-information, which APT-RPM expects to find in the base/ subdirectory. Now APT-RPM has learned to deal with a special kind of local repository, which does not require the maintenance of meta-information. Instead, information is fetched directly from the packages, and new packages dropped into these directories are automatically recognized. The configuration of this new kind of local repository is straightforward. One only needs to replace the rpm source type with the rpm-dir source type in the sources.list repository configuration file. For example, to maintain a repository in /repos/conectiva/RPMS.local and to have every package dropped in this directory automatically recognized by APT-RPM, one should include the following line in the sources.list file: rpm-dir file:///repos conectiva local No additional configuration is needed. The same rules apply to source repositories as well, using rpm-src-dir instead of rpm-src in the sources.list repository configuration file. Extending the example above, the following line would allow one to maintain SRPM packages in /repos/conectiva/SRPMS.local without any further work: rpm-src-dir file:///repos conectiva local 4. Installing packages by filenames One feature that seems logical for most package tool users is the ability to install packages by providing file names instead of package names. This feature was only available in APT-RPM through an external Lua extension, until recently. Now this is available internally in APT-RPM, which is able to translate any filename included in the meta information of the remote repository. The following example shows the feature working. The filename is translated to the package name, and with user confirmation, the package is downloaded and installed. % apt-get install /usr/bin/rpmver Reading Package Lists... Done Building Dependency Tree... Done The following extra packages will be installed: rpmver The following NEW packages will be installed: rpmver 0 upgraded, 1 newly installed, 0 removed and 8 not upgraded. Need to get 6404B of archives. After unpacking 5552B of additional disk space will be used. Do you want to continue? [Y/n] Get:1 ftp://mapi8.distro.conectiva latest/conectiva/all rpmver 2.0-13498cl [6404B] Fetched 6404B in 17s (376B/s) Committing changes... Preparing... ########################################### [100%] 1:rpmver ########################################### [100%] Done. Notice that it's possible to exclude some files from the meta-information, for space saving purposes. 5. APT Shell Another interesting feature developed recently is the apt-shell. This is a tool which offers a shell to make the user experience more pleasant while navigating through the APT cache and installing, erasing, and upgrading packages. Here is a very short list of available features to leave the reader excited enough to look further:
Here is an interactive sample section, hacked for objectiveness. Reading Package Lists... Done Building Dependency Tree... Done Welcome to the APT shell. Type "help" for more information. apt> install dum [TAB pressed] dummy dump apt> install dummy= [TAB pressed] 0.1-1cl 1.0-1cl 1:1.0-1cl 1:1.5-1cl apt> install dummy=1:1.0-1cl apt> install alsa-lib-devel Unrequested changes are needed to execute this operation. The following packages will be upgraded alsa-lib-devel libalsa2 (...) Do you want to continue? [Y/n] n Abort. apt> commit The following NEW packages will be installed: dummy (...) Executing RPM (-Uvh)... Preparing... ########################################### [100%] 1:dummy ########################################### [100%] (...) apt> ls dum* dummy dump apt> ls -v dum* Name Installed Candidate ---- --------- --------- dummy 1:1.0-1cl 1:1.5-1cl dump - 0.4b28-11400cl apt> ls -s dum* dummy - Dummy package doing weird stuff. dump - Programs for backing up and restoring filesystems. 6. Meta-components Unlike the Debian strategy of maintaining main, contrib, and a few other major components, some users of APT-RPM, like Conectiva Linux, have chosen to split components in a more finely grained fashion with, for example, devel, doc, audio, and many others. This introduces some interesting possibilities, as users may "subscribe" to just those components they are really interested in. On the other hand, splitting components like this also introduces some annoyance to those who want to simply subscribe to every available component. It also creates problems when a new component is created, since nobody is currently subscribed to it, and would take some time to discover that the new component was available. The meta-component was created to solve these issues. Using meta-components one is able to create components that contain the information of other real components. For APT-RPM users, there's no difference between a meta-component and a real component. For repository maintainers who want to create a meta-component, it's just a matter of using the appropriate genbasedir options. For example, suppose that the components main, extra, kde, and gnome are available, one may create an additional all meta-component which includes information from all of them with the following command: genbasedir --meta=all /repos/path main extra kde gnome Doing this, users may either include any combination of individual components in their sources.list repository configuration file, or use the meta-component. 7. Lua interface One exciting new capability recently introduced into APT-RPM is the embedding of the Lua interpreter. This allows customization and extension of APT-RPM in any direction, being limited mostly by the imagination and scripting ability of anyone who needs extensive packaging features. The scripting interface is done using a slot concept. Slots are places in APT-RPM execution where any number of scripts may be plugged in to introduce special behavior. Some of these slots allow scripts to introduce new commands in the APT-RPM tools apt-get, apt-cache and apt-shell. Additionally, scripts may also be called with the script subcommand. Currently, a good part of the APT API is already available to Lua scripts, and complete documentation is available at AptRpm/Scripting. To give you an idea of how it works, the following script may be executed with apt-get script install-devel.lua (assuming it has this name), to install development packages with suffixes -devel and -devel-static, as long as the main package is already installed.
function installdevel(name)
pkg = pkgfind(name.."-devel")
if pkg and not pkgvercur(pkg) then
markinstall(pkg)
end
pkg = pkgfind(name.."-devel-static")
if pkg and not pkgvercur(pkg) then
markinstall(pkg)
end
end
for i, pkg in pairs(pkglist()) do
if pkgvercur(pkg) then
installdevel(pkgname(pkg))
end
end
8. Upgrading algorithm Some changes in APT-RPM go mostly unnoticed by the everyday user. One of these changes happened during the release period of Conectiva Linux 9.0. Several days were spent to improve the behavior of APT-RPM in complex situations, like when whole distributions are upgraded. Today, APT-RPM is the only tool used for upgrading Conectiva Linux, and heavy tests were done, even upgrading two distribution versions in a single step. Recently, users of other distributions have reported that the upgrading algorithm is indeed working more reliably, as they are also able to upgrade their own distributions, and problems during upgrades are mostly related to packaging bugs. 9. Internal committing of changes One behavior that many users disliked, was the way it committed the programmed changes to the system. Until recently, APT-RPM did this by executing the rpm binary itself. This behavior indeed had a few disadvantages, like splitting the transition into more than one unit, since the rpm binary doesn't currently support installation and erasure of packages in a single step. Fortunately, since version 0.5.15cnc3, APT-RPM supports internal committing of changes, through the use of the rpmlib API, which leaves the mentioned problems behind. Using the old method is still supported, and is runtime configurable. 10. Synaptic Synaptic is very good graphic interface for package installation which supports both APT-RPM and APT. Lately this software is receiving special attention, and is evolving at large steps. If one would like to have access to the features of APT-RPM or APT with a graphical interface, looking at the Synaptic project is highly advisable. 11. Changes in the original APT Changes being done in the original version of APT, maintained by Debian developers, are constantly being integrated in APT-RPM as well. A small example of this is the recently introduced argument of the install subcommand of apt-get which shows which versions are going to be installed in the system, if the transaction is committed, as shows the following example: % apt-get install -V rpmver Reading Package Lists... Done Building Dependency Tree... Done The following NEW packages will be installed: rpmver (2.0-13498cl) 0 upgraded, 1 newly installed, 0 removed and 8 not upgraded. Need to get 6404B of archives. (...) 12. Credits The core maintenance of APT-RPM is done by the Conectiva developer Gustavo Niemeyer, but it would be unfair to take the credits and not mention other people (without any special order), like Panu Matilainen and Richard Bos, which have always been persistent APT-RPM contributors; Michael Vogt, a Debian developer that has been doing a wonderful job maintaining Synaptic with Gustavo, Jeff Johnson, the RPM software maintainer; ALT-Linux, which constantly send patches "upstream"; Vine Linux, which pushes APT-RPM in the oriental side of the world; the Debian developers which contribute to the original APT project, and many others which contribute to the continuous progress of APT-RPM. 13. Links
(Log in to post comments)
New features in APT-RPM Posted Dec 1, 2003 23:05 UTC (Mon) by proski (subscriber, #104) [Link] One thing I really like in yum is "yum list extras" - the ability to list packages not found in any online repositories. These are the packages downloaded and installed locally. It's useful to review that list and decide if all those packages are still needed. Specifically if the system is used to build packages distributed to other systems, it's very important to be aware of all installed non-standard packages to avoid dependencies that cannot be satisfied from the standard repository.If apt-rpm adds support for local packages, it would be very natural to add such command as well.
New features in APT-RPM Posted Dec 2, 2003 1:28 UTC (Tue) by niemeyer (guest, #17169) [Link] Looks interesting. Here is a Lua script that will do the same thing in APT-RPM:
for i, pkg in pairs(pkglist()) do
ver = pkgvercur(pkg)
verlist = pkgverlist(pkg)
if ver and not verisonline(ver)
and table.getn(verlist) == 1 then
print(pkgname(pkg) .. "-" .. verstr(ver))
end
end
To run it, execute the following command:
apt-cache script ./list-extras.lua
If you want more comfort you may easily turn this into an additonal apt-cache command, and run it like:
apt-cache list-extras
To do that, insert the following lines at the top of the Lua file:
if command_args[1] ~= "list-extras" then
return
end
command_consume = 1
And register it in your configuration with:
Scripts::AptCache::Command:: "/path/to/list-extras.lua";
Notice that, unfortunately, this will only work with the upcoming APT-RPM 0.5.15cnc4, since the verisonline() function was implemented a few seconds ago. :-) The functionality was already there, but wasn't being exported to Lua. If you want it working now, you can check the revision 243 in the repository.
New features in APT-RPM Posted Dec 2, 2003 5:40 UTC (Tue) by proski (subscriber, #104) [Link] That's the kind of reply I like to get. Thank you!Actually, my Red Hat machines use yum (I switched from apt-rpm just recently) and my Debian machine uses apt-get, quite expectedly. I wanted to convert everything to Debian unstable, but the Debian security breach changed my plans. people.debian.org is still closed, and I just cannot use XFree86 older than 4.3.0 with some hardware. I started updating Red Hat machines to Fedora. It's exciting to learn about development of the tool I've been using until recently, but it seems I'm not going to use the results of your work until one of the following events occur:
New features in APT-RPM Posted Dec 2, 2003 7:23 UTC (Tue) by schabi (subscriber, #14079) [Link] I successfully installed XFree86 3.0 using the following apt-sources.list line:deb http://ftp.freenet.de/debian/ ../procject/experimental main contrib non-free HTH,
New features in APT-RPM Posted Dec 1, 2003 23:34 UTC (Mon) by dmantione (guest, #4640) [Link] What is the point of this article?
New features in APT-RPM Posted Dec 2, 2003 0:23 UTC (Tue) by snitm (guest, #4031) [Link] Um, how about... to inform people of the really useful features that APT-RPM has to offer beyond Debian's APT. APT-RPM is a great tool; YUM is really slow. Making yum-users APT-aware is a good thing.Also, many of the features that have been added to APT-RPM could very easily be merged into Debian's APT. Maybe this article will drum up some Debian-interest for the mentioned APT-RPM features; creating Debian user demand for said features could easily help the 2 projects work toward a broader merge. The Progeny APT subversion tree was made available today; Progeny added features like TLS authentication, and more, to Debian's APT. There is a lot of APT-related work going on; convergence of this work (Debian's APT, APT-RPM, Progeny's APT) on a single source tree would be a huge step forward for the utility APT (be it Deb or RPM based). But, this could prove really hard given Debian's APT and APT-RPM use different pkglist formats (text vs binary), and other differences.
I personally like yum Posted Dec 2, 2003 2:30 UTC (Tue) by scottt (subscriber, #5028) [Link] In yum's defence, it's not that the program is slow so mush as it assumes you have a fat pipe to the yum repository. This probably works well in an university environment where each department can offer a local repository server for its special packages but less well over the internet.Having to download all that rpm header is sure painful but it lets yum use librpm for dependency handling locally. Also note that apt-rpm is implemented in C and it has to make a special effort to "export" its functionality to the lua scripting language, I think the apt-rpm functionality is quite impressive but the difficulty in merging the debian,apt-rpm and Progeny trees is only complicated when the code is in 40000 lines of C instead of 5000 lines of python.
I personally like yum as well Posted Dec 2, 2003 3:22 UTC (Tue) by niemeyer (guest, #17169) [Link] First, let me position myself. I'm not against yum. I do like the idea of someone working on a completely different implementation, and I'd really like to see yum evolving. Also, I'm a big Python fan, being even one of the contributors of the core language, as you can see in my (wiki|web)log and my projects page. Now... "Having to download all that rpm header is sure painful but it lets yum use librpm for dependency handling locally."APT-RPM meta-information is also based on rpm headers. OTOH, unuseful information in that context is stripped when the "pkglist" (AKA. header list) is created. "Also note that apt-rpm is implemented in C and it has to make a special effort to "export" its functionality to the lua scripting language"APT-RPM is implemented in C++, and the important thing about the Lua extension is not the effort I've done to implement it, but how it satisfy users needs. "The lua code looks quite nice, though a bit too imperative since most people are used to writing filters under the unix shell?"I don't understand this point. You seem to like the fact that yum is written in Python (as I do). "While yum is implemented in python utilizing the same python bindings to rpm etc used by the Redhat system management tools."And APT-RPM is implemented using the C bindings used by every distribution using the RPM API, including the python bindings. Indeed, I belive APT-RPM is the only software today supporting the API of RPM version 3.0.X, 4.0.X, 4.1.X, 4.2.X, and CVS. "IMHO, programs like yum or apt-get shouldn't be implemented in C."Agreed. I'd be glad if APT was implemented in Python, but it isn't. I think the apt-rpm functionality is quite impressive but the difficulty in merging the debian,apt-rpm and Progeny trees is only complicated when the code is in 40000 lines of C instead of 5000 lines of python.Today, Michael Vogt has sent me a patch of 12kb which makes APT-RPM work in Debian. Also, the APT-RPM port has always been up-to-date with the changes done in the Debian APT, as was written in the article. Progeny is one more in the merging group, and is very welcome. Debian has a slower merging politic, but is merging changes from APT-RPM and Progeny as well. About the number of lines, I'm sorry, but I don't understand why you belive that this information is relevant, since they have a different set of features and Yum will need a considerable amount of work to get where APT-RPM currently is. Of course, it will still have less lines of code if it ever get into the APT-RPM stage, since it's implemented in Python, but that's not the case right now.
I personally like yum as well Posted Dec 2, 2003 4:48 UTC (Tue) by scottt (subscriber, #5028) [Link] First I would like to thank you personally for your python contributions. I was just using the bz2 module at work yesterday. I didn't recognize your name until I read your projects page. My comments about the extra effort to support lua and the code size of apt-rpm is really just another way to say I think C/C++ is not the best choice for an implementation language in this case. Though 40000 lines isn't large for a C++ project, complexity still has its cost.The python culture favors C/C++ libraries having python bindings instead of embedding an interpreter in a C/C++ app for a reason. About the lua code being too "imperative", what I ment was perhaps using a more "functional" style with "iterators" and "map" etc, instead of for-loops and "markinstall". ie. install_devel() in python using an imaginery library:
def devel_packages_to_install():
def dev_pkgs(pkgs):
for p in pkgs:
try:
yield package(p.name+'-deve')
yield package(p.name+'-deve-static')
except:
pass
return [ p for p in dev_pkgs(uptodate_packages())
if not p.is_uptodate() ]
map(lambda p: p.install(),devel_packages_to_install())
As you can see I'm dying for generator comprehensions ;) By the way, does lua have a standard iterator/lazy-list interface?
I personally like yum as well Posted Dec 2, 2003 5:10 UTC (Tue) by niemeyer (guest, #17169) [Link] About the "imperativeness", I understand. Lua syntax is indeed a little bit more limited. OTOH, that's expected in a language for embedded environments. About Lua iterators, yes, the generic 'for' loop allows the underlying function to provide data step-by-step. Lua also implements coroutines, which are the counterpart of Python's generators. For more information, please, check the Lua manual.
Installing random remote packages Posted Dec 2, 2003 4:46 UTC (Tue) by niemeyer (guest, #17169) [Link] While we're talking about APT-RPM features, here's a simple but cool one, which will be available in the next version. Of course, unsatisfied dependencies are solved with the registered repositories after the packages are downloaded. % apt-get install ftp://mapi8.distro.conectiva/pub/(...)/rpmver-2.0-13498cl.i386.rpm Get:1 ftp://mapi8.distro.conectiva/(...)/rpmver-2.0-13498cl.i386.rpm [6404B] Fetched 6404B in 1s (5844B/s) Reading Package Lists... Done Building Dependency Tree... Done Selecting rpmver for 'rpmver-2.0-13498cl.i386.rpm' The following NEW packages will be installed: rpmver 0 upgraded, 1 newly installed, 0 removed and 5 not upgraded. Need to get 0B/6404B of archives. After unpacking 5552B of additional disk space will be used. Committing changes... Preparing... ########################################### [100%] 1:rpmver ########################################### [100%] Done.
Installing random remote packages Posted Dec 2, 2003 4:53 UTC (Tue) by scottt (subscriber, #5028) [Link] Do you think registering a special url space for apt and making this "point-and-click" in browsers will work better than a front end like synaptic?I remember reading this idea from a gnome-related mailing list.
Conflicts? Posted Dec 2, 2003 19:15 UTC (Tue) by southey (subscriber, #9466) [Link] "Dependencies are automatically handled, so if one tries to install a package that requires others to be installed, it will download all needed packages and install them."So what happens to old versions that are required by other programs? Especially older and poorly maintained projects require old libraries so you end with conflicts that are not easy to resolve. Even things like the new threads in the kernel meant some projects (like Wine) crashed and these take time to get resolved.
Conflicts? Posted Dec 2, 2003 19:25 UTC (Tue) by TimCunningham (guest, #10316) [Link] Generally (at least in my experiance, with Debian) what happens is that the latest version of the library that breaks things will be held off until all of the other software is made to work with it. This means you sometimes get delays in having the latest everything.
Conflicts? Posted Dec 2, 2003 21:22 UTC (Tue) by JoeBuck (subscriber, #2330) [Link] In such cases (where some programs need an old version and some need a new version of some package), apt will tell you that if you proceed with the install, certain packages will be removed, and will then ask you if you want to proceed. Then it's up to you to choose.
Conflicts? Posted Dec 11, 2003 14:04 UTC (Thu) by Peter (guest, #1127) [Link] Especially older and poorly maintained projects require old libraries so you end with conflicts that are not easy to resolve. The usual solution to that problem is, when a new library comes out which is not ABI-compatible with an older version, the new one gets a new name. And installs itself so as not to stomp on the old version. At least that's how things are done with sane developers and distributors. If your version of Linux doesn't do that as a matter of course, you ought to consider upgrading to one that does.
New features in APT-RPM Posted Dec 14, 2003 2:06 UTC (Sun) by nyo (guest, #17806) [Link] Do you really think, that Debian uses only main/contrib/non-free sections? And "APT-RPM uses more frendly audio/devel/etc." sections? In Debian i see audio, devel, contrib/graphics, non-free/web, etc. And the Non-US sections too. :)
New features in APT-RPM Posted Jan 22, 2004 16:04 UTC (Thu) by niemeyer (guest, #17169) [Link] Please, avoid quoting something you haven't read.I never said that the APT-RPM component model was more friendly. I said that the *Conectiva Linux* component model was more *finely grained*, and that's true. We have about 30 components in our repository now. Also notice that I haven't said that this was better than the Debian model. I said it was different, and I have implemented the meta-component concept to improve the user experience.
|
|||||||||||||