Advances in Debian's package manager

At the 2015 edition of DebConf in Heidelberg, Germany, David Kalnischkies provided an update on several recent enhancements to the Apt package-management tool that plays such a key role in Debian and its derivatives. His talk dealt with the upcoming 1.1 release, which was recently uploaded to Debian experimental. The new branch features a number of security enhancements and implements some oft-requested features. As he put it, the Apt team is setting its sights on higher-level package managers like Aptitude.

Kalnischkies began by pointing out that one of the top search-engine suggestions to complete the phrase "apt-get is " was the rather unflattering "broken." While that was discouraging, he pointed out that searching for a description of the program also turns up Justin Rye's somewhat rosier summary: "It's a 100% bug-free solution to all life's problems, requiring no human intervention at any stage". Somewhere in between the two extremes, he settled on his title: "Apt has super(cow) powers."

The "cow" is a reference to one of Apt's best-known Easter eggs: typing apt moo generates an ASCII-art cow image. But most people are unaware, he said, that there are actually multiple cow-related Easter eggs in the tool (which he proceeded to demonstrate). Similarly, many Apt users are unaware of everything that the program does behind the scenes. Though it is most frequently used to simply install Debian packages, it also handles resolving package lists from multiple repositories and local sources, as well as figuring out the proper order to install or remove dependencies, and handling security checking.

In the dark ages of 1997, he said, it was first conceived as a replacement for the dselect tool—which was hard to use and confusing for novices. Apt only reached 1.0 in 2014, when the developers decided that it finally met all of the functional and ease-of-use criteria set out in 1997. With that milestone out of the way, the project has now undertaken new work and experimental features. At DebCamp the week before DebConf, he said, the team closed more than 300 outstanding issues—some of which were quite old.

Security

Kalnischkies then turned to the new features that users could try out in the 1.1 development branch. Many are network-security enhancements, he said. First, downloads are now performed as the unprivileged user "_apt" rather than as root. Second, there are many more hash checks performed—namely, every package's hash is checked when the file is downloaded and again when it is uncompressed. Earlier releases only checked uncompressed hashes, and only on some files—a disparity that was confusing for developers as well as being insecure.

The new version also checks for the presence of every file it expects to see before starting a download, rather than assuming that what it expects is present in the repository. In old releases, the assumption that certain files were present was only made for auxiliary files like Translation-* files, but it was a minor security hole nevertheless. Those URLs were predictable, and there was some chance that an attacker could exploit them to make Apt download something malicious. Speaking of auxiliary files, Apt can now download any arbitrary file. Apt front-ends like Aptitude and apt-file had been implementing their own downloading code in order to handle things like screenshots and AppData metadata files, he explained. By performing those downloads itself, Apt can at least verify that the downloads are being done securely.

At DebConf, attendees were encouraged to use an on-site mirror of the Debian archive in order to conserve bandwidth. Kalnischkies did a live demonstration next, to illustrate that Apt now reports back to the user which mirror it is using. Doing so provides a bit of added security (against man-in-the-middle attacks), but it is also practical: users can now report any errors to the administrator of the correct mirror. Apt also checks for signatures on each repository's Release file, and issues a warning if no signature is present. In the future, he said, "it will get harder and harder to run an unsigned repository," until they are deprecated entirely.

At that point, an audience member asked if requiring such signed files would make it impossible to use a local package repository—which is often needed when bootstrapping a new system. Kalnischkies replied that an unsigned local repository can be marked as "trusted" in the Apt configuration file and the bootstrapper can thus avert any trouble.

Additional features

The list of other new features is a long one; Kalnischkies highlighted just a few. Using PDiffs (package diffs, which are binary deltas of updated packages) now works for everyone, and Kalnischkies demonstrated that downloading with PDiffs is, in fact, faster than downloading without them—something that Apt's critics had often said was not true in the past.

Similarly, a change that landed during the DebCamp hacking sessions was a fix for pinning packages (that is, designating that a particular, installed version of a package should not be updated even when a new release appears). In prior releases, the pinning algorithm could get confused by details like packages that changed their name or their version-number format between releases. "Pinning now works as advertised," he said. "I could fill a whole session just discussing how pinning works; right now you'll just have to trust me."

The new version of Apt will also support several additional parameters for the repository entries in sources.list files. Among the new options is a reference to an OpenPGP key or keyring; adding the parameter locks the repository to a specific key for signature checking. That allows the user to do a form of key pinning, which would help alert them if an attacker compromised a repository and tampered with its packages. Under the current system, the attacker could simply upload a new Releases.gpg file signed by their own key, and an Apt user would not know that the old key had been replaced.

Other new parameters include an indicator that the repository includes other content types (such as the AppData files mentioned earlier) and one disabling the check on a repository's "Valid Until" setting. Users that run tests against old packages in historical Debian archives asked for the validity-date disabling feature in order to silence the deluge of warnings Apt would throw when it thought that a repository was past its expiration date.

Apt can also be used to install individual Debian packages from their .deb files, he continued; in the past, users had to use dpkg to install a package from a standalone file, then immediately run apt install -f (for "fix broken"), which was far from ideal. Apt has also gained the ability to automatically detect and download the build dependencies of a source package—whereas, in previous releases, the build-dep command only worked on binary packages. Thus, the user would have to manually locate the dependencies by reading through the source package's control files.

There were additional new features he wanted to discuss, Kalnischkies said, but he cut the presentation short to reserve plenty of time for questions. A number of attendees wanted to know how the enhancements to Apt would affect other programs, such as gdebi. He replied that some of those programs may go away in the future, but would not do so soon. One advantage of gdebi, he said, was its set of language bindings, which Apt would have to add in order to be a direct competitor.

Another attendee asked if there was any advantage to using dpkg instead of Apt. He replied that Apt attempts to handle complex things like dependency resolution, but that sometimes it decides it cannot do what the user requests. In those situations, if the user really does know what the answer is, they can do it with dpkg.

On the whole, though, Apt is making improvements in many areas where it was regarded as weak in the past. It may not be perfect yet, but the strides it has taken are significant.

[The author would like to thank the Debian project for travel assistance to attend DebConf 2015.]