Distributions

Debian adds continuous integration

In June, Debian formally unveiled the Debian Continuous Integration (debci) project, which is an effort to integrate dependency-based automated testing into the rather sizable Debian package archive. So far, only a fraction of Debian's packages are making use of the service, but many more could be added with a little effort—and the benefits for Debian are potentially large.

For years, Debian has enjoyed the reputation of having one of the largest collections of available packages among Linux distributions; according to various project statistics, there are more than 20,000 source packages in the unstable distribution, which result in more than 40,000 binary packages and an untold number of inter-package dependencies. For a project without a corporate parent and relying on volunteer package maintainers, that adds up to quite a quality-assurance challenge. Some of the larger packages (such as GNOME and KDE) can pull in well over 1,000 dependencies, which makes manual testing at the very least impractical.

Consequently, as debci developer Antonio Terceiro explained in a blog post, there has long been an interest in deploying an automated testing system. A key piece of the puzzle was Ian Jackson's autopkgtest, which can execute tests on installed binary packages within a testbed system. In 2011, the autopkgtest framework was accepted as Debian Enhancement Proposal 8 (DEP-8).

DEP-8 was accompanied by a specification for defining tests within a Debian source package. The autopkgtest tool can run tests against the source package itself, but it can also execute tests designed for the corresponding binary package. The debci effort is geared toward finding regressions (particularly those that might result from changes in a dependency) rather than build errors, though, so it focuses on the binary tests.

Although not all Debian packages include their own tests, quite a few already do. At the end of 2013, Terceiro started experimenting with a tool that would run autopkgtest against the entire archive. That codebase is what eventually became debci.

Currently, an instance of debci is running on ci.debian.net, and it executes tests against roughly 350 packages. The server updates its Apt cache four times per day, looking for updated packages that have autopkgtest tests defined. If a package or any of its dependencies have changed since the last test was run against it, it is scheduled for a new run. In addition, tests are run once every month even if there are no changes.

Several test-execution backends are supported in autopkgtest, including LXC containers, QEMU/KVM virtual machines, and "ssh testbeds" (which assume nothing other than a working ssh connection). Each option provides a different way to isolate the test environment from the host system. The ci.debian.net server makes use of the schroot backend for most packages. Schroot creates a clean chroot environment and, according to the documentation, is the fastest option that provides filesystem isolation, although it has some limitations (such as tests not being able to start system services or alter the network configuration).

For a given package, debci sets up the schroot environment, updates the target package and all of its dependencies, then extracts and executes the target package's test suite using autopkgtest and logs the results. Autopkgtest supports quite a bit of variety in its test output (including no output, simple TEST: FAIL messages, and more detailed reports). The ci.debian.net server publishes new test results once every hour. In addition to global success/failure statistics for the entire archive, each individual package's results are published as an Atom feed, and there is a combined whole-archive feed.

The individual test results are also accessible through the site's web interface. For example, here is the apt package's test-result history. The debci log for each entry indicates what triggered the test (i.e., a change in the package itself or a change in a dependency), the current dependency list, the test result, and the test execution time. The autopkgtest logs and raw test output are also available for each run. In the test run linked to above, the test session seems to have failed due to a configuration problem in the test, rather than a bug in Apt; debci will also mark some problems as tmpfail situations, such as this evolution-data-server test for which the dependencies could not be satisfied.

Terceiro noted in his blog post that the number of packages including test suites has grown from around 200 to its current level of about 350 since the server was first brought online in January. In addition—and perhaps more importantly—the percentage of packages passing all of their own tests has grown considerably over that same time frame, from less than half to over 75%.

Of course, 350 packages is a drop in the bucket compared to 40,000, but one must start somewhere. The fact that the number of package maintainers who are participating is on the rise, coupled with the fact that more tests are passing, suggests that debci is proving to be a useful tool for the Debian project. Terceiro also noted in his blog post that there are now multiple contributors to the project, including current autopkgtest maintainer Martin Pitt and two Google Summer of Code (GSoC) students. It may still take quite a while for debci to achieve majority coverage (much less full coverage) of Debian's vast package archive, but with easy-to-see results, its popularity might also catch on in hurry.

Comments (none posted)

Distribution quote of the week

-- Ian Stakenvicius

-- Chuck Forsberg

Comments (none posted)

Debian 7.6 released

The Debian project has announced the sixth update of its stable distribution Debian 7 "wheezy". "This update mainly adds corrections for security problems to the stable release, along with a few adjustments for serious problems. Security advisories were already published separately and are referenced where available."

Full Story (comments: none)

FreeBSD 9.3 released

The FreeBSD Release Engineering Team has announced the availability of FreeBSD 9.3. This update adds support for the bookmarks feature in the zfs filesystem and the fetch library now supports SNI (Server Name Identification). See the release notes for details on these and many other updates and improvements.

Comments (none posted)

Distribution newsletters

• DistroWatch Weekly, Issue 567 (July 14)

Comments (none posted)

Boyer: At the playground

Fedora kernel team member Josh Boyer, writes about a Fedora kernel-playground Copr (Cool Other Project Repository) on his blog. The idea is to provide an unsupported kernel that has some new features for those who want to help develop and test them. "OK, now that we have that out of the way, let's talk about what is actually in kernel-playground. At the moment there are two additions on top of the standard rawhide kernel; overlayfs (v22) and kdbus. Overlayfs is one of the top competing "union" filesystems out there, and has actually been posted for review for the past few releases. It has the best chance of landing upstream sometime this decade, and there has been interest in it for quite a while. I believe things like Docker would also be able to make use of it as a backend. I'll track upstream submissions and update accordingly. kdbus is of course the thing that Lennart Poettering and Kay Sievers have been talking about at various conferences for a while now. It is the in-kernel d-bus replacement. It has not been submitted for upstream review yet, but systemd already has support for it and things seem to be progressing well there."

Comments (4 posted)

Best Linux Desktop: Top 10 Candidates (Datamation)

Matt Hartley picks his favorites for best Linux desktop: some for newcomers and some for experienced enthusiasts. Some popular distributions are completely missing from the list (e.g. Debian, Fedora, and openSUSE), while you may not have heard of some of his other choices. In the newcomer category we have Ubuntu, Linux Mint, Zorin OS, SolydX, PCLinuxOS, and Lubuntu/Xubuntu. For more experienced users he starts with a brief mention of Arch Linux, Gentoo, and Slackware, but moves on to Antergos, Sabayon Linux, WattOS Microwatt Edition, and VectorLinux.

Comments (none posted)