The role of the Debian ftpmasters

Linux distributions don't simply appear on mirrors and BitTorrent networks fully formed. A great deal of work goes on behind the scenes before a release sees the light of day. Linux users who aren't involved in the production of a Linux distribution may not fully appreciate all of that work. Take, for example, the work done by Debian's ftpmasters team.

While it's obvious, or should be, that someone has to actually create the packages that go into Debian and one assumes that there is Quality Assurance (QA) and so forth, the ftpmasters team is largely invisible to users. This was highlighted recently by Joerg Jaspert's call for new volunteers for the ftpmaster team.

The essence of the job is maintaining the Debian archive, accepting new packages (NEW), maintaining the scripts for processing incoming packages, and pulling packages when asked by QA, among other things. The ftpmasters also move packages from testing to stable when the time comes, though the decision to do this is made by the release managers team. The canonical description of the ftpmaster job is more detailed (unfortunately ftpmaster.debian.org was down at the time this article was written), but the basic gist is that the team deals with new packages and keeps the archive going.

The job is unique to Debian, in that other distributions don't have an exact analog to the ftpmasters team. openSUSE, for example, has an autobuild team to ensure package quality, and Ciaran Farrell looks over licenses when there's a question. The various teams decide what packages do or don't go in, and if there's a dispute at some point it can be handled by the release manager and product manager for openSUSE.

Debian is, as Jaspert alluded to, "not getting smaller" and managing the number of new packages is a "kind of Sisyphean task." The Debian archive contains thousands of packages, and the NEW queue can have hundreds of packages awaiting approval. NEW packages are those entering Debian for the first time, which do not have source packages in the archive, or those adding new binary packages. New versions of existing packages are moved automatically into the pool.

Currently, there are fewer actual "masters" than Jaspert would like, which is to say only Jaspert and Mark Hymers are currently serving as ftpmasters. The team also has six assistants at the moment: Barry deFreese, Chris Lamb, Frank Lichtenheld, Mike O'Connor, Alexander Reichle-Schmehl, and Torsten Werner. Though it doesn't sound like a lot, Jaspert says he'd really just like to add one more ftpmaster, as the bulk of the work is done by the assistants:

It takes some time to bring new assistants, and masters, into the fold. It requires a fair amount of knowledge to do the job. Jaspert says it's not only necessary to have a basic understanding of "just about every programming language you can imagine" but also have a love of reading and dealing with legal texts. The team is responsible for digging through new packages and sussing out all manner of problems (particularly legal ones). Though it is not responsible for the actual QA work, the ftpmasters team is the line of defense before packages enter the archive, which is an enormous responsibility. Before one is added to the assistants team, there's a training period to learn the ropes of working with packages in NEW:

While one may be able to graduate from trainee to assistant in six days, Jaspert says that six months is the minimum stay to graduate to master:

The ftpmasters team wields a considerable amount of power over what does, and doesn't, make it into Debian package archive. For the vast majority of packages, the decisions may be cut and dried. At least there are relatively clear-cut guidelines based on known license problems, lack of licensing information, a failure for packages to build from source, policy violations, or any number of other known issues.

The ftpmasters also have room for discretion in applying the rules and may reject packages for other reasons. Consider, for example, the decision to reject qmail packages from inclusion. This was less about Debian Policy and more, apparently, about the ftpteam's opinion of Qmail.

Though the reasons for rejecting qmail or other packages may not be specifically enumerated in the guidelines, Jaspert says that it had various policy and Filesystem Hierarchy Standard (FHS) violations, and that the guidelines are not all-inclusive. "Basically its an 'apply common sense' [policy] and nothing one can hardcode. 'Is this package one thats about to go bitrot even before its in the next stable release?'"

There have also been other issues centered around the team. For this reason, the team not only needs to have a depth of programming knowledge and interest in licensing, but also a very thick skin. As Jaspert writes, the team needs to be able to deal with unpopular decisions and take some flames. But he also says that the team "doesn't (usually) bite" and hopes that people will talk to the team when there's a disagreement over decisions made by them.

By and large, the work done by the ftpmasters is invisible to most users, but crucial to the success of Debian and all its downstream projects.