Linux distributions don't simply appear on mirrors and BitTorrent networks fully formed. A great deal of work goes on behind the scenes before a release sees the light of day. Linux users who aren't involved in the production of a Linux distribution may not fully appreciate all of that work. Take, for example, the work done by Debian's ftpmasters team.
While it's obvious, or should be, that someone has to actually create
the packages that go into Debian and one assumes that there is Quality Assurance (QA) and so forth, the ftpmasters team is largely invisible to users. This was highlighted recently by Joerg Jaspert's call for new volunteers for the ftpmaster team.
The essence of the job is maintaining the Debian archive, accepting new packages (NEW), maintaining the scripts for processing incoming packages, and pulling packages when asked by QA, among other things. The ftpmasters also move packages from testing to stable when the time comes, though the decision to do this is made by the release managers team. The canonical description of the ftpmaster job is more detailed (unfortunately ftpmaster.debian.org was down at the time this article was written), but the basic gist is that the team deals with new packages and keeps the archive going.
The job is unique to Debian, in that other distributions don't have an
exact analog to the ftpmasters team. openSUSE, for example, has an
autobuild team to ensure package quality, and Ciaran Farrell looks over
licenses when there's a question. The various teams decide what packages do
or don't go in, and if there's a dispute at some point it can be handled by
the release manager and product manager for openSUSE.
Debian is, as Jaspert alluded to, "not getting smaller" and managing the number of new packages is a "kind of Sisyphean task." The Debian archive contains thousands of packages, and the NEW queue can have hundreds of packages awaiting approval. NEW packages are those entering Debian for the first time, which do not have source packages in the archive, or those adding new binary packages. New versions of existing packages are moved automatically into the pool.
Currently, there are fewer actual "masters" than Jaspert would like,
which is to say only Jaspert and Mark Hymers are currently serving as
ftpmasters. The team also has six assistants at the moment: Barry deFreese,
Chris Lamb, Frank Lichtenheld, Mike O'Connor, Alexander Reichle-Schmehl,
and Torsten Werner. Though it doesn't sound like a lot, Jaspert says he'd
really just like to add one more ftpmaster, as the bulk of the work is done
by the assistants:
[Three] is a good number for masters. Though two also works, as long as its not just one. The majority of work is with the assistants, NEW and removals and overrides, masters usually have the background work. Keeping the archive running. Merging patches to the software and making sure the archive still runs when deploying it. Such stuff.
It takes some time to bring new assistants, and masters, into the fold. It requires a fair amount of knowledge to do the job. Jaspert says it's not only necessary to have a basic understanding of "just about every programming language you can imagine" but also have a love of reading and dealing with legal texts. The team is responsible for digging through new packages and sussing out all manner of problems (particularly legal ones). Though it is not responsible for the actual QA work, the ftpmasters team is the line of defense before packages enter the archive, which is an enormous responsibility. Before one is added to the assistants team, there's a training period to learn the ropes of working with packages in NEW:
"The way this setup works is simply letting trainees access the ftpmaster machine and the NEW queue. You can look at packages and their source as any other team member. But trainees can not do the actual ACCEPT or REJECT. Instead you have a special ability to leave notes about the packages, explaining what action you would take and why. The other team members will then review those notes and either follow your advice or tell you why they decided to do something different.
After a while we and you will know if you actually fit the team, but more important we (and you yourself) will know if you should (want to) continue doing NEW and will promote you up to assistant. We set ourself a time limit of 6 months as a maximum stay in the trainee group, but none of the current team members has ever stayed in trainee that long. The longest is 3 months, the shortest is 6 days.
While one may be able to graduate from trainee to assistant in six days, Jaspert says that six months is the minimum stay to graduate to master:
For masters its also 6 months, but this time a minimum before we look at "upgrading" them. And then its a discussion between the candidate and the existing masters. Not all of the assistants want to become masters, some are happy with the assistant role, for various reasons. Some of them private, some of them just do not want more power, various. We accept that and be happy that they are assistants and do their share of work in that role.
So the short summary for "graduation": Both, the candidate and the existing masters are happy with it. Then we go the usual road in Debian and voila, one more is there.
The ftpmasters team wields a considerable amount of power over what does, and doesn't, make it into Debian package archive. For the vast majority of packages, the decisions may be cut and dried. At least there are relatively clear-cut guidelines based on known license problems, lack of licensing information, a failure for packages to build from source, policy violations, or any number of other known issues.
The ftpmasters also have room for discretion in applying the rules and may reject packages for other reasons. Consider, for example, the decision to reject qmail packages from inclusion. This was less about Debian Policy and more, apparently, about the ftpteam's opinion of Qmail.
Though the reasons for rejecting qmail or other packages may not be specifically enumerated in the guidelines, Jaspert says that it had various policy and Filesystem Hierarchy Standard (FHS) violations, and that the guidelines are not all-inclusive. "Basically its an 'apply common sense' [policy] and nothing one can hardcode. 'Is this package one thats about to go bitrot even before its in the next stable release?'"
There have also been other
issues centered around the team. For this reason, the team not only
needs to have a depth of programming knowledge and interest in licensing,
but also a very thick skin. As Jaspert writes, the team needs to be able to deal
with unpopular decisions and take some flames. But he also says that the
team "doesn't (usually) bite" and hopes that people will talk
to the team when there's a disagreement over decisions made by them.
By and large, the work done by the ftpmasters is invisible to most users, but crucial to the success of Debian and all its downstream projects.
to post comments)