Gentoo debates recruitment schemes

Cultivating contributors is a tricky endeavor; large projects like Linux distributions need to add developers, testers, packagers, and more on an ongoing basis. But recruiting involves more than extending an invitation; training new talent on the ins-and-outs of the development process is vital, as is deftly handling volunteers that don't quite have their act together. Debian, for instance, has a multi-step process for joining that requires both contribution and a recommendation from existing members.

Gentoo has long taken the recruiting game seriously as well, but it recently decided to shut down its developer-recruitment web application, and return to its previous method of email-submitted "quizzes." Opinion is divided as to which direction the recruitment process should take; the distribution has historically found success with its structured process, pairing new volunteers with mentors for training. But if the mechanics of maintaining that process become a burden, it can drive off new contributors and mentors.

In the past, the Gentoo recruitment training process centered around a set of quizzes that each new recruit had to complete successfully before getting commit privileges. There were two quizzes: one for those who would be working with the distribution's ebuild build system or Portage tree, and another for those who would be working on infrastructure and other non-build components. Both quizzes include a mix of policy and technical questions, requiring (sometimes lengthy) essay-style answers. For example, the ebuild quiz asks:

in the "Organizational structure" section, and:

in the "Ebuild technical" section. The recruit would send the completed quiz to his or her mentor and, upon receiving a satisfactory grade, would advance to the next step: opening a recruitment "bug" to track the recruit's progress, which eventually culminates in account setup.

The term "quiz" might be slightly confusing to some who associate the word with brief and/or quick tests. In Gentoo's case, the recruit could take as much time as necessary to complete the questions, and might be asked by the mentor to try again through several iterations. On the plus side, this technique emphasizes letting the new developer get familiar with the documentation and the distribution's way of doing things. But it also led to new recruits taking months or even years to complete the quizzes, often while they continued to contribute back in "unofficial" ways.

The web application (at recruiting.gentoo.org) was deployed in 2010, with the goal of streamlining the process by storing the questions and answers online. Mentors could provide feedback through the application, without juggling email threads. But the web application has not proven so useful in practice: it is buggy, it has UI issues, it runs on an outdated version of Rails, and there is insufficient developer-power in the project to fix it up. All of those factors combined to convince the recruiters to move back to the old-style, email-driven quiz process.

Markos Chandras from the Gentoo Recruiters team announced the decision on July 14, saying that new recruits (not including those who have already started using the web application) should use the old quizzes instead. "We understand that quizzes is not an ideal way to 'hire' people either, but they worked ok for all these years and it is the only alternative we have at the moment." Chandras added that hopefully a future Google Summer of Code (GSoC) project will be able to improve the application.

But not everyone agreed that the old-style quizzes worked acceptably, or that the web application was the only alternative to consider. Ben de Groot said that the time involved takes away from time the new recruit could contribute to Gentoo:

De Groot proposed improving the wiki documentation to cover the quiz material, and having mentors walk their recruits through the documentation while simultaneously helping them learn development work. Alternatively, he said, mentors could assign tasks for recruits to complete. Chandras replied that the quizzes cover a specific set of material to ensure consistency between mentors, and that doing away with them would necessitate someone else monitoring the mentors to ensure they cover the proper work. On the other hand, he did like the idea of improving the wiki, and suggested that the post-quiz review steps could be simplified, particularly if a recruit has already been contributing.

Rich Freeman expressed surprise at the lengths of time taken by some recruits, but agreed that the quizzes have weaknesses, saying "I did struggle because policies were not always spelled out" and "sometimes the indirectness of some of the questions was frustrating," but that he completed his quiz in eight hours, and learned a lot in the process. He also suggested improving documentation, in particular by creating step-by-step tutorials for ebuild, which could guide new recruits through learning the system (in contrast to the existing documentation, which is predominantly reference material).

Several people responded that the absolute time required to complete the quizzes was not the issue, rather it was finding the free time to devote to the process amidst all other responsibilities (including Gentoo contributions). Peter Stuge commented, jokingly, that "the idiots that the quizzes are designed to keep out can spend two (or four/eight if they need) days to pass anyway with a little dedication, while less idiotic idiots such as perhaps myself need years because we're doing whatever work as opposed to learning foundation bylaws by heart."

Freeman also speculated in several messages on the impact the recruitment process has on the overall Gentoo culture, apart from the method used to indoctrinate new developers. On one hand, he suggested that the need to ensure a training regimen stemmed from technical choices like using CVS (where commit access is required). Using Git instead would alleviate some of the concern about adding new developers, because they could still do useful work in their own trees. More developers with more freedom to improve packages, he said, "would be a good thing. The all-or-nothing model too often turns out to be nothing."

On the other hand, Freeman argued that the non-technical topics in the quizzes — such as learning to work within teams, ask questions, and build consensus — are more important in the long term:

Freeman may have summed up the feelings of many Gentoo developers: the specifics of the training process are less important than the fact that is it deliberate and guided by active mentors and recruiters, because the end goal is to integrate the new developers into the Gentoo community — not to train them on a particular suite of tools. On that front, the web application still has its share of fans. Theo Chatzimichos said he prefers it to the email-driven quizzes because it simplifies keeping track of recruits' answers. Chatzimichos said he mentors two or three recruits at a time, and proposed putting out a call for volunteers to revamp the web application, while making sure to not let the web application get out-of-sync with the quizzes.

In an email, Chandras said that Gentoo averages 10 to 15 recruits per year. That may not be many when compared to large distributions like Ubuntu or Fedora, but in a sense it only makes the recruitment process more critical. It is clear from the discussion that neither the old email-driven quizzes nor the recent web application quite meet everyone's needs — but at least the recruiters and the mentors are committed to sticking with the process even in spite of its awkward points.