Lost user questions and GitHub

GitHub is currently the dominant hosting service for open-source software projects. As such, the company is in a unique position to observe and report on trends across a wide swath of the open-source community. At OSCON 2016 in Austin, GitHub's Rachel Berry presented a unique look at the lessons learned by GitHub's support team—"unique" because that team catches a surprisingly high number of support questions from users unable to contact the projects themselves. "There are thousands of questions that aren't making it into your issue tracker where they belong," she said. "The question is how you can capture that and turn these people into your user community."

Berry described her position as "technical supportocat." She works on a team of thirteen who answer questions about the GitHub platform, API, Git in general, and essentially anything else not related to user-account management (which has a separate team). GitHub has about 14 million active user accounts, not to mention all of the users without accounts. They generate about 300 questions a day for the team. Over the past seven years, she said, the team has logged more than 27,000 questions about GitHub-hosted projects—although the frequency seems to be on the increase. She reported that more than 4,000 project-related questions had come in since she submitted her OSCON talk proposal a few months ago.

"These are people who contact us instead of contacting you," she said. "These are questions about your projects." Looking at those requests, some common threads emerge. The first is that 57% of the project-related questions come from people without GitHub accounts—a number that she and several in the audience said was surprisingly high. The question-askers break down into three broad groups, she said: the people who cannot figure out how to contact you, the people who have contacted your project but have never heard back, and the people who are either too embarrassed or too intimidated to contact your project.

When people don't know how to contact you

The first category of people is the one with the easiest solution: put contact information everywhere. But even that solution can entail some subtlety. Berry showed some brief, anonymous snippets from the confusing support questions the team encounters in this category, including those that dive right into the particulars of some technical, project-specific detail. But in one of those snippets, Berry highlighted a phrase: "Command prompt window says to report the error to github.com." Clearly, the prompt's instructions were not detailed enough, at least for that user.

The best practice Berry recommends is to have a formal contact page on your project's web site. But she noted that too many project sites feature a "contact us" button that links to the project's issue tracker on GitHub. Any user who is not already familiar with GitHub and issue trackers, though, will find it baffling to suddenly be taken to a new site. Worse still, some projects link their contact button to GitHub's "new issue" form. And, for any user not logged in to a GitHub account, clicking on that link takes them to a generic GitHub account-creation page instead of the intended target. It is hardly surprising, then, that many people landing on such a page see the "contact us" link in the page footer, and send their request to GitHub support.

The right approach is to separate contacting the project from opening an issue. She pointed to several projects that do an excellent job at this task: Bower, for example, provides multiple contact options (StackOverflow, IRC, Twitter, and mailing lists). RethinkDB also provides multiple options, though it loses a few points by calling the relevant page "community," which new users might not look for.

Regardless of what one puts on the project web site, Berry said, the same information should be duplicated in the GitHub repository's README file. That is because the GitHub README is frequently the user's entry point, rather than the web site. Thus, it should point visitors to the information they seek. All-too-common problems on GitHub include projects burying their contact information at the bottom of a multi-page project description README, or having a blank (or nearly blank) README file. GitHub's server logs indicate that most visitors who file support requests spend less than one minute on the repository page before contacting support, she said; projects need to optimize for that and put their contact information where it cannot be missed.

People who never hear back from your project

The second group of support requests comes from people who tried to reach out to the project and never heard back. Clearly, this lack of interaction is not beneficial for the user or the project. In some cases, the user's attempt to contact the developers is foiled by the project owner turning off the GitHub issue-tracking feature entirely. That should almost never be done, Berry said. Perhaps it is a viable approach for mothballed projects but, even then, the fact that the project is not accepting issues should be explained in the README.

Opening an issue and never hearing back is frustrating for the user, of course. But the simple reality is that many developers work on their GitHub projects in their limited free time—she referred the audience to Michael Bromley's recent "Why I haven't fixed your issue yet" blog post. She proposed three strategies to make responding to issues less arduous and, therefore, reduce the amount of lag time experienced by users who open bug reports.

The first is GitHub's issue-template system. This is a new feature of the site, by which projects can create a file named ISSUE_TEMPLATE.md that will automatically be inserted into the new-issue form. Projects can use this to help ensure that bug reports are sufficiently detailed and include as much relevant information as possible. Similarly, the second suggestion is GitHub's saved-replies feature. This allows users to save text snippets that they can then insert into issue discussion threads, to (for example) ask for clarifications, request more information, or acknowledge a new issue.

The third suggestion is to have a dedicated bug-triage team. That is a hard approach for smaller projects, she admitted, because of the time it requires. But she encouraged even small projects to try it, as a part of the "support-driven development" strategy. And most developers begin by doing their own bug triage, she noted, only stopping when the number of issues becomes too large. At that point, she said, it is worth considering that "bug triager" can be an important job for new team members, and an opportunity for mentoring incoming contributors.

People too intimidated to try contact

The final group of people who contact GitHub with support requests about hosted projects is those who are either too embarrassed or too intimidated to try. Here again, Berry showed some anonymous quotes from such support requests. One noted that a developer "checked the code and added a commit, and is asking me to merge his sample code, which is what I don't understand." Another said they were interested in contributing to a project, however "I can't find the process to follow." In both cases, what the project is missing are instructions for the user to follow.

At the very least, she said, projects ought to have a CONTRIBUTING file in their repository that describes the procedures to follow when a bug is found, when the user wants to fix a bug or add a feature, and when the user wants to ask a question. If a file named CONTRIBUTING exists in the repository, GitHub will provide a link to it along with a short message at the top of each new-issue page. These procedures can also be documented on GitHub Pages and project wikis, she said. It can be time-consuming, she agreed, "but I guarantee it has returns for a long time to come."

It is also useful to tag issues with labels like "first-timers only" to encourage new contributors. Berry also recommended an "up-for-grabs" tag, since few things are as discouraging as starting work on a bug only to discover at the end that someone else has fixed it and the time was wasted. There are several initiatives promoting this sort of project-management approach, she said, like up-for-grabs.net and First Timers Only. Both are designed to help new contributors get the hang of the project's preferred workflow.

It is also helpful to give detailed feedback on pull requests—at least, to those users that really want it. She noted that it can burn out developers to invest time responding to a pull request and get nothing in reply, as Sindre Sorhus had said on Twitter. Berry suggested making a GitHub saved reply to send that could be used to test whether or not the user who made the pull request was interested in doing more. It could include a few quick notes and a detailed next step, then end with "I can help you if you want, but this could be time-consuming. Let me know if you want to proceed." She referred the audience to a issue at the ReactiveUI repository as a good example.

Ultimately, she said, the goal of this sort of interaction is to keep the new contributor coming back for more. That is a big problem space involving many factors—in essence, it is the focus of the entire community-management discipline. But making your project's community the best that it can be is a critical part of making your project the best it can be, she said. "There are people out there trying to participate, just waiting for you to meet them."