Rethinking python-ideas

The communication style of a project can often have an effect—for good or ill—on its ability to attract new developers. The established channels, which are often email-based, may work reasonably well for the existing developers, but frustrate new people. Finding the right balance can be difficult, which is something the Python community is currently discussing.

The specific channel under discussion is the python-ideas mailing list, which is meant for more speculative discussions of potential Python features. On that list, Michael Selik wondered if some forum other than a mailing list would help new people figure out which opinions were more valuable than others: "One defect of a mailing list is the difficulty of viewing a weighted average of opinions". He likened the situation to the US Senate, where each state has the same representation even though they may have vastly different populations.

Selik suggested that perhaps Stack Overflow might be a better tool for forums like python-ideas. That short post was a springboard for considering the tradeoffs of different communication mechanisms, with an eye toward alternatives for some Python development discussions. Nick Coghlan described the feature discussion process, which can take place in a variety of forums, including the bug tracker, mailing lists, and the Python Enhancement Proposal (PEP) process. How the right forum is determined is more of an art than a science, he said; the core developers help guide that. The mailing lists definitely play a role:

That's not to say that there might not be better ways to host those discussions, Coghlan said in response to a "vote" for StackExchange (which is the parent of Stack Overflow). The limiting factor is typically the ability of the Python Software Foundation (PSF) to add and manage new infrastructure.

Selik summarized some of the problems he sees with a list like python-ideas. In particular, it is hard for a new participant to recognize who are the decision makers, whose opinion is not particularly well-regarded, what the real consensus is, and so on. These are difficult to discern from a mailing list, but might be easier to pick up on using a discussion forum with a reputation system or voting.

There is some attraction to a forum like Stack Exchange, which Guido van Rossum summed up in a message supporting an experiment to augment python-ideas:

Van Rossum noted that there is at least one downside: StackExchange is not particularly usable for those with an email-centric workflow. Some questions were raised about the fit for StackExchange's "question and answer" format and the kinds of discussions that go on in python-ideas. Brett Cannon suggested some other possibilities:

It turns out that, for those who were posting in the thread, the voting is not a particularly popular feature from StackExchange. What Van Rossum is looking for is the standard URL that gets created for a discussion that can be referenced later; he also thinks the format will help new participants:

The conversation soon turned toward Discourse and, to a lesser extent, HyperKitty (which is part of the Mailman 3 suite). Those seem to have many of the features of interest without completely leaving email users behind. But some were unhappy that there was any thought of moving to some other communication mechanism. Cannon said that it is important to do so from time to time:

He noted that the tools used by longtime Python developers (NNTP, newsgroups, even IRC) may be completely unknown by those university students. But the community needs to "evaluate whether our ways of communicating are too antiquated for new participants in open source and whether they are no longer the most effective (because old does not mean bad, but it does not mean better either), while balancing it with not having constant churn or inadvertently making things worse". It is a balancing act, but the way forward is to try remove the emotional component from the discussion, he said:

That technique could—perhaps should—be applied more widely in open-source project discussions. However, one of the main friction points between email and web-based discussion forums soon reared its head. Marc-Andre Lemburg noted that he had needed to reformat a reply because the original was in HTML, rather than plain text. To Cannon, though, that was "an example of the OSS community trying to swim against the stream where the rest of the world has moved on and it is slowly making it harder to get new people to participate in OSS".

Lemburg sees things a bit differently, noting that archivers and text-only email readers have to do an imperfect conversion of HTML, which can lose formatting information. He also pointed out: "Email is not going to go away, but I certainly have seen a lot of other communication tools come and go - and I bet there's something to learn in that :-)".

As part of that same exchange, Cannon and Lemburg also discussed the Discourse/HyperKitty question. Cannon suggested that someone that wants to stay on the email side look into Discourse to see where it falls short; he would also like to hear something about what HyperKitty offers. Lemburg said that he hasn't "really seen a compelling argument for switching away from mailing lists completely yet". But that can only come from "an evaluation to properly show what we would gain or lose from a switch to something like Discourse or HyperKitty", Cannon said.

As Donald Stufft pointed out, though, there is a risk in only looking at what new contributors might want. He does think that the "mailing list interface is kind of crummy and relies on people hand tailoring a bunch of filters to make it in any way reasonable", but is concerned that Discourse or HyperKitty may just be "differently bad". Furthermore:

There is also a risk of "community fracture" when changing the discussion mechanism, as Barry Warsaw described:

For a core developer, there are already multiple forums to follow to keep up with the development; adding another just makes that problem worse. "How many different channels do I have to engage with to keep track of what's happening in core Python?", Warsaw asked. He is in favor of experimentation, but feels that the project should proceed somewhat cautiously.

The split into various forums is going on now and will continue, but major decisions still need to go through python-dev, Van Rossum said. But python-ideas can be "too noisy", which has led some core developers to stop following it. Some other, less noisy discussion mechanism (or one that allows some topics to be muted more easily) might help.

Discourse appears to come the closest to having the features that would fit the bill—and Van Rossum clearly liked Discourse—though its email integration is not quite there yet for the email-only users. HyperKitty has some of those features on its radar, but has not had the time to implement them. There looks to be some momentum toward an experiment with Discourse, but that will require someone to manage the effort, perhaps to request some funding or assistance from the PSF, and so on. We shall see if that materializes.

There are some free-software projects that still communicate the same way projects have been doing so for decades: plain-text email, perhaps augmented by IRC. Some have branched out and added bug-tracker discussions and web-based forums of various sorts. Finding the right mix to keep existing contributors happy, while attracting new ones, is a delicate matter. We will likely see it play out in other communities as well. There is no "one size fits all" solution here, but seeing what works and doesn't—and why—will be beneficial to others. That is certainly one of the advantages of the free-software ecosystem.