Emacs discusses web-based development workflows
Discussions on ways to "modernize" the Emacs editor have come up in various guises over the past few years. Changes of that nature tend to be somewhat contentious in the Emacs community, pitting the "old guard" that values the existing features (and keybindings) against those who argue for changes to make Emacs more approachable (and aesthetically pleasing) to newcomers. Those discussions tend toward mega-thread status, so it should be no surprise that a query about possibly moving Emacs development to a "forge" (e.g. GitHub or GitLab) got similar treatment. As always in Emacs-land, there are multiple facets to the discussion, including the desirability of moving away from an email-based workflow, accommodating younger, forge-centric developers without forcing existing developers into that model, and—naturally—licensing.
As a newcomer to the emacs-devel mailing list, Daniel Fleischer may not
have expected the voluminous response he got to an August 26
post
asking about the status of a "move to a new VC [version control] system,
e.g. Gitlab
". The somewhat provocative subject of the email, "Gitlab
Migration
", probably helped draw eyes (and responses) as well.
There are no current plans to make a migration of that sort, of course, and
a two-year-old feature
request at GitLab shows a "pretty daunting
" level of work
needed, Dmitry Gutov said.
Richard Stallman had a different
concern:
We used to recommend GitLab as better than GitHub (though only barely acceptable). However, GitLab has got worse, and we should stop recommending it at all.
He suggested that sourcehut or NotABug.org might be a better match from a licensing and philosophy perspective. Other than a request from Emacs co-maintainer Eli Zaretskii for an evaluation of NotABug.org, not much was heard about that development-hosting site in the thread; sourcehut, on the other hand, came up multiple times. Stallman said that any potential solution would need to run on GNU infrastructure, rather than as a hosted service; he was also worried about the free-software status of any JavaScript needed.
In the original post, Fleischer noted two benefits that he saw with a switch to some kind of forge. Because younger developers are more familiar with the forge-style workflow, providing that will lower the barrier to entry for new developers, he said. In addition, not scattering the different portions of the project among multiple systems makes it easier to work with the project:
Having the code + issues + discussions in the same place as opposed to now, where the code and discussions (lists) are in 3 different places (Savannah, Gnu mailing lists and Gnu bug tracker). With a modern VC system, one can jump easily between issues, discussions, code commits back and forth easily as opposed to now, where if it's a bug you can use its number to search lists and commits messages but if it's a discussion, it's not "connected" to anything.
He also noted that an email-based workflow should still be supported, so
that developers can use Emacs for all of their project
interaction, as they do now. Emacs co-maintainer Lars Ingebrigtsen called that "the
biggest hurdle
", noting that it is important not to risk losing the
current contributors; "Can you interact with
Gitlab via a mail-only system?
" While that question was not
directly answered in response to that, the GitLab feature request and other
discussion makes it clear that the answer is "no", at least for now, and it
is not clear that there is any real work going on to change that.
Web advantages?
Philip Kaludercic objected to the idea
that using the web for development tasks was actually any easier, which is
a common response from those who are used to email style. Similarly, he
said that while the various parts of the development process were in
separate places, they share an important characteristic: all are
email messages. He suggested that the "biases against mailing list
development might be illegitimate
", but Ingebrigtsen said that there is a
different dynamic at play:
It seems like it should be easier to just send a patch, but feedback we're getting shows that it's not for a number of developers. Many don't use mail at all for development, and all they're used to is the GitLab/Hub way of doing it.So it's easier for them -- it feels safe and familiar for them to do development by clicking around in a web browser.
On the other hand, Jim Porter explained
that as relatively new contributor, the feeling of intimidation with using
an email-based workflow is real, but it turns out not to
actually be that hard to adapt to it. He said
that maintaining a project using the mailing list might be different, "but that's
not my job, so it's not a problem
for me
". He did suggest some places where the documentation might
be improved for those who are used to the pull-request-style workflow and
plans to propose some documentation patches. But Tim Cross thinks that email is a
dead-end for younger developers:
I think virtually all developers are forced to suffer email, but a [growing] number don't use it. Often, all the discussions, notifications, comments etc are actually consumed via a mobile 'app'. For these users, logging into their inbox is frustrating and inconvenient because their inbox is full of pointless and old messages/notifications/alerts they have already seen/received via other channels. For these users, the primary reason they have an email address is to have something to put into the 'login' box for web services they use. Telling these users to use email to submit a patch is very similar to me being told when I started using email that I had to send in a hard copy via snail mail.
Fleischer concurred, noting that he is in his 30s (half of Cross's self-reported 60s) but only uses email for identity purposes and for receiving official documents:
I don't talk to family, friends or coworkers via mail. Personally, I think it's old, not secure or private by default, very inconsistent (HTML rendering is arbitrary vs. text, multiple MUA [mail user agent]) and just can't imagine using it as a software engineering tool.
The security of email, at least versus the app-centric alternatives, was
hotly contested by some. Zaretskii said that the
"security issues with email have been solved
long time ago
" and that the email-based workflow used where he
works allows his company to "run circles around those of our partners and
competitors
" who do not use it. Cross pointedly disagreed
about email security, however:
Despite what others have claimed, the security problems with email have NOT been addressed. It is still one of the major vectors for compromising access via social engineering, major vector for infection from [ransomware], frequent source of privacy [breaches] and a common means used to get sensitive data out of an organisation. Even having encrypted messages is overly complex for most users and a nightmare to administer for large organisations.
Stefan Monnier thought that the problem was less about email per se, but that mailing lists are intimidating:
I think the issue is not use of email as such, but use of mailing-lists. In my experience the reluctance to use email is that they feel uncomfortable sending email to a potentially large number of persons. In contrast posting on a forum doesn't impose anything on anyone (in their mind) because those who read it have to actively go and look for it.
[ Of course, it doesn't make much sense, really, but this is about people's perceptions, not about anything rational. ]
There are a number of problems with the email-based workflow that
forge-style development alleviates, Clément Pit-Claudel said.
One of those is that mistakes can be edited on a web site, which is not the
case for mailing list; some state tracking and maintainer tasks may be
simplified as well. But Zaretskii was quick to point out various
deficiencies he sees with the web interfaces. He does not think that adding
GitLab (or similar) will provide much to maintainers, it would simply"be more welcoming
to casual contributors
", which is worth doing, but the advantages
for existing Emacs developers are far less clear.
Whatever choices are made, Stallman is adamant
that no "online dis-services
" be recommended for doing Emacs
development. It is fine to have adjunct mechanisms, but email must remain
viable:
It's fine if users have the _option_ of communicating with the maintainers and developers via some interface other than email. (Or ten different interfaces other than email.) But when some of them use it. that must not compel the maintainers and developers to use it too.
Multiple workflows
There is no way to objectively determine which development style is easier,
Fleischer said, so
it is fruitless to discuss it. "I can say objectively that one form
of workflow is more popular than
another so people would be more familiar with it.
" The conversation
soon turned to supporting multiple workflow styles; since there had been
little progress with GitLab, sourcehut was discussed.
Two threads from 2020 were noted; one where sourcehut creator Drew DeVault
self-evaluated
the hosted service under the GNU ethical
repository criteria and another
where Emacs development using sourcehut (either as a service or by hosting
the code on GNU systems) was discussed. At the time of the latter thread,
email-based workflows were supported well, "but what about the GitLab/Github-like
features?
", Zaretskii asked. DeVault pointed to a video
showing "the sourcehut equivalent of the github/lab
pull/merge-request flow
".
That led to further discussion of sourcehut's capabilities with respect to
the needs of Emacs development. Gutov noted
a few different things he saw as "frictions/downsides
"
compared to "the Git**b workflow
". That discussion proceeded
to hash out some of those areas, with DeVault participating and being quite
interested in trying to bridge the gap between sourcehut's current features
and the needs of the Emacs project. In another sub-thread, he summed up the
current status:
First, we would be very pleased to host emacs on our service, or with our software on your own infrastructure. It should more-or-less work with everyone's existing workflows, given that emacs is an email-oriented project, and over time we are developing an improved web-based experience which should allow more and more users access to emacs development over time without necessarily requiring the maintainers or those who prefer their email-oriented workflow to have to change their workflow to [accommodate] a platform like GitLab. We should also rate quite highly in terms of free-as-in-freedom, since the entire service is free software, mostly AGPL.
He did note that the sourcehut bug tracker works differently than the
existing bug
tracker, which is based on an older version of the Debian Bug Tracking System (BTS).
But there are plans to make the sourcehut bug tracker "more
email-oriented
". Though the philosophies of sourcehut and GNU are
similar, Stallman would prefer not
to host Emacs on the service:
I think Drew DeVault is right in saying that his philosophy is close to ours. If we had to use some outside hosting service, I might well say there is no better choice than his. (I don't want to criticize the other options without knowing what they are; they may be equally good.) But we'd rather have this on a GNU machine with GNU Project people managing it.
The sourcehut service is not the only option, as DeVault said in an earlier message:
You can run the sourcehut software on GNU servers, it is 100% free software, mostly AGPL. I would encourage you to do so.
Gutov said, that adopting sourcehut would be a net win, but that there are still things lacking in sourcehut that will be difficult to add in the email realm:
For example, those quality-of-life features that Gitlab has in the browser which I previously figured would be difficult to translate to email (the code review workflow, with inline comments and updates from the branch; automatically updated CI indicators and links to builds; editing of messages) are predictably absent.
Efforts are being made to describe the kinds of web-based features that Emacs would want in sourcehut (or any other forge it might adopt) so that users of the web-based workflow would feel at home. Fleischer listed some of those types of features. But some commenters are still not entirely convinced that adding support for a web-based workflow will lead to more and better contributions. João Távora posted some thoughts on what he has observed:
In recent $DAYJOBs I worked with these two GL/GH platforms fully, using them liberally and without restrictions. In these recent experiences the undeniable contemporarity and newcomer friendliness of these platforms does NOT seem to translate into quality of code, quality of discussion or any kind of benefic developer agility in any way. Again, just anecdotal evidence which you may take for what it's worth, but in fact I believe that the "slow", unfamiliar, peculiar, old-school whatever-you-want-to-call-them methods used in Emacs development may in fact be "aces up our sleeve", not just a means to appease those that have been using them for a number of years.
On the flipside, though, Pit-Claudel saw the opposite in a project that he works on which switched to GitHub, but also cautioned that it is easy to give these kinds of anecdotes too much weight:
What this doesn't say is whether any "modern" workflow would have helped, or whether it was specifically Github, because of network effects (the barrier for contribution is lower if you already have an account and you are already familiar with the UI).For Monnier, the choice of tools is obvious at this point. Rather than continuing to look at other options, he would just like to get on with an evaluation of sourcehut:In fact, it doesn't even say whether it was the move itself that helped, or whether the move was an irrelevant manifestation of a more welcoming approach and a general effort to attract contributions.
Reading this discussion made me realize why I'm rooting for SourceHut: it's the only such tool I've seen whose goals align exactly with ours. Both philosophically (most other tools just happen to be Free Software and but are more philosophically aligned with Open Source) and technically (it aims for first class email support while providing as much as possible of the shiny web features of other systems).So at this stage, I personally don't see much point looking for other tools. Instead I'm just wondering how to get us to use SourceHut (i.e. how to resolve the problems that might stand in the way, such as getting an instance up and running on GNU machines, starting to use it "on the side" to see in practice what issues we'll need fixed/improved/added before we can really switch, etc...).
It is clear that any change—if one is made—will need to support the email-based workflow unchanged (or nearly so) from today's methods, but that adding other workflow styles might be a "nice to have". Stallman noted that there are both practical and moral considerations at play for keeping email as a first-class option:
You also need to have an internet connection at the moment that you try to use the web forum. Not so for sending email.[...] To make an account on a web forum, you have to use that platform in the way it is set up to require. You have to accept its terms and conditions, which may be morally unacceptable.
When you send email, you use your own choice of tools and platforms -- the same ones you used to send mail to a different forum the day before.
Network effects
Though some are enthusiastic about sourcehut getting to a point where it can satisfy both old- and new-school developers, it may still be too far afield for many of today's younger developers. The network effects of GitHub (and, to a lesser extent, GitLab) far outstrip those of sourcehut, and it may well be that users of those other forges are put off by whatever differences there are at sourcehut. As Ingebrigtsen put it:
Emacs will change to a new system when we can find a system that's good enough, and offers both an email work flow (for those that prefer that) as well as a non-email work flow (for those that prefer that).We haven't found such a system yet. sr.ht [sourcehut] is perhaps an option, but my first impression is that it's too different from GitHub/Lab, really.
In the end, it will probably come down to whether or not the Emacs developer community finds the energy to start working with sourcehut (or some alternative, though that seems far less likely at this point). There is clearly work needed even to get it to a point where the email-based workflow is fully supported (e.g. bug tracking); enough web features to bring in users of Git**b needs even more. It is not at all clear that doing that work will change the situation all that much, either; drive-by contributors, or even those who are more serious about doing Emacs development, may not be all that interested in signing up for yet-another web site and learning its interface and quirks.
Emacs is not alone in feeling that workflow modernization is needed for the future, of course. Python has largely already made the switch to GitHub, for example, and the Linux kernel is dipping its toes into that world as well. Other projects are likely struggling with it as well. To the extent that the future can be predicted with any accuracy, at this point one would have to guess that 20—or 30—years from now email-based workflows will be dead or nearly so. Navigating from now until then is going to be somewhat messy for many, generally older, projects.
