|
|
Subscribe / Log in / New account

A new free-software forge: sr.ht

By Jake Edge
January 8, 2019

Many projects have adopted the "GitHub style" of development over the last few years, though, of course, there are some high-profile exceptions that still use patches and mailing lists. Many projects are leery of putting all of their project metadata into a proprietary service, with limited means of usefully retrieving it should that be necessary, which is why GitLab (which is at least "open core") has been gaining some traction. A recently announced effort looks to kind of bridge the gap; Drew DeVault's sr.ht ("the hacker's forge") combines elements of both styles of development in a "100% free and open source software forge". It looks to be an ambitious project, but it may also suffer from a lack of "social network" effects, which is part of what sustains GitHub as the forge of choice today, it seems.

The announcement blog post is replete with superlatives about sr.ht, which is "pronounced 'sir hat', or any other way you want", but it is a bit unclear whether the project quite lives up to all of that. It combines many of the features seen at sites like GitHub and GitLab—Git hosting, bug tracking, continuous integration (CI), mailing list management, wikis—but does so in a way that "embraces and improves upon the email-based workflow favored by git itself, along with many of the more hacker-oriented projects around the net". The intent is that each of the separate services integrate well with both sr.ht and with the external ecosystem so that projects can use it piecemeal.

There are two sides to the sr.ht coin at this point; interested users can either host their own instance or use the hosted version. For now, the hosted version is free to use, since it is still "alpha", but eventually one will need to sign up for a plan, which range from $2 to $10 per month, to stay on the hosted service. There are instructions for getting sr.ht to run on other servers; it uses nginx, PostgreSQL, Redis, and Python 3 along with a mail server and a cron daemon.

While, overall, the documentation is rather terse and a bit scattered, it is not difficult to get started using the service by following the tutorial. Logging in allows one to create a Git repository; adding an SSH public key to the account then allows pushing an existing repository up to the system. From there, it can be browsed, as shown in the core sr.ht repository, cloned by others, and so on.

[sr.ht-dev mailing list]

As mentioned, sr.ht has not taken the approach of being yet another GitHub clone. Instead, it is geared toward a mailing-list-centric approach, possibly using the sr.ht mailing list component. The sr.ht-dev mailing list (seen at right) provides an example of the user interface for that component. Unlike some other forges or mailing-list replacements, it is not JavaScript-heavy—in fact, sr.ht uses no JavaScript at all, so pages are small (less than 10KB on average) and load quickly.

There is a guide to the preferred development and collaboration style for sr.ht. It is based around git send-email to a mailing list with copies to potential reviewers, much like Linux kernel development is done. Instead of forking a repository on the server, as is done for GitHub and others, a local clone is made, changes are made and committed, then posted for review. Once a patch is ready for merging, maintainers can apply it using git am. As can be seen, this is much different than the web-centric "pull request" model used by GitHub and others.

Wikis for sr.ht can be created using the man component. Wikis are simply a Git repository of Markdown files that get converted to HTML and served when they are visited. In addition, any sr.ht Git repository can have a top-level README.md, which will be shown when the repository is browsed and could provide a link to a project-specific wiki.

The build and CI component, builds.sr.ht, is what DeVault calls "the flagship product from sr.ht". His announcement notes that he has been working with both Linux and non-Linux (e.g. BSD, Hurd) distributions to have them start using it because "it's the only platform which can scale to the automation needs of an entire Linux distribution". He also says that smaller users are switching away from Travis CI and Jenkins to builds.sr.ht.

On builds.sr.ht, simple YAML-based build manifests, similar to those you see on other platforms, are used to describe your builds. You can submit these through the web, the API, or various integrations. Within seconds, a virtual machine is booted with KVM, your build environment is sent to it, and your scripts start running. A diverse set of base images are supported on a variety of architectures, soon to include the first hardware-backed RISC-V cycles available to the general public. builds.sr.ht is used to automate everything from the deployment of this Jekyll-based blog, testing GitHub pull requests for sway, building and testing packages for postmarketOS, and deploying complex applications like builds.sr.ht itself. Our base images build, test, and deploy themselves every day.

The build manifests specify more than just how to build the project, "test" tasks can be specified as well. The manifests also specify the platform (e.g. Alpine Linux, FreeBSD) that should be used for the build and test tasks. Build manifests can be placed in particular locations (.build.yml, .builds/*.yml) in a Git repository in order to run them automatically when new code is pushed to the repository. More information about builds.sr.ht can be found in the tutorial, manual, and API reference.

There is also a bug/issue tracking component called "todo". Its user manual is particularly brief as of this writing ("TODO: write these docs"). There are other places one will run into missing documentation pages, perhaps most critically for the code review page that is referred to in the lists.sr.ht documentation for those new to mailing lists. One would guess those holes will be filled in before too long.

The project is written in Python 3 and licensed under the Affero GPLv3. As noted, it is an ambitious project, but one has to wonder whether the mailing-list-centric workflow will survive long term. The instructions on how to set up mail clients and descriptions of proper mailing-list etiquette may not sit well with newer developers. Email is painful to set up and use any more—many are finding alternatives far more attractive.

Ultimately, what a project like sr.ht needs in order to fill out its feature base, grow, and thrive is new projects. There is an existing stable of projects that are run in a way that is compatible with sr.ht, but not very many new projects are going that route—for good or ill. In addition, the social effects of GitHub (and, to a lesser extent, GitLab, at least in the free-software world) are a major piece of what makes that model so successful; it is hard to see sr.ht replicating that to any significant degree. It is an interesting project, though, and one that deserves well-wishes; for compatible projects looking for a home, it is certainly worth a look.


Index entries for this article
PythonApplications


to post comments

A new free-software forge: sr.ht

Posted Jan 8, 2019 22:52 UTC (Tue) by brouhaha (subscriber, #1698) [Link] (9 responses)

I haven't used sr.ht yet, but being written in Python 3 makes it a good deal more attractive to me than the alternatives I'm aware of. I don't know much PHP, Ruby, or Go, and what I do know of PHP turns my stomach. Ruby and Go are probably fine languages, but I don't have time to learn them.

A new free-software forge: sr.ht

Posted Jan 9, 2019 1:28 UTC (Wed) by vstinner (subscriber, #42675) [Link]

There is Pagure which is written in Python as well: https://pagure.io/pagure It's used by Fedora for example.

A new free-software forge: sr.ht

Posted Jan 9, 2019 15:37 UTC (Wed) by claude.bing (subscriber, #127877) [Link] (7 responses)

As long as the service works as advertised, why does it matter what language it is written in?

A new free-software forge: sr.ht

Posted Jan 9, 2019 16:28 UTC (Wed) by lkundrak (subscriber, #43452) [Link] (6 responses)

This matters *a lot* to people who are going to operate servers that run the software.

* Am I able to understand the code?
* Will I be able to rebuild the thing?
* Is the certain class of bugs likely to be present? (hello PHP!)
* Am I able to reasonably and safely install the correct versions of the required dependencies? Will I be able to do so next week? (hello nodejs!)
* Will it run with the next year's version of the interpreter? (hello Python)
* What is the resource consumption? (hello Java)

A new free-software forge: sr.ht

Posted Jan 9, 2019 17:00 UTC (Wed) by claude.bing (subscriber, #127877) [Link] (1 responses)

Ah, from the self-hosted approach I completely agree. I was referring to the SaaS sr.ht site itself, sorry for the misunderstanding.

A new free-software forge: sr.ht

Posted Jan 9, 2019 17:20 UTC (Wed) by k8to (guest, #15413) [Link]

Well if the attraction is the open source aspect, then these concerns will have an influence even if you plan to use the service for now.

A new free-software forge: sr.ht

Posted Jan 10, 2019 3:54 UTC (Thu) by jkingweb (subscriber, #113039) [Link] (3 responses)

> Is the certain class of bugs likely to be present? (hello PHP!)

What class of bugs would that be, out of curiosity?

A new free-software forge: sr.ht

Posted Jan 10, 2019 7:41 UTC (Thu) by lkundrak (subscriber, #43452) [Link]

I had this on my mind: http://php.net/manual/en/security.globals.php
Likely not relevant for present-day PHP which I don't know much about

Then there's other such classes; i.e. you probably have somewhat higher chance of messing up handling of files with whitespace characters or begining with an "-" when writing a shell script.

A new free-software forge: sr.ht

Posted Jan 10, 2019 14:20 UTC (Thu) by domenpk (guest, #12382) [Link] (1 responses)

Very liberal type conversion can be responsible for some of those bugs. See https://www.cryptologie.net/article/268/how-to-compare-pa... for example.

A new free-software forge: sr.ht

Posted Jan 11, 2019 15:47 UTC (Fri) by jkingweb (subscriber, #113039) [Link]

I had been under the mistaken impression that such string insanity had ended, but apparently only some forms have to date been stamped out; notably 0eAnything is still alive and well, it seems. :(

A new free-software forge: sr.ht

Posted Jan 8, 2019 23:00 UTC (Tue) by ddevault (subscriber, #99589) [Link] (2 responses)

Thank you for the write-up, Jake! If any LWN subscribers have comments/quesitons/feedback about sr.ht, I'm always a short email away: sir@cmpwn.com

Cheers!

A new free-software forge: sr.ht

Posted Jan 9, 2019 15:39 UTC (Wed) by ejr (subscriber, #51652) [Link] (1 responses)

Oh, email centric, I like... Any plans for patchwork or something equivalent? (I see the "Write this doc" under code review.)

A new free-software forge: sr.ht

Posted Jan 9, 2019 15:43 UTC (Wed) by ddevault (subscriber, #99589) [Link]

Yep! You can see the beginnings of this today, here's an example mailing list where patches are coming in often:

https://lists.sr.ht/~emersion/mrsh-dev

The plan is to extend this to support a web-based review style similar to Gerrit, but where comments on the web become emails in the typical mailing-list-code-review fashion popular in other projects - and vice versa, participating via email shows up on the web.

A new free-software forge: sr.ht

Posted Jan 8, 2019 23:14 UTC (Tue) by roc (subscriber, #30627) [Link] (8 responses)

The announcement has a quote describing Gitlab as a "Github wannabe", which is very unfair. Gitlab is well ahead of Github in some areas, e.g. CI/CD; Github simply doesn't have anything built in for that.

A new free-software forge: sr.ht

Posted Jan 9, 2019 0:06 UTC (Wed) by q3cpma (subscriber, #120859) [Link] (3 responses)

It may be ahead, but it's quite behind when it comes to basic functionalities without Javascript enabled, sadly.

A new free-software forge: sr.ht

Posted Jan 9, 2019 2:32 UTC (Wed) by roc (subscriber, #30627) [Link] (2 responses)

If I was building a Web client I wouldn't bother making it work at all for the negligible number of users who would insist on disabling JS on my site. Any effort spent on that would give a bigger payoff by spending it on something useful for the vast majority of JS-enabled users.

A new free-software forge: sr.ht

Posted Jan 9, 2019 2:34 UTC (Wed) by roc (subscriber, #30627) [Link]

Especially if it's open source so that ornery minority can maintain their own JS-free fork if they really want to.

A new free-software forge: sr.ht

Posted Jan 17, 2019 14:07 UTC (Thu) by oldtomas (guest, #72579) [Link]

This is hardly surprising from you, knowing your background.

See, I have a love-hate relationship with Mozilla, and this post from you clearly falls in the "hate" part.

It is jarring to be told "you don't exist" -- especially by those who want to be one's ally.

A new free-software forge: sr.ht

Posted Jan 9, 2019 14:13 UTC (Wed) by jpsamaroo (subscriber, #129727) [Link] (3 responses)

While I agree that "Github wannabe" is a bit strong, as a user of Gitlab, I definitely can see it as an more featureful clone of Github. Many of the features of Github are replicated in Gitlab, just with added functionality and a different name.

I think the most important thing to consider is that unlike sr.ht and related software forges/methodologies, Github and Gitlab are solely focused on the Web UI model, and so end up reinventing a lot of functionality that either already exists in a git-send-email world, or works around issues caused by not using email as a collaboration platform.

A new free-software forge: sr.ht

Posted Jan 10, 2019 5:30 UTC (Thu) by marcH (subscriber, #57642) [Link] (1 responses)

> Github and Gitlab are solely focused on the Web UI model,

I'm always surprised how rarely Gerrit is mentioned in these discussions. It's more comparable to email because it doesn't "confuse" code review with bug and release tracking, CI etc. It's far from perfect but open-source, actively developed and has been used in production by some really big projects for years. It's fairly scriptable and sends (too many and very poor) emails.

> and so end up reinventing a lot of functionality that either already exists in a git-send-email world, or works around issues caused by not using email as a collaboration platform.

Wow, I've seen a lot of discussions on this topic but I think it's the first time I see this "email has more features" angle, it's rather the opposite as in email keeps things simple, flexible and not "bloated" etc. Can you list some of these features?

Hopefully you didn't just mean: "email can be scripted hence it can do anything".

A new free-software forge: sr.ht

Posted Jan 11, 2019 4:09 UTC (Fri) by begriffs (guest, #118062) [Link]

> I think it's the first time I see this "email has more features" angle, it's rather the opposite as in email keeps things simple, flexible and not "bloated" etc. Can you list some of these features?

I think that the email collaboration style of development does offer features that Github-style development lacks:

https://begriffs.com/posts/2018-06-05-mailing-list-vs-git...

But, as mentioned in this thread, Github has some advantages too.

A new free-software forge: sr.ht

Posted Jan 11, 2019 15:44 UTC (Fri) by BenHutchings (subscriber, #37955) [Link]

... Github and Gitlab are solely focused on the Web UI model, ...

Primarily focused on web UI, perhaps, but both have APIs that can be used to script many operations.

A new free-software forge: sr.ht

Posted Jan 9, 2019 0:00 UTC (Wed) by awilfox (guest, #124923) [Link] (11 responses)

> Email is painful to set up and use any more—many are finding alternatives far more attractive.

This is an outright falsehood. Email is not painful to set up, and it is miles easier to use than the webby JavaScript slow-as-hell BS GitHub and, to a lesser extent, modern GitLab are full of.

To hell with the webby BS. I don't think sr.ht is the project that will fix code forges, but it definitely has the right ideas at heart.

A new free-software forge: sr.ht

Posted Jan 9, 2019 0:28 UTC (Wed) by mageta (subscriber, #89696) [Link] (4 responses)

You should at least try to get some form of coherent argument, instead of just calling it names.

A new free-software forge: sr.ht

Posted Jan 9, 2019 8:29 UTC (Wed) by Archimedes (subscriber, #125143) [Link] (3 responses)

My (personal) main argument in the comparison of email vs web based system would be:
I can choose my email client, I am allowed to configure that client as I wish, I can enhance that client (depending on it) in plenty of ways which help me to work as I would like to, and the great thing is that all what I like on my setup does not influence anybody else.

On Web based systems in all (or maybe most) the user has no real way to change the "websie client" as he would like to use it. Also the workflow, shorcuts, ui, whatever is given by solely the web client. For me that feels like the mainframe times, root == god, user == nothing+epsilon ...

A new free-software forge: sr.ht

Posted Jan 9, 2019 21:10 UTC (Wed) by roc (subscriber, #30627) [Link] (1 responses)

Actually there's quite a lot one can do to customize Github and Gitlab. Both of them have APIs that let you write applications that read/write data and customize workflow. On the UI side, browser extensions let you customize look and feel.

A new free-software forge: sr.ht

Posted Jan 10, 2019 17:57 UTC (Thu) by ddevault (subscriber, #99589) [Link]

An important difference is that email is governed by standards and enjoys a lush ecosystem of open source implementations.

A new free-software forge: sr.ht

Posted Jan 10, 2019 23:11 UTC (Thu) by neilbrown (subscriber, #359) [Link]

> My (personal) main argument in the comparison of email vs web based system would be:

Mine would be "federation". By this, I particularly mean a global name-space for contributors, together with (at least) minimal interaction possibilities between all platforms.

If I want to add "Reported-by:" to a patch that I generate, based on a report that I received on some weird platform (case-in-point: github issues), the name attached to the problem report should be a universal name that I can use in the Reported-by tag.

If I want to include someone in a conversation - I should be able to request that within the platform and they should be able to receive my request, review the history, and make a comment without having to sign-in or anything - just follow a link to read, and then reply to the original message to contribute.

A new free-software forge: sr.ht

Posted Jan 9, 2019 8:38 UTC (Wed) by smurf (subscriber, #17840) [Link]

Yes, email is painful. Just try to get outgoing SMTP to work when you're behind a restrictive ISP. Blame the spam mail viruses …

A new free-software forge: sr.ht

Posted Jan 9, 2019 18:21 UTC (Wed) by Beolach (guest, #77384) [Link] (4 responses)

"De gustibus non est disputandum" - "In matters of taste, there can be no dispute."

I consider web interface vs. email vs. whatever else to be a matter of taste. Different people will prefer different things. My personal taste in this matter aligns more w/ yours (I dislike "webby JavaScript" & prefer email, or at least non-JS plain HTML web), but I readily acknowledge the existence of people with other tastes, and that it could be nice to be able to collaborate w/ them w/out having to sacrifice either of our preferences.

IMO that's the biggest issue for sr.ht - that will be the most difficult to get right, and the most important thing that will make them succeed over their competition if they can get it right - is bridging between web & email & git CLI & whatever else people like to use. Upthread SirCmpwn said:

The plan is to extend this to support a web-based review style similar to Gerrit, but where comments on the web become emails in the typical mailing-list-code-review fashion popular in other projects - and vice versa, participating via email shows up on the web.

If they can get that right, that's a killer feature. But I think it's hard to do right - at least, nothing I know of has succeeded at it yet.

A new free-software forge: sr.ht

Posted Jan 10, 2019 5:45 UTC (Thu) by marcH (subscriber, #57642) [Link]

Databases with wildly different (web *and* non-web) interfaces and representation is a long solved problem. The only issue is parsing email's DATA lack of structure which must be why patchwork's database doesn't even try.

A new free-software forge: sr.ht

Posted Jan 10, 2019 9:39 UTC (Thu) by grawity (subscriber, #80596) [Link] (1 responses)

Additionally, a matter of experience influencing how "easy" something seems to be.

I'd agree that setting up `git send-email` is not very difficult at all – point it at the SMTP server and go. (Once you know that you need to use `git send-email` and not e.g. Thunderbird, that is.) But setting up a suitable mail client for receiving patches is a bit more of a hassle – it seems like you have to either use Mutt or Emacs, basically.

I really hope that OP didn't mean to say that setting up a mail server is easy; I've set up mine twice, and even though I enjoyed the process, I wouldn't call it any easier than e.g. setting up Kerberos or BGP: it seems to work at first, but there are pitfalls everywhere you go, your mail gets dropped for reasons you can't control, and if you don't faithfully follow the correct tutorial word-by-word, it'll take weeks if not months to work the kinks out.

A new free-software forge: sr.ht

Posted Jan 25, 2019 14:57 UTC (Fri) by aigarius (subscriber, #7329) [Link]

Locations also differ. Try setting up git send-email when your ISP simply blocks all incoming and outgoing connections to all known email ports. For security. And the selection of alternative ISPs is limited or even non-existent. Email is a really complex system to set up on your own, so most people just use GMail or something similar, so you are effectively trading a WebUI that is optimized for code review for a WebUI that is not.

A new free-software forge: sr.ht

Posted Jan 10, 2019 11:16 UTC (Thu) by jani (subscriber, #74547) [Link]

Personally I've come to the conclusion that while I very much prefer emailed patches for code review, it's not necessarily the best mode of transmission of the actual code changes. I think it would be an interesting idea to git push the changes to a repo, and have the server side git send-email the patches to the right mailing lists and people. This would reduce the patchwork style guesswork on the mailing list, as the server would have more data available to it.

A new free-software forge: sr.ht

Posted Jan 9, 2019 3:26 UTC (Wed) by unixbhaskar (guest, #44758) [Link] (2 responses)

Well, bumped on it. PGP key upload always said it's too big, In spite of using their method to get the minimal key and tried to upload the same ...still the result is same. I do hope someone might look into that.

A new free-software forge: sr.ht

Posted Jan 9, 2019 7:51 UTC (Wed) by Shugyousha (subscriber, #93672) [Link]

Sounds like a bug. Maybe you can report the issue you had here?: https://todo.sr.ht/%7Esircmpwn/git.sr.ht

A new free-software forge: sr.ht

Posted Jan 15, 2019 3:17 UTC (Tue) by aaronmdjones (subscriber, #119973) [Link]

It may be because, as in my case, you have a photo UID on the key. This can be many kilobytes; in my case it results in a key more than 600 KiB. You can temporarily delete photo UIDs and export it, and then refresh your key from a keyserver to get your UIDs and their signatures back. After all, encrypting email to you doesn't need photos anyway. For some unknown reason, '--export-options export-minimal' doesn't remove photos.

$ gpg --edit-key ...
(gpg) uid 3
(gpg) deluid
(gpg) save
$ gpg --export ...
$ gpg --refresh-keys ...

Comparison of code development service platforms and services?

Posted Jan 9, 2019 11:46 UTC (Wed) by ber (subscriber, #2142) [Link]

It is good to see more efforts to improve Free Software products and services build upon them, so thanks for reporting about it.

What I'm missing from the announcement and the article is an attempt at comparing the different options, for instance there is allura.apache.org (and their first link is at least a start of an comparison).

There are also many other products and service providers like Phabricator / Phacility and others already mentioned.

Regards, Bernhard

A new free-software forge: sr.ht

Posted Jan 9, 2019 20:58 UTC (Wed) by iabervon (subscriber, #722) [Link]

It would be an interesting next step to offer some email-client-inspired web interfaces to archives of the mailing list; having an email-style development method doesn't has to mean that programs have to use SMTP and POP/IMAP. I see this style as being fundamentally that development is represented with a sequence of human-readable messages which tend to be a couple thousand words each, where it is possible to search the database across all threads associated with a single project (i.e., see all new messages to a mailing list), rather than having to look at a number of separate places to see whether each of them has new messages, and you can see a list of results with one line per message when the message is much longer than that in full.

Being able to insert and receive messages by SMTP is useful for a lot of people's existing tooling, but I think the more important aspect of this project style is that model of what an interaction consists of, and I think it would be valuable to offer that to people who don't prefer to use SMTP for development.

A new free-software forge: sr.ht

Posted Jan 10, 2019 5:34 UTC (Thu) by marcH (subscriber, #57642) [Link]

For people who can't get enough of these debates there was also:
https://lwn.net/Articles/768483/ A farewell to email October 16, 2018


Copyright © 2019, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds