LWN: Comments on "Fedora gathering requirements for a Git forge"

Fedora gathering requirements for a Git forge

notriddle — Sat, 08 Feb 2020 01:41:46 +0000

> Not in my evaluation. Its a known target of spambots and the recommended solution of upstream is to run nonfree software to stop them.

Fedora already uses Wordpress (https://fedoramagazine.org/ source code mentions the Yoast SEO plugin in its source code, which is a Wordpress plugin) and MediaWiki (https://fedoraproject.org/w/index.php?title=Fedora_Projec... is extremely familiar). You are not going to tell me that GitLab has a worse spam problem than those things do.

These apps already provide reCAPTCHA plugins. Fedora, instead, uses this thing <https://admin.fedoraproject.org/accounts/user/new>. They already said that if they adopted GitLab, they'd tie it into their existing SSO service, so it'll be the same thing here.

> As far as I can tell, the free users are treated as beta testers, their "complete solution" is clearly documented and advertised, including in the salsa.debian.org ui: give up your freedom.

If Pagure reaches significant adoption, it'll be targeted by spam bots anyway. Building non-reCAPTCHA spam filters on top of GitLab still sounds a lot easier than building non-reCAPTCHA spam filter *plus the entire rest of the Git Forge functionality*.

Unfortunately, comfort seems to win

christian_couder — Sun, 02 Feb 2020 08:23:21 +0000

(GitLab employee working on Git itself here.)
GitLab has on average 185 contributions from the wider community per month growing 128% per year, see: https://about.gitlab.com/community/contribute/

Unfortunately, comfort seems to win

christian_couder — Sun, 02 Feb 2020 07:39:41 +0000

(GitLab employee working on Git itself here.)
GitLab takes security quite seriously.
There is now an application security team with 6 persons (and a vacancy, see https://about.gitlab.com/company/team/org-chart/). Their work is not limited to GitLab itself but all the stack, see for example how Joern Schneeweisz of GitLab was credited in the latest Git security release:
https://lore.kernel.org/git/xmqqr21cqcn9.fsf@gitster-ct.c...
There is also a bounty program, see: https://about.gitlab.com/blog/2020/01/14/celebrating-one-...

Fedora gathering requirements for a Git forge

jzb — Sun, 02 Feb 2020 01:32:58 +0000

If my memory serves, Canonical slow-walked releasing Launchpad in a state that it could be stood up by other communities -- preferring to try to get major projects to host their projects on Launchpad to make it the popular watering hole for FOSS projects and trying to find a revenue stream. It was, ahem, launched in 2004, and source wasn't released until 2009. (See one of Shuttleworth's responses to criticism about lack of source.)

The other problem being, unless something has changed, that Canonical requires use of the so-called "Harmony" CLA to contribute to its projects. It's highly unlikely that many folks in the Fedora community are going to be eager to adopt Launchpad as long as that would require signing the Harmony CLA to contribute upstream.

Fedora gathering requirements for a Git forge

m4rtink — Fri, 31 Jan 2020 12:42:59 +0000

But can you actually deploy it yourself ? IIRC, last time someone tried to self host Launchpad, they found that the graphical theme of the web interface had some weird licensing and there were no public deployment scripts (paraphrasing what I have been told quite a while ago from memory).

I guess things could have been improved since then?

Fedora gathering requirements for a Git forge

raof — Fri, 31 Jan 2020 03:25:01 +0000

Launchpad!.

Only partially in jest. It is a git forge (along with bzr, should you need that ;)), and it (unsurprisingly!) has a whole bunch of features which are useful for distros' specific use-cases.

Unfortunately, comfort seems to win

ballombe — Thu, 30 Jan 2020 23:41:35 +0000

> There weirdly was no widespread argument against Debian adapting GitLab CE for their self-hosted repositories, despite the lack of the #1 communication platform (mailing lists).

It was also sad to see the 'download page' disappear.
Bye bye tarballs that does not require autoconf/automake to build.
Bye bye signed tarballs.

Fedora gathering requirements for a Git forge

IanKelling — Thu, 30 Jan 2020 19:21:13 +0000

> The open core nature of GitLab hasn't been a problem in Debian since we only use the free bits anyway

There's a lot of problems, and I bet some Debian developers are running nonfree bits. There's a ton of ways admins and users and use optional nonfree dependencies / "integrations", just a normal part of the interface tacitly encouraging users to run them. Want to contribute upstream? Good chance you need to run a nonfree recaptcha program on your computer. A few more problems: https://libreplanet.org/wiki/Fsf_2019_forge_evaluation

> it's a complete solution on its own without needing any of the non-free bits to function

Not in my evaluation. Its a known target of spambots and the recommended solution of upstream is to run nonfree software to stop them. When you say it works "on it's own", its arguably misleading people into giving up their privacy, because salsa.debian.org sent my private web browsing information to gravatar without my consent. As far as I can tell, the free users are treated as beta testers, their "complete solution" is clearly documented and advertised, including in the salsa.debian.org ui: give up your freedom.

Unfortunately, comfort seems to win

mirabilos — Thu, 30 Jan 2020 18:47:20 +0000

Perhaps not Alioth as it was, but FusionForge was still superiour to all the other choices and is changeable.

Ever tried to submit a change to an open core project? Haha… (even if the paid users of it also want it, and it’s trivial, it won’t end up even in the paid version…)

Fedora gathering requirements for a Git forge

m4rtink — Thu, 30 Jan 2020 17:29:44 +0000

The Fedora mailing list discussion mentions[0] that GitLab actually moved some features from the Enterprise Edition to the Community Edition to make it more suitable the Gnome Project, which did migrate to it in the end. While that's certainly nice of GitLab, I wonder how this would go with a smaller project needing GitLab EE features.

Also mentioned in the Fedora ML discussion, full text search is apparently a Gitlab EE exclusive feature[1].

I wonder if the Debian project also had to solve some of the CE vs EE edition issue ? Did GitLab also move some features from EE for Debian & what about the missing full text search ?

[0] https://lists.fedorahosted.org/archives/list/devel@lists....
[1] https://lists.fedorahosted.org/archives/list/devel@lists....

Fedora gathering requirements for a Git forge

pj — Thu, 30 Jan 2020 15:39:53 +0000

There's an Apache-Licensed github clone called 'gitbucket' available at https://github.com/gitbucket/gitbucket

Fedora gathering requirements for a Git forge

Felix — Thu, 30 Jan 2020 14:25:08 +0000

Could you comment on the resource usage? A Fedora contributor mentioned that pagure uses much less resources than gitlab so we would need to use likely 3-5x the resources (compared to pagure).

The main issue seems to be if Fedora can integrate gitlab with their custom infra tooling (e.g. Fedora Account System, Koji/build infrastructure). Is there some kind of overview which challenges Debian faced and how these were resolved?

I think Debian uses gitlab to version all control files (not sure about the Debian terminology, the packaging "recipes"/spec files). Any issues doing per-branch or per-repo permissions (e.g. we have maintainers per package and each package is a separate repo)?

Unfortunately, comfort seems to win

Conan_Kudo — Thu, 30 Jan 2020 14:03:41 +0000

The irony is that this is happening in large part because the Pagure community has been slow to grow. We would likely not be having this discussion at all if other distributions had started adopting Pagure, using it, and contributing to its development.

Fortunately, most of the Fedora community does not want to see Pagure replaced, and would rather see it modestly improved for supporting their workflows, so I foresee a formal assent to allocate resources to Pagure by Red Hat's CPE team.

Part of this is coming from the fact that the CPE team "maintains" over 100 codebases comprising of hundreds of applications and web services. There have been inquiries for *every* application in a similar manner with the exception of three: Koji, Bodhi, and MirrorManager. Koji is maintained by another team, Bodhi is considered critical and irreplaceable, and there are no maintained alternatives to MirrorManager to consider (MirrorBrain stopped being developed several years ago).

The bigger surprise here was this "Open Decision Framework" that nobody knew about before now. This wasn't mentioned in previous inquiries, which has made the conversation very strange.

Unfortunately, comfort seems to win

hmh — Thu, 30 Jan 2020 11:15:25 +0000

Well, supply chain attacks are no joke, and in active use by all sort of Actors.

GitLab is Hell to keep tight control on the supply chain, hopefully gitlab-the-company has a team dedicated to this that is forbidden to do anything else (you need to veto every changed line on every dependency, test suits are useless for this).

Hopefully Pagure is less insane on that area?

This was less of a problem when people used forges as a read-only platform as long as the code goes. But the moment you have pull request merging, it becomes trivial to hide undue changes where not many would look.

And one better really look into how the verified commit thing was implemented before trusting that thing, too.

I can use gitlab on Debian and gitlab.com safely from that kind of nastiness because I either never trust a merge or never use it as anything but a read-only depot and communication platform. And I verify all signatures locally after a clone or pull. But I can't really expect others will always do the same, neither at work nor in Debian. Can Fedora?

Debian had little choice but take that risk, but Fedora seems happy enough with Pagure right now, so they do have a second choice.

Obviously, a supply chain attack could always go after the forge-using developer's browser and local machine to try to steal keys and passwords, instead. But I expect that is a risk most of us is exposed to anyway.

And maybe the gitlab CTO will read this and tell us they do have a supply chain security team that cannot be diverted from its true purpose AND have absolute veto power on the supply chain, so you can trust they have enough human power to do their job well at all times, even when it gets in the way of "getting things done" of other teams... It would be really nice, but sounds like a hopeless dream.

Unfortunately, comfort seems to win

olasd — Thu, 30 Jan 2020 11:00:40 +0000

salsa.debian.org doesn't use the omnibus packages provided by upstream, they use a git checkout of (a light fork of) the upstream repositories.

All deployment manifests are public and available on https://salsa.debian.org/salsa (ansible: https://salsa.debian.org/salsa/salsa-ansible, terraform: https://salsa.debian.org/salsa/salsa-terraform, salsa's GitLab "fork": https://salsa.debian.org/salsa/gitlab) if you want to double-check this assertion.

Unfortunately, comfort seems to win

nodens — Thu, 30 Jan 2020 10:51:55 +0000

> Shockingly, they didn’t even use the packaged version (which, it turns out, is totally shit anyway) but the pre-made “packages” from the GitLab company.

No, they don't. They use a local fork (no modification AFAIK) and build it from source, as you can see here: https://salsa.debian.org/salsa/salsa-ansible.

Fedora gathering requirements for a Git forge

highvoltage — Thu, 30 Jan 2020 10:32:47 +0000

The open core nature of GitLab hasn't been a problem in Debian since we only use the free bits anyway, and it's a complete solution on its own without needing any of the non-free bits to function. If that was not the case then it wouldn't have been appropriate, but the GitLab instance hosted on salsa.debian.net proved to provide even more functionality that was initially anticipated as useful (the CI runners proved so popular that the team behind it now has to figure out how to make that scale out properly). I don't think that GitLab would be a bad choice for Fedora/CentOS, it's quite possibly the best choice they could go with.

Unfortunately, comfort seems to win

lobachevsky — Thu, 30 Jan 2020 10:17:04 +0000

I think that change was a good thing. Debian's systems can sometimes seem arcane and the previous cacophony where most everything was hosted somewhere else and in a different VCS was exhausting. With salsa.debian.org I don't have to think about Debian specifics and can find my way around more easily. It greatly increased the times I look into the innards of specific packages and I think this would be a benefit for Fedora as well. As terrible as the Github-monoculture is, the clone that is Gitlab at least lets me look at stuff in a way that doesn't feel foreign, greatly lowering the bar compared to Pagure.

Unfortunately, comfort seems to win

philh — Thu, 30 Jan 2020 09:23:06 +0000

At the time I was in favour of Pagure.

This news proves me wrong, which is nice, as it means that Alexander made the right choice for Debian.

Given that nobody else had stepped up to the plate to do the work involved, that choice was absolutely his to make, and I applaud his good judgement.

Note that the only contenders being considered by Fedora are: GitHub, GitLab, and Pagure.

I think it's clear that GitLab is a better choice than GitHub (for Debian at least, and probably for Fedora for similar reasons).

I would guess that Pagure is only in their list now because they already use it, whereas Debian was stuck with an unsupportable fusionforge so continuing with what we had was not an option.

Complaining that the new thing does not have every feature that existed in the old one seems a bit pointless, since continuing with Alioth as it was was not an available option.

Unfortunately, comfort seems to win

pabs — Thu, 30 Jan 2020 02:28:49 +0000

I expect Debian folks who disagreed with the GitLab switch went elsewhere or just reduced their involvement in the community. It certainly felt like a foregone conclusion so people probably felt like there was no chance to change the decision.

At the time SourceHut was not yet an option but if the decision were repeated today I think it would have been a contender.

Unfortunately, comfort seems to win

mirabilos — Thu, 30 Jan 2020 01:29:48 +0000

There weirdly was no widespread argument against Debian adapting GitLab CE for their self-hosted repositories, despite the lack of the #1 communication platform (mailing lists). Shockingly, they didn’t even use the packaged version (which, it turns out, is totally shit anyway) but the pre-made “packages” from the GitLab company.

Sic transit gloria mundi.

Thankfully, I can still self-host (or, currently, use the open-ish FusionForge instance of my employer). It’s worrisome, though.

Fedora gathering requirements for a Git forge

IanKelling — Wed, 29 Jan 2020 23:49:30 +0000

https://www.fsf.org/blogs/sysadmin/the-fsf-tech-team-doin... https://libreplanet.org/wiki/Fsf_2019_forge_evaluation

An FSF blog post about this project is in progress.