LWN: Comments on "Pulling GitHub into the kernel process"

Pulling GitHub into the kernel process

mathstuf — Tue, 27 Jul 2021 15:04:26 +0000

Thanks. I'll try and carve out some time to spruce up the descriptions and such.

Pulling GitHub into the kernel process

tpo — Tue, 27 Jul 2021 09:35:31 +0000

> Feel free to email me about any questions you might have (email address is in the history) or we can discuss here.

The Repo https://gitlab.kitware.com/utils/rust-ghostflow doesn't have a README.md and it's description is: "Routines which implement various parts of "ghostflow", or the git-hosted workflow.", which basically says "foo implements foo", which isn't helpful to the reader's understanding of what *actually* the purpose of that software is.

So I suggest to link from the description of https://gitlab.kitware.com/utils/rust-ghostflow to https://gitlab.kitware.com/utils/ghostflow-director/, which has minimal info on what ghostflow is, or even better to add a README.md to https://gitlab.kitware.com/utils/rust-ghostflow with minimal info that at least contains that link.

Pulling GitHub into the kernel process

mathstuf — Mon, 26 Jul 2021 16:22:24 +0000

Alas, there isn't a fancy webpage for it (it's used internally and the code is public, but no "marketing" behind it). Feel free to email me about any questions you might have (email address is in the history) or we can discuss here.

The way we deploy it is here: https://gitlab.kitware.com/utils/ghostflow-director/ (as a webhook handler for gitlab and github) and the core routines (including a nascent cli tool) are here: https://gitlab.kitware.com/utils/rust-ghostflow

Pulling GitHub into the kernel process

tpo — Mon, 26 Jul 2021 07:12:05 +0000

> And to plug ghostflow again,

When I search the nets for ghostflow, then there's almost nothing. The first result is www.ghostflow.com. When I go there the only thing I can do there is to log in. Is there a place in the electric sphere that would actually tell me what ghostflow can offer me (as opposed to me offering my email address to ghostflow)?

Pulling GitHub into the kernel process

mathstuf — Sat, 03 Jul 2021 20:27:26 +0000

I could understand if it were "I don't want to interact with anyone via GitHub", but that's not what was said. It's like if I said "I don't want to interact with anyone who uses public transport" versus "I don't want to use public transport" (purely hypothetical, I personally have loved public transport where I have successfully used it).

Pulling GitHub into the kernel process

Wol — Sat, 03 Jul 2021 19:11:11 +0000

Not necessarily. Not knowing Github, but if interacting via Github introduces friction, especially if it's painful friction, then that statement is both understandable and excusable.

Cheers,
Wol

Pulling GitHub into the kernel process

mathstuf — Sat, 03 Jul 2021 14:31:34 +0000

I see this mentioned in various places, then I get to the blockchain part of the webpage and remember why I end up closing the tab. It seems like an interesting project, but I feel like it has some of that "add the blockchain and it gets better" mentality. I'd really like some details as to what adding blockchain magic pixie dust would improve here for distributed hosting or software processes and why the alternatives to whatever it is doing aren't sufficient.

Pulling GitHub into the kernel process

Hi-Angel — Sat, 03 Jul 2021 10:26:59 +0000

So, Konstantin didn't mention Radicle? I find this interesting, it should be a nice fit for kernel development once mature. I wonder if their project didn't work out, or is there some other reason.

Pulling GitHub into the kernel process

mathstuf — Sat, 03 Jul 2021 02:37:37 +0000

FWIW, adding these kinds of things to ghostflow[1] seems like something within its wheelhouse. Willing to discuss if anyone is interested in implementing these actions (which can then be exposed through the `ghostflow-cli` tool). FWIW, format checking is already supported on a "check the whole file" level, but I suppose it can be checked if it conflicts with the diff to know if the modified lines need updated (rather than forcing the whole file to be formatted all the time; something not really feasible with the kernel).

[1]https://gitlab.kitware.com/utils/rust-ghostflow

Pulling GitHub into the kernel process

ms-tg — Fri, 02 Jul 2021 22:36:35 +0000

> "I don't want to interact with anyone who uses GitHub"

See: https://www.kernel.org/doc/html/v5.4/process/code-of-cond...

> Our Standards
>
> Examples of behavior that contributes to creating a positive environment include:
> * Using welcoming and inclusive language
> * Being respectful of differing viewpoints and experiences
> [...]
>
> Examples of unacceptable behavior by participants include:
> [...]
> * Trolling, insulting/derogatory comments, and personal or political attacks
> * Other conduct which could reasonably be considered inappropriate in a professional setting

Am I alone in seeing the above statement "I don't want to interact with anyone who uses GitHub" as being a very clear violation of the statements above?

Hmm...

Pulling GitHub into the kernel process

mblinov — Thu, 01 Jul 2021 22:07:13 +0000

I just had a "great" idea: Why doesn't the Linux Foundation provide a Docker container, which will contain a small email server that forwards your email to the "real" kernel mailing list, but before doing so will run all the checks and do all the formatting for you?

So I can do "git format-patch", send it to localhost:14121 (or wherever I launch the Docker instance), and then the Docker instance, upon receipt of the patch, will:
- Check the formatting,
- Check the "CC" list,
- Forward to lkml, etc.

And if anything isn't quite right, it just sends it right back to you and documents why and what needs changing, e.g. "Hey, your patch touched subsystem X but you didn't add X@so-and-so.org in the CC", and if you know what you're doing, I suppose you could have some mechanism for passing "flags" to the server to override warnings...

And then if you're really paranoid, it could contain a mirror of the linux mailing list, and you could choose to "emulate" sending your patch, which would really just send it to this internal mirrored mailing list. That way you can convince yourself that it really does get sent the way you want and expect it to.

Come to think of it, you could put a mini-webserver in it with all the clicky buttons anyone could want.

Too crazy?

Pulling GitHub into the kernel process

andy_shev — Thu, 01 Jul 2021 17:23:38 +0000

The curve of quality over time is on its falling phase, integration with GitHub and similar will speed up downfall even more. Agree with Laurent's opinion.

Pulling GitHub into the kernel process

pclouds — Thu, 01 Jul 2021 05:09:18 +0000

I believe the common way to deal with patches getting lost is resend, if the submitter is still interested in getting it merged. And it might be better that way: if you found an old "open" PR and reviewed it but the submitter already lost the interest to follow it up, it would be a waste of time.

Pulling GitHub into the kernel process

misc — Tue, 29 Jun 2021 14:12:32 +0000

Another project that could be looked at (and that was just started a few months ago) is https://fedeproxy.eu/blog/2021/01/16/what-is-fedeproxy/

Pulling GitHub into the kernel process

hkario — Mon, 28 Jun 2021 15:02:29 +0000

No, the experience is still "sub-par" at best.

I'm using Reviewable.io precisely because github makes it impossible to understand what happened during rebase if both the branch and master were updated in the mean-time.

Pulling GitHub into the kernel process

andyc — Sat, 26 Jun 2021 12:54:15 +0000

I interpreted his comment as "code review over email doesn't work", projects like the very one we're discussing! puts that idea to rest.

Pulling GitHub into the kernel process

flussence — Sat, 26 Jun 2021 09:57:14 +0000

I don't think any set of house rules has a specific clause about not dumping on someone else's workflow, but it wouldn't be necessary. That kind of software elitism rarely exists in a vacuum.

Pulling GitHub into the kernel process

cpitrat — Sat, 26 Jun 2021 06:52:49 +0000

My first interrogation reading this was why GitHub rather than GitLab? I guess the idea is to touch the large audience, not just to offer a nicer workflow ...

Pulling GitHub into the kernel process

Cyberax — Sat, 26 Jun 2021 01:02:25 +0000

git-send-email doesn't work with Exchange either. At my previous job I had to ask to enable IMAP-based mailboxes.

Pulling GitHub into the kernel process

LtWorf — Fri, 25 Jun 2021 22:53:46 +0000

That's what I was saying… I think the MTA is a serious barrier for people, while figuring out how to use git-send-email is (to me) acceptable.

Pulling GitHub into the kernel process

mathstuf — Fri, 25 Jun 2021 20:55:51 +0000

Not sure I'd consider myself "in the know" :) . Just that I have wishes for things in Git that eventually get addressed (`git range-diff` and `git bisect --first-parent` being the latest two) by other kind souls who actually have time and ability to get them done. I've scratched my own itch (`git remote get-url` is my contribution), but some of my wishes are just to big to work on in the background. I think ReviewBoard had a way to render them as a side-by-side view, but it's been so long…

Pulling GitHub into the kernel process

ms-tg — Fri, 25 Jun 2021 19:13:34 +0000

> "I don't want to interact with anyone who uses GitHub"

Is this anything that the code of conduct for the kernel might apply to?

Pulling GitHub into the kernel process

marcH — Fri, 25 Jun 2021 17:51:53 +0000

Since you're in the know, any chance for the git range-diff CLI to some day have a side by side mode like Gerrit does? Diffing diffs adds another dimension so using another axis is basically necessary, the display with two columns of pluses and minuses is unreadable for everything but the simplest changes. Has this been discussed already?

Pulling GitHub into the kernel process

bartoc — Fri, 25 Jun 2021 17:14:23 +0000

I hope github grows support for just like, dropping (or, god forbid, emailing) a textual patch file onto a repository to create a pull request, without having to go through the trouble of forking. This would be quite nice for package maintainers trying to get any patches upstream.

On another note I read the headline as "Github is moving their web framework into kernel space" and had a double take!

Pulling GitHub into the kernel process

mathstuf — Fri, 25 Jun 2021 16:51:56 +0000

Just a point of note that what was upstreamed to Git came from Gerrit, but ReviewBoard was "diffing the diffs" way back in 2012 or so to extract out the useful update diff for rebases. I'm sure there is prior art elsewhere as well, but I'm not aware of it.

And yes, I've been poking GitLab occasionally to finally implement that, but they always seem to have higher priority things :/ . If you plan ahead, you can rebase the diff as-is, push, then edit the topic to make useful inter-push diffs. Conflicts obvious get lost in the noise, but it's about the best one can do on the web interfaces these days.

Pulling GitHub into the kernel process

marcH — Fri, 25 Jun 2021 15:31:08 +0000

It's github's preferred model because it supports very well while supporting force pushes very poorly. Just read the link I posted if you're interested.

Pulling GitHub into the kernel process

marcH — Fri, 25 Jun 2021 15:28:56 +0000

It's pretty old and is one of the very few things that works when you force push to github. However it still does not perform a "git range-diff"[*] so the output has all the rebase noise and is unusable in case of a "real" rebase with an actual base change.

[*] range-diff was implemented by Gerrit in 2018, before git itself https://gitlab.com/gitlab-org/gitlab/-/issues/24096

Pulling GitHub into the kernel process

bluca — Fri, 25 Jun 2021 13:01:10 +0000

That's not Github's preferred model, it's the maintainer's preferred model for the projects you encountered. I work on literally dozens of OSS projects on Github, and none of them do that. There's nothing forcing anybody to do "well-formatted chains of commits" via email either, it's just the maintainers (rightly) enforcing that via reviews. That's because it's the maintainers who are enforcing determinate styles. Of course forges could always make these workflows easier and better integrated - it's not very obvious where to see diff-of-diffs on force pushes on Github and it wasn't possible at all until recently, and commit-by-commit reviews interface used to be much worse with comments getting lost (but that's no longer the case today).

Pulling GitHub into the kernel process

bluca — Fri, 25 Jun 2021 12:53:47 +0000

The delta is accessible on Github too nowadays - there's a notitication line in the PR when a force push happens: "<USER> force-pushed the <USER>:<BRANCH> branch from 1234 to 5678 x time ago", and the "force-pushed" text is a link that opens the delta before/after the force push.

Pulling GitHub into the kernel process

bluca — Fri, 25 Jun 2021 12:49:14 +0000

It's still not great, but nowadays you can click on the "force-pushed" line in a PR when a force-push happens, and you see the diff before/after the push. I believe this is relatively new, or at least I wasn't fully aware of it until this year.

Pulling GitHub into the kernel process

mathstuf — Fri, 25 Jun 2021 11:47:12 +0000

I believe you replied to the wrong comment, but I'll add my experience here since it's on my comment :) .

I'm not so sure about it being the "best" solution. Neither are the forges. Each have their problems. One issue with email is that if it is sent to the wrong lists, adding Cc members means:

- one can make a new Message-id for the same patch or reply to a patch with a new version (hopefully everyone sees this one too)
- seeing that the patch is indeed the same, just metadata is added is a manual process
- one can also bounce the email, but this is advanced and that it was sent to those lists is not in the normal metadata

As always, there is also a lack of transparency into what is happening with a patch through email. I had to manually see whether my patch was merged since there was never any feedback on the list about it after "I'll add it to my patch queue" from a relevant maintainer.

submitGit and GitGitGadget

jnareb — Fri, 25 Jun 2021 09:48:50 +0000

I wonder if porting existing *GitGitGadget* (https://gitgitgadget.github.io/) GitHub app, that is used for specifically git.git development via GitHub Pull Requests, to be used for LKML and Linux kernel is feasible. Like the Linux kernel, Git itself does not accept code contributions via Pull Requests, but uses patch submission via git mailing list.

There also is submitGit (https://submitgit.herokuapp.com/), but I think nowadays GitGitGadget is the preferred tool.

Pulling GitHub into the kernel process

Wladmis — Fri, 25 Jun 2021 09:47:05 +0000

> But that "email-based" part has proven to be problematic for some potential contributors, especially those who might want to simply submit a small bug fix

For small bug fix email is the best solution, Github and its clones are pain.

Pulling GitHub into the kernel process

Wladmis — Fri, 25 Jun 2021 09:43:56 +0000

> But that "email-based" part has proven to be problematic for some potential contributors, especially those who might want to simply submit a small bug fix

For small big fix email is the best solution, Github and its clones are pain.

Pulling GitHub into the kernel process

mathstuf — Thu, 24 Jun 2021 22:52:07 +0000

Yes, I agree that GitHub's interface is subpar for workflows that rewrite patches a lot. I'm not disputing that. I think it's that people think it's "Github or nothing".

In any case, ghostflow could push its check refs to the repository to force them to not be garbage collected. Any other system could probably also do the same.

Pulling GitHub into the kernel process

pbonzini — Thu, 24 Jun 2021 22:21:47 +0000

> there'd be a single place to find out about the entire history of a change. Review comments, change over time, the status of its journey to Linus' tree, etc.

As long as GitHub doesn't decide to garbage collect those commits, of course. And the delta between submissions of the same patch would be inaccessible from the web interface, because there's no way to do a range-diff from upstream to a PR.

GitLab is better, but not too much.

Pulling GitHub into the kernel process

amarao — Thu, 24 Jun 2021 22:00:53 +0000

Given the terrible uptime gihub actions has, I would add 'often broken' to the list of considerations.

Pulling GitHub into the kernel process

mathstuf — Thu, 24 Jun 2021 21:08:46 +0000

I'm not sure what you mean. What is not retrievable via the API about a GitHub (or GitLab) review that you're referring to?

Pulling GitHub into the kernel process

marcH — Thu, 24 Jun 2021 20:52:41 +0000

> In any case, the important bits are still retrievable just as easily in an email workflow.

Depends whether you consider the code review important or not.

Pulling GitHub into the kernel process

marcH — Thu, 24 Jun 2021 20:51:33 +0000

> the "gigantic list of things to do before contributing" that you mention, was the least of her worries compared to actually getting the code in good shape for submission.

Of course but not true for correcting a typo. Even github does not come easy some doc writers BTW.

Not true for the employees of many companies either: https://lwn.net/Articles/859859/