Pulling GitHub into the kernel process

By Jake Edge
June 23, 2021

There is an ongoing effort to "modernize" the kernel-development process; so far, the focus has been on providing better tools that can streamline the usual email-based workflow. But that "email-based" part has proven to be problematic for some potential contributors, especially those who might want to simply submit a small bug fix and are not interested in getting set up with that workflow. The project-hosting "forge" sites, like GitHub and GitLab, provide a nearly frictionless path for these kinds of one-off contributions, but they do not mesh well—at all, really—with most of mainline kernel development. There is some ongoing work that may change all of that, however.

Konstantin Ryabitsev at the Linux Foundation has been spearheading much of this work going back at least as far as his September 2019 draft proposal for better kernel tooling. Those ideas were discussed at the 2019 Kernel Maintainers Summit and at a meeting at Open Source Summit Europe 2019 in October. Throughout, Ryabitsev has been looking at ways to make it easier for non-email patch submitters; along the way, he has also released the b4 tool for collecting up patches and worked on patch attestation.

A recent post to the kernel workflows mailing list shows some progress toward a bot that can turn a GitHub pull request (PR) into a well-formed patch series to send to the proper reviewers and mailing lists. "This would be a one-way operation, effectively turning Github into a fancy 'git-send-email' replacement." He also laid out some of the benefits that this bot could provide both for maintainers and patch submitters:

submitters would no longer need to navigate their way around git-format-patch, get_maintainer.pl, and git-send-email -- nor would need to have a patch-friendly outgoing mail gateway to properly contribute patches
subsystem maintainers can configure whatever CI pre-checks they want before the series is sent to them for review (and we can work on a library of Github actions, so nobody needs to reimplement checkpatch.pl multiple times)
the bot should (eventually) be clever enough to automatically track v1..vX on pull request updates, assuming the API makes it straightforward

He had some questions about whether the bot should be centralized in a single repository (per forge platform) that would serve as the single submission point, or whether subsystem maintainers would want to configure their own repositories. The latter would give maintainers the opportunity to set their own criteria for checks that would need to pass (e.g. checkpatch.pl) before the PR was considered valid, but would mean that they might have to ride herd on the repository as well.

In addition, Ryabitsev wondered when and how PRs would get closed. The bot could potentially monitor the mainline and auto-close PRs once the patch set gets merged, but that won't be perfect, of course. An easier approach for him would be "to auto-close the pull request right after it's sent to the list with a message like 'thank you, please monitor your email for the rest of the process'", but he was unsure if that would be best.

As might be guessed, reactions from those participating in the thread were all over the map. While there is a lack of many kinds of diversity within the kernel community, opinions on development workflow—opinions, in general, in truth—do not have that problem. Some maintainers have zero interest in this kind of effort at all. As Christoph Hellwig put it: "Please opt all subsystems I maintain out of this crap. The last thing I need is patches from people that can't deal with a sane workflow."

Hellwig's complaint, which Jiri Kosina agreed with, may be more about the expectations of those who use GitHub (and the like), and less about the possibility of having a web-based interface to kernel development. Dmitry Vyukov asked why Hellwig and Kosina would be unwilling to accept patches from the system if they cannot really distinguish them from a regular submission. Vyukov said that he is currently experiencing a Git email submission problem that he is uninterested in working around, so he can see why others might be similarly inclined. Meanwhile, though, he sees benefits from this kind of bot:

On the other hand this workflow has the potential to ensure that you never need to remind to run checkpatch.pl, nor spend time on writing code formatting comments and re-reviewing v2 because code formatting will be enforced, etc. So I see how this is actually beneficial for maintainers.

Hellwig is not opposed to a web-based solution, though he wants nothing to do with GitHub. But Ryabitsev seems uninterested in "reimplementing a lot of stuff that we already get 'for free' from Github and other forges". Both Mark Brown and Laurent Pinchart suggested that there are mismatches between GitHub-normal practices and those of the kernel community. Pinchart mentioned the inability to comment on a patch's commit message on GitHub as something that generally leads to poor messages; the platform is training these developers to a certain extent:

Developers who have only been exposed to those platforms are very likely to never have learnt the importance of commit messages, and of proper split of changes across commits. Those are issues that are inherent to those platforms and that we will likely need to handle in an automated way (at least to some extent) or maintainers will become crazy [...]

But Miguel Ojeda thinks that it is really no different from new developers showing up on the mailing list with patches. "The same happens in the LKML -- some people have sent bad messages, but we correct them and they learn." He also noted that automated checking of patches can help both developers and maintainers:

[...] it is particularly useful to teach newcomers and to save time for maintainers having to explain things. Even if a maintainer has a set of email templates for the usual things, it takes time vs. not even having to read the email.

Ojeda is working on the Rust for Linux project, which we looked at back in April; he said that he has also been working on a bot:

For Rust for Linux, I have a GitHub bot that reviews PRs and spots the usual mistakes in commit messages (tags, formatting, lkml vs. lore links, that sort of thing). It has been very effective so far to teach newcomers how to follow the kernel development process.
I am also extending it to take Acks, Reviewed-by's, Tested-by's, etc., and then performing the merge only if the CI passes (which includes running tests under QEMU, code formatting, lints, etc.) after applying each patch.

But Ojeda is taking things in a rather different direction than what Ryabitsev is envisioning. Ojeda wants to move the main place for patch review and the like from the mailing lists to GitHub. He is also considering having his bot pick up patches from the mailing list and turning them into GitHub PRs—the reverse of what Ryabitsev is doing.

For his part, Ryabitsev said: "That's pretty cool, but I'm opposed to this on theological grounds. :)" In particular, he is concerned about the "single point of failure" problem for the kernel-development infrastructure. If his bot is unavailable for any reason, it may be inconvenient for those who use it, but that will not hobble development. He sees GitHub as simply a "developer frontend tool".

Somewhat similar to Ojeda's intentions, Brendan Higgins has a tool to pick up patches from a mailing list (kselftest in this case) and upload them to a Gerrit instance. He sees some potential synergies between his bot and the one Ryabitsev is working on. Similarly, Drew DeVault has been working on the reverse direction, from a mailing list to a project forge, as well. Patchwork is a longstanding code-review project that also collects up patches from mailing lists to populate a web application. It would seem that much of the focus is on getting patches out of mailing lists, though, which is not where Ryabitsev is headed.

While some maintainers want no part of this "GitHub Future", others are enthusiastic about the possibilities it could bring. Vyukov thinks that having a single GitHub repository with multiple branches will help consolidate the kernel-development landscape, which is currently fragmented on subsystem lines. He sees it as an opportunity to apply consistent coding-style standards; it does not matter which, he said, "as long as it's consistent across the project". It would also allow testing consistency throughout the tree and the same for the development process:

For once: it will be possible to have proper documentation on the process (as compared to current per-subsystem rules, which are usually not documented again because of low RoI [return on investment] for anything related to a single subsystem only).

It is not at all clear that Vyukov's interest in consistency throughout the tree is shared widely, but there have certainly been complaints along the way about the difficulty of navigating between the different subsystem processes and requirements for submissions. There is also interest in making things easier for quick, one-off contributions; as Ryabitsev put it:

Our code review process must also allow for what is effectively a "report a typo" link. Currently, this is extremely onerous for anyone, as a 15-minute affair suddenly becomes a herculean effort. The goal of this work is to make drive-by patches easier without also burying maintainers under a pile of junk submissions.

Clearly keeping "junk submissions" to a bare minimum is going to be important. Linus Torvalds said that he has had to turn off email from GitHub because it is too noisy; people have apparently signed him up as a project member without any kind of opt-in check. Beyond that, any kind of patch submission from PRs would need to have some sanity checks, including size limits, so that PRs like one pointed out by Ryabitsev do not end up on the mailing list.

That kind of PR highlights another problem: repository maintenance. Greg Kroah-Hartman said that there will be a need to monitor whatever repositories are being used for this purpose. It is not a small task:

What ever repo you put this on, it's going to take constant maintenance to keep it up to date and prune out the PRs that are going to accumulate there, as well as deal with the obvious spam and abuse issues that popular trees always accumulate.

Torvalds does not want his GitHub tree used for this purpose and Kroah-Hartman said the same. However it plays out, someone will have to be tasked with keeping the repository tidy, which is "a thankless task that will take constant work". But Ryabitsev is hopeful that the Linux Foundation could fund that kind of work if it becomes necessary.

In the end, it will likely come down to how seamlessly the GitHub bot fits in. If maintainers truly cannot really tell the difference in any substantive way, it is hard to see many of them rejecting well-formed patches that fix real problems in their subsystems. That ideal may not be reached right away, however, which might lead to a premature end to the experiment. It will be interesting to see it all play out over the coming months and years.

Index entries for this article
Kernel	Development tools/Forges

to post comments

Pulling GitHub into the kernel process

Posted Jun 23, 2021 23:46 UTC (Wed) by NYKevin (subscriber, #129325) [Link] (5 responses)

My 2¢: "I don't want to use or interact with GitHub in any way" is a completely reasonable position to take. "I don't want to interact with anyone who uses GitHub" is developer-hostile, and exactly the sort of toxicity that discourages me from becoming involved with LKML or Linux development. You do what you want with your workflow. Do not presume to dictate my workflow.

Pulling GitHub into the kernel process

Posted Jun 25, 2021 19:13 UTC (Fri) by ms-tg (subscriber, #89231) [Link] (4 responses)

> "I don't want to interact with anyone who uses GitHub"

Is this anything that the code of conduct for the kernel might apply to?

Pulling GitHub into the kernel process

Posted Jun 26, 2021 9:57 UTC (Sat) by flussence (guest, #85566) [Link] (3 responses)

I don't think any set of house rules has a specific clause about not dumping on someone else's workflow, but it wouldn't be necessary. That kind of software elitism rarely exists in a vacuum.

Pulling GitHub into the kernel process

Posted Jul 2, 2021 22:36 UTC (Fri) by ms-tg (subscriber, #89231) [Link] (2 responses)

> "I don't want to interact with anyone who uses GitHub"

See: https://www.kernel.org/doc/html/v5.4/process/code-of-cond...

> Our Standards
>
> Examples of behavior that contributes to creating a positive environment include:
> * Using welcoming and inclusive language
> * Being respectful of differing viewpoints and experiences
> [...]
>
> Examples of unacceptable behavior by participants include:
> [...]
> * Trolling, insulting/derogatory comments, and personal or political attacks
> * Other conduct which could reasonably be considered inappropriate in a professional setting

Am I alone in seeing the above statement "I don't want to interact with anyone who uses GitHub" as being a very clear violation of the statements above?

Hmm...

Pulling GitHub into the kernel process

Posted Jul 3, 2021 19:11 UTC (Sat) by Wol (subscriber, #4433) [Link] (1 responses)

Not necessarily. Not knowing Github, but if interacting via Github introduces friction, especially if it's painful friction, then that statement is both understandable and excusable.

Cheers,
Wol

Pulling GitHub into the kernel process

Posted Jul 3, 2021 20:27 UTC (Sat) by mathstuf (subscriber, #69389) [Link]

I could understand if it were "I don't want to interact with anyone via GitHub", but that's not what was said. It's like if I said "I don't want to interact with anyone who uses public transport" versus "I don't want to use public transport" (purely hypothetical, I personally have loved public transport where I have successfully used it).

Pulling GitHub into the kernel process

Posted Jun 24, 2021 0:07 UTC (Thu) by q_q_p_p (guest, #131113) [Link] (11 responses)

As long as GitHub doesn't support PRs from git repositories not hosted on GitHub servers and requires an account to even send a PR or small patch, fuck GitHub. This vendor lock-in and Ojeda approach ("Ojeda wants to move the main place for patch review and the like from the mailing lists to GitHub.") shouldn't be acceptable for kernel development.

Any solution to kernel-development problems, should be at least as decentralized as email and git are. I don't mind being subscribed to the kernel mailing lists with email address hosted on my server (still subscription is not necessary to contribute to the kernel), I do mind if I'm forced to have an account on microsoft-owned platform to contribute even the smallest patch.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 6:45 UTC (Thu) by zoobab (guest, #9945) [Link] (2 responses)

Github is not open source.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 7:26 UTC (Thu) by mjg59 (subscriber, #23239) [Link]

Neither was Bitkeeper. With luck the lessons from there will still be strong enough to avoid any vendor lockin in terms of the primary approaches to contributing to the kernel, but there are already people who use proprietary software to contribute to Linux and there are already people who use Github who would like to contribute to Linux, and I think there's a meaningful benefit in enabling that.

Pulling GitHub into the kernel process

Posted Jun 26, 2021 6:52 UTC (Sat) by cpitrat (subscriber, #116459) [Link]

My first interrogation reading this was why GitHub rather than GitLab? I guess the idea is to touch the large audience, not just to offer a nicer workflow ...

Pulling GitHub into the kernel process

Posted Jun 24, 2021 8:20 UTC (Thu) by LtWorf (subscriber, #124958) [Link] (6 responses)

Yes they'd be putting themselves in the hands of a single corporation, the same way it was before git. That would not be wise, but I don't think it will be allowed any time soon.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 10:46 UTC (Thu) by ojeda (subscriber, #143370) [Link] (5 responses)

We are not putting ourselves in the hands of a single corporation.

GitHub offers a service which is useful to us. If tomorrow GitHub stops being useful or a new, better service appears, we will migrate. Migrating is easy enough.

And, to be clear, the usual ML-based workflow still works and GitHub is not stopping us from using it.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 14:58 UTC (Thu) by oever (guest, #987) [Link] (4 responses)

Migration is not easy. All issues and pull requests are tied to GitHub user accounts. All 'stars' are tied to GitHub user accounts. These cannot be moved.

GitHub has a lock-in by login.

Using email for development removes the information intermediary.

Any web based front-end that uses cryptographically signed issues, PRs, 'stars' and other contributions where the private keys are not part of the website would be fine.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 15:43 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (3 responses)

Stars are pretty meaningless to workflows, so that doesn't seem all that important. Yes, issues and PRs being attributed to user accounts makes it harder to find out who to contact about it once moving, but that's the case if someone changes their email address (say, employment change, de-Googlification, or whatever).

In any case, the important bits are still retrievable just as easily in an email workflow. Using that information can be easier or harder depending on circumstances, but there's no guarantee that any of that works in an arbitrary future anyways.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 20:52 UTC (Thu) by marcH (subscriber, #57642) [Link] (2 responses)

> In any case, the important bits are still retrievable just as easily in an email workflow.

Depends whether you consider the code review important or not.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 21:08 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (1 responses)

I'm not sure what you mean. What is not retrievable via the API about a GitHub (or GitLab) review that you're referring to?

Pulling GitHub into the kernel process

Posted Jun 26, 2021 12:54 UTC (Sat) by andyc (subscriber, #1130) [Link]

I interpreted his comment as "code review over email doesn't work", projects like the very one we're discussing! puts that idea to rest.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 10:32 UTC (Thu) by ojeda (subscriber, #143370) [Link]

> This vendor lock-in and Ojeda approach ("Ojeda wants to move the main place for patch review and the like from the mailing lists to GitHub.") shouldn't be acceptable for kernel development.

To be clear: there is no requirement to have a GitHub account -- you can still send us patches through the mailing list and we can still review them there.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 0:18 UTC (Thu) by JohnVonNeumann (guest, #131609) [Link] (2 responses)

The issue I see with the negativity directed at "github-first" developers is that most young devs started programming using github (including myself). I understand the comments about bad commit messages, also one of my pet peeves for devs, but I don't think it's an insurmountable issue to direct people to the appropriate places, I learnt a lot from the git commit message guide on one of the "become a linux contributor" pages, and continue to use it in my day-to-day programming, but have never contributed to Linux. I would say that again, as a newer, younger dev, that the processes and gigantic list of things to do before contributing is off putting. I get why that amount of pre-work exists for such a large project, but I understand both sides of the argument.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 15:23 UTC (Thu) by pbonzini (subscriber, #60935) [Link] (1 responses)

Last month, as part of Google Summer of Code I worked with my student on her first submissions to QEMU. She had never seen any of "git rebase -i", "git add -p", "git format-patch", "git send-email" but she was able to learn everything in a matter of hours. The "send patches as email" part, which really is the "gigantic list of things to do before contributing" that you mention, was the least of her worries compared to actually getting the code in good shape for submission.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 20:51 UTC (Thu) by marcH (subscriber, #57642) [Link]

> the "gigantic list of things to do before contributing" that you mention, was the least of her worries compared to actually getting the code in good shape for submission.

Of course but not true for correcting a typo. Even github does not come easy some doc writers BTW.

Not true for the employees of many companies either: https://lwn.net/Articles/859859/

Pulling GitHub into the kernel process

Posted Jun 24, 2021 0:31 UTC (Thu) by ejr (subscriber, #51652) [Link] (15 responses)

Um, I've sent one-off patches via email as a nobody. They either were late to the party or superseded by better fixes (except for one trivial patch), but there was no friction. And this was before many of the layers of porcelain.

Because of my current work, I have to deal with one of the "forges." Not really useful to me, and in general we do better with email / teleconf for issues anyways. I wish I could redirect us to jit.si for the teleconf bits, but whatever.

Email sits on my laptop. I can search and process it in a zillion ways not yet imagined by the servers. But that's *my* workflow. Plus: git was designed to be distributed, learning from some other centralized systems that fell down and went boom. Cross-system pull requests have yet to reach any consensus and likely never will (commercialization requirements, just like how IETF ical doesn't support exceptions). But git format-patch provides a consensus and cross-platform format for some projects.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 2:44 UTC (Thu) by NYKevin (subscriber, #129325) [Link] (14 responses)

> but there was no friction.

Developers who have been doing the "email a patch" workflow for years tend to underestimate the amount of friction involved, because most of it is "obvious stuff." Of course you have to format your patches correctly, send email as plain text, CC the mailing list for the right subsystem, etc. Those things are all "obvious," and once you know how they are done, they are genuinely easy to do, so they don't get counted towards the total amount of friction in the system. But for new contributors, they are indeed friction and should be considered, assuming you actually want to attract new developers.

The main advantage that GitHub and other forges offer is discoverability. If you want to make a fork of someone's repository, there's a button for that. If you want to send a pull request, there's a button for that. If you want to file a bug, there's a button for that. You don't have to read some FAQ that tells you which mailing list to email and in what format, you just click around the interface until you find the button that does what you want.

Sure, this makes things easier for the developer. That doesn't mean the developer is dumber, just that they are expending fewer cognitive resources on meaningless administrivia, and more on actually programming. It used to be, if you made a syntax error, you'd find out about it the next day, when you got back the output of your code from the sysadmins. I'm sure the programmers of that era were better at spotting trivial syntax errors than we are today, but I'm skeptical that "spotting syntax errors" has anything to do with actual programming skills.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 8:27 UTC (Thu) by marcH (subscriber, #57642) [Link] (13 responses)

> The main advantage that GitHub and other forges offer is discoverability.

The main advantages are:
- Hyperlinks from anything to anything else and back. Compare to: not long ago there wasn't even any easy way in the kernel to find older revisions of a series!
- Fine-grained notifications to anything - from nothing to everything. The recipient is in control.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 16:29 UTC (Thu) by tlamp (subscriber, #108540) [Link] (12 responses)

> - Hyperlinks from anything to anything else and back. Compare to: not long ago there wasn't even any easy way in the kernel to find older revisions of a series!

In GH there's no real revision method, I constantly click on links only to get "did not find what you're looking for" (paraphrased) as the branch got force pushed, or a pull requested abandoned/closed and the new one was not linked in the old, as that is really only happening if a user comments "replaces #12345" or the like.

> - Fine-grained notifications to anything - from nothing to everything. The recipient is in control.

x-sieve (I'm not maintaining those directly, open-xchange has a nice web-gui for it) and mailing list subscriptions are way more flexible though, GH got a bit better recently, the recipient may even be in control, but what use if the choices are "scalding hot" or "freezing cold".

So I'm really not sold that *those* are the main advantages.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 17:58 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (4 responses)

Yes, Github's institutional denial of rebase being useful is…not nice. GitLab has been better AFAIK. And to plug ghostflow again, it keeps refs to every update it sees[1] if folks want archival information). Any commits that are checked get their own specific ref (the `refs/check/mr/X/Y` is the Yth update to MR X) so they stick around.

[1]Of course you can race and push twice before ghostflow fetches the middle one, but if you're winning that race, what are you really gaining?

Pulling GitHub into the kernel process

Posted Jul 26, 2021 7:12 UTC (Mon) by tpo (subscriber, #25713) [Link] (3 responses)

> And to plug ghostflow again,

When I search the nets for ghostflow, then there's almost nothing. The first result is www.ghostflow.com. When I go there the only thing I can do there is to log in. Is there a place in the electric sphere that would actually tell me what ghostflow can offer me (as opposed to me offering my email address to ghostflow)?

Pulling GitHub into the kernel process

Posted Jul 26, 2021 16:22 UTC (Mon) by mathstuf (subscriber, #69389) [Link] (2 responses)

Alas, there isn't a fancy webpage for it (it's used internally and the code is public, but no "marketing" behind it). Feel free to email me about any questions you might have (email address is in the history) or we can discuss here.

The way we deploy it is here: https://gitlab.kitware.com/utils/ghostflow-director/ (as a webhook handler for gitlab and github) and the core routines (including a nascent cli tool) are here: https://gitlab.kitware.com/utils/rust-ghostflow

Pulling GitHub into the kernel process

Posted Jul 27, 2021 9:35 UTC (Tue) by tpo (subscriber, #25713) [Link] (1 responses)

> Feel free to email me about any questions you might have (email address is in the history) or we can discuss here.

The Repo https://gitlab.kitware.com/utils/rust-ghostflow doesn't have a README.md and it's description is: "Routines which implement various parts of "ghostflow", or the git-hosted workflow.", which basically says "foo implements foo", which isn't helpful to the reader's understanding of what *actually* the purpose of that software is.

So I suggest to link from the description of https://gitlab.kitware.com/utils/rust-ghostflow to https://gitlab.kitware.com/utils/ghostflow-director/, which has minimal info on what ghostflow is, or even better to add a README.md to https://gitlab.kitware.com/utils/rust-ghostflow with minimal info that at least contains that link.

Pulling GitHub into the kernel process

Posted Jul 27, 2021 15:04 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

Thanks. I'll try and carve out some time to spruce up the descriptions and such.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 20:41 UTC (Thu) by marcH (subscriber, #57642) [Link] (6 responses)

> > - Hyperlinks from anything to anything else and back. Compare to: not long ago there wasn't even any easy way in the kernel to find older revisions of a series!

> In GH there's no real revision method, I constantly click on links only to get "did not find what you're looking for" (paraphrased) as the branch got force pushed

I tried (and failed) to separate into two different comments two very different Github facts:

- Hyperlinking in github is awesome. This thread.
- Github hates amending commits and force-pushing. This other post: https://lwn.net/Articles/860741/

Pulling GitHub into the kernel process

Posted Jun 25, 2021 12:49 UTC (Fri) by bluca (subscriber, #118303) [Link] (5 responses)

It's still not great, but nowadays you can click on the "force-pushed" line in a PR when a force-push happens, and you see the diff before/after the push. I believe this is relatively new, or at least I wasn't fully aware of it until this year.

Pulling GitHub into the kernel process

Posted Jun 25, 2021 15:28 UTC (Fri) by marcH (subscriber, #57642) [Link] (3 responses)

It's pretty old and is one of the very few things that works when you force push to github. However it still does not perform a "git range-diff"[*] so the output has all the rebase noise and is unusable in case of a "real" rebase with an actual base change.

[*] range-diff was implemented by Gerrit in 2018, before git itself https://gitlab.com/gitlab-org/gitlab/-/issues/24096

Pulling GitHub into the kernel process

Posted Jun 25, 2021 16:51 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (2 responses)

Just a point of note that what was upstreamed to Git came from Gerrit, but ReviewBoard was "diffing the diffs" way back in 2012 or so to extract out the useful update diff for rebases. I'm sure there is prior art elsewhere as well, but I'm not aware of it.

And yes, I've been poking GitLab occasionally to finally implement that, but they always seem to have higher priority things :/ . If you plan ahead, you can rebase the diff as-is, push, then edit the topic to make useful inter-push diffs. Conflicts obvious get lost in the noise, but it's about the best one can do on the web interfaces these days.

Pulling GitHub into the kernel process

Posted Jun 25, 2021 17:51 UTC (Fri) by marcH (subscriber, #57642) [Link] (1 responses)

Since you're in the know, any chance for the git range-diff CLI to some day have a side by side mode like Gerrit does? Diffing diffs adds another dimension so using another axis is basically necessary, the display with two columns of pluses and minuses is unreadable for everything but the simplest changes. Has this been discussed already?

Pulling GitHub into the kernel process

Posted Jun 25, 2021 20:55 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

Not sure I'd consider myself "in the know" :) . Just that I have wishes for things in Git that eventually get addressed (`git range-diff` and `git bisect --first-parent` being the latest two) by other kind souls who actually have time and ability to get them done. I've scratched my own itch (`git remote get-url` is my contribution), but some of my wishes are just to big to work on in the background. I think ReviewBoard had a way to render them as a side-by-side view, but it's been so long…

Pulling GitHub into the kernel process

Posted Jun 28, 2021 15:02 UTC (Mon) by hkario (subscriber, #94864) [Link]

No, the experience is still "sub-par" at best.

I'm using Reviewable.io precisely because github makes it impossible to understand what happened during rebase if both the branch and master were updated in the mean-time.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 0:47 UTC (Thu) by Conan_Kudo (subscriber, #103240) [Link] (1 responses)

I wonder how kernel developers would feel about using Pagure as a bridge to offer contemporary PR-style workflows with CI, bots, etc. Unlike GitHub, it's completely Free and Open Source software. It's also fairly minimalist and the specialized Linux kernel CI systems can easily integrate with it. Unlike most forges, Pagure stores nearly all of its project data as Git repositories and also supports a concept of a "remote pull request" where the user tells Pagure to pull the branch from another Git server's repo (which can even be a plain Git server!).

There's also work on a ForgeFed plugin to fully enable federation across Git systems that implement the ForgeFed protocol to file issues and submit pull requests.

The main missing piece is nobody has implemented a bridge between email and PR style workflows in Pagure (though the developers are interested in an implementation). With that, it'd be a fully free platform that could be tailored to handle preferred workflows. Even without it, it seems to have attracted a tiny bit of success so far...

Pulling GitHub into the kernel process

Posted Jun 29, 2021 14:12 UTC (Tue) by misc (subscriber, #73730) [Link]

Another project that could be looked at (and that was just started a few months ago) is https://fedeproxy.eu/blog/2021/01/16/what-is-fedeproxy/

Pulling GitHub into the kernel process

Posted Jun 24, 2021 1:38 UTC (Thu) by ploxiln (subscriber, #58395) [Link] (1 responses)

> Similarly, Drew DeVault has been working on the reverse direction, from a mailing list to a project forge, as well.

I think this misunderstands Drew DeVault's project, SourceHut aka sr.ht, somewhat. I think it actually goes both directions - it has an optional web interface which helps to generate emails, and encourages using email to review and merge patches.

https://man.sr.ht/git.sr.ht/send-email.md
https://man.sr.ht/git.sr.ht/#sending-patches-upstream

Pulling GitHub into the kernel process

Posted Jun 24, 2021 15:41 UTC (Thu) by ddevault (subscriber, #99589) [Link]

Not exactly. We have a web UI for preparing patches which is definitely helpful for incorporating people into the email workflow. The tool I mentioned in this email thread is different: we wrote a little web service which bidirectionally syncs emails sent to a mailing list with a gitlab merge request. The former, which you were referring to, is not related to any github or github-style forges; while the latter, is.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 2:16 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (9 responses)

I know this isn't exactly the place to introduce it to the kernel community (though I have posted about it on LWN before), but it's late here and I'd like to at least get things down initially. Plus, some from the thread may read this too :) .

As part of my work doing software process at $DAYJOB, I implemented what we call "ghostflow"[1] which is a mashup of "git-hosted workflow". It currently has backends for GitLab and GitHub, but I've been poking around at a Pagure one off and on. Basically, it implements actions which are useful for our workflow, but it should be flexible enough to implement all kinds of actions (including doing `format-patch` behind the scenes). I'd like it to work as a command line tool (so that it doesn't need to be "deployed" anywhere, but that's not how we use it and it hasn't seen much work beyond "it's possible" and implementations for checking and reformatting code.

Actions that I forsee which should be implemented for the kernel (by no means exhaustive, but informed by my contributions):

- making sure the right lists are in Cc
- make sure metadata is correct (stable, fixes tags, etc.)
- sanity checks on content (kernel-style)
- DCO checks
- collecting intra-push submission info before sending to the list(s)

Of particular interest is that PRs should *not* be closed before merging. This is one of my biggest gripes with the current workflow: when my patch lands, it's up to me to make sure it actually landed. There's no insight into the actual maintainer workflow without traipsing around the Git forest myself. *That's* what I'd like to see improved (as a contributor).

[1] https://gitlab.kitware.com/utils/rust-ghostflow

Pulling GitHub into the kernel process

Posted Jun 24, 2021 5:54 UTC (Thu) by laf0rge (subscriber, #6469) [Link] (6 responses)

I see a distinct lack of the non-free (as in FOSS) nature of github in this discussion, or at least the coverage here.

Whether or not to integrate a web based platform into the development process is one question, and there are many pro's and con's.

However, whether or not to rely on third-party, non-free SaaS is an entirely different question. I for myself am quite happy that the bitkeeper days are long over, and I wouldn't want to see a return of them.

So, IMHO, if at all one wants related integration with a web based "pull request" type tool, the discussion should be whether or not to integrate with a [potentially self-hosted] FOSS gitlab instance, or gitea, or whatever else might be out there.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 6:55 UTC (Thu) by NYKevin (subscriber, #129325) [Link] (1 responses)

In fairness, I don't see anything in the article suggesting that anyone is going to "rely" on GitHub. It sounds more like it's going to become an additional option for contributing.

If it became *the sole* option for contributing, that would obviously be a problem. But I can see nothing* wrong with adding many integrations for many different forges, and letting developers use what they want to use. If anything, that should reduce the risk** of having a single point of failure.

* True, someone has to maintain the integrations. But presumably, if an integration is not being maintained, then nobody finds it useful enough to bother, and it can be allowed to die in peace. This is a problem that solves itself.
** This risk is somewhat applicable to email, too. It is getting progressively harder and harder to turn up your own private SMTP server, in such a way that the big email providers are willing to exchange messages with it. I don't think email is going to die tomorrow, or in the current decade, but it's something to think about in the long run.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 7:30 UTC (Thu) by mjg59 (subscriber, #23239) [Link]

> If it became *the sole* option for contributing, that would obviously be a problem.

I'd go further than that. Back in the Bitkeeper days, people who were unwilling or unable to accept the Bitkeeper license could still make use of the CVS gateway, so it was never the sole option - but I think it was pretty clear that you were at a significant disadvantage by not relying on proprietary local tooling. I'm enthusiastic about making it more straightforward to use Github as a mechanism to get involved in kernel development, but I think there would need to be pushback against it being the *primary* option, not merely against it being the sole option.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 13:22 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (3 responses)

I'm not seeing anything about "move the Linux contribution workflow to GitHub". It's more "make GitHub a viable onramp to contributing". Everything still ends up on the ML, but maybe there'd be a single place to find out about the entire history of a change. Review comments, change over time, the status of its journey to Linus' tree, etc.

For example, I fixed a bug that was reported by some automated test setup. I used the same set of Cc emails and sent it off. However, it was missing the list that needed to hear about the patch. It wasn't until I pinged after a week of silence that this had been realized. Why did it miss the target list? I don't know, but checkpatch and whatever hooks `send-email` has didn't figure it out (and there were a dozen+ emails already, so spotting a missing one wasn't easy).

So this isn't about going anywhere near what BitKeeper was. It's about adding more roads to Rome and putting up better signage and helpful tourist centers along the way.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 22:21 UTC (Thu) by pbonzini (subscriber, #60935) [Link] (2 responses)

> there'd be a single place to find out about the entire history of a change. Review comments, change over time, the status of its journey to Linus' tree, etc.

As long as GitHub doesn't decide to garbage collect those commits, of course. And the delta between submissions of the same patch would be inaccessible from the web interface, because there's no way to do a range-diff from upstream to a PR.

GitLab is better, but not too much.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 22:52 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

Yes, I agree that GitHub's interface is subpar for workflows that rewrite patches a lot. I'm not disputing that. I think it's that people think it's "Github or nothing".

In any case, ghostflow could push its check refs to the repository to force them to not be garbage collected. Any other system could probably also do the same.

Pulling GitHub into the kernel process

Posted Jun 25, 2021 12:53 UTC (Fri) by bluca (subscriber, #118303) [Link]

The delta is accessible on Github too nowadays - there's a notitication line in the PR when a force push happens: "<USER> force-pushed the <USER>:<BRANCH> branch from 1234 to 5678 x time ago", and the "force-pushed" text is a link that opens the delta before/after the force push.

Pulling GitHub into the kernel process

Posted Jun 25, 2021 9:43 UTC (Fri) by Wladmis (guest, #152006) [Link] (1 responses)

> But that "email-based" part has proven to be problematic for some potential contributors, especially those who might want to simply submit a small bug fix

For small big fix email is the best solution, Github and its clones are pain.

Pulling GitHub into the kernel process

Posted Jun 25, 2021 11:47 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

I believe you replied to the wrong comment, but I'll add my experience here since it's on my comment :) .

I'm not so sure about it being the "best" solution. Neither are the forges. Each have their problems. One issue with email is that if it is sent to the wrong lists, adding Cc members means:

- one can make a new Message-id for the same patch or reply to a patch with a new version (hopefully everyone sees this one too)
- seeing that the patch is indeed the same, just metadata is added is a manual process
- one can also bounce the email, but this is advanced and that it was sent to those lists is not in the normal metadata

As always, there is also a lack of transparency into what is happening with a patch through email. I had to manually see whether my patch was merged since there was never any feedback on the list about it after "I'll add it to my patch queue" from a relevant maintainer.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 8:31 UTC (Thu) by LtWorf (subscriber, #124958) [Link] (10 responses)

Talking about my personal experience:

Most people will not have a MTA configured locally, and that's needed to contribute using an email based workflow, but it's not very very easy to configure one.

After that is done, basically the hard part is getting used to using command line rather than clicky GUI. Which for some is an insurmountable issue.

However, to be honest, the fact that a pull request from github is so easy to make from a browser does bring issues of low quality contributions.

I've had pull requests from bots adding powerpc as a test platform (for python projects??), I guess with the sole purpose of consuming my travisCI credit, since this happened about the time when they stopped offering free builds and added a one time credit thing.

I've had many others "fix" a thing that would break things for every other user than themselves and just never bother to reply on the request to improve their contribution.

I've also had a censor edit the README file, and bugs opened asking me to change license, because copyleft is bad (because they can't reuse my stuff in their proprietary stuff).

To be honest, I accept pull requests because since my projects are really small, I have no power to dictate anything. But if they were larger projects, I'd absolutely want an email based workflow just to weed out the low effort contributions.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 8:58 UTC (Thu) by mfuzzey (subscriber, #57966) [Link] (9 responses)

> the hard part is getting used to using command line rather than clicky GUI. Which for some is an insurmountable issue.

Maybe in the general case but I think the intersection of those likely to find a CLI insurmountable and those able and willing to contribute to the kernel is pretty small.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 12:53 UTC (Thu) by LtWorf (subscriber, #124958) [Link] (8 responses)

It's what I'm saying. I'm quite ok with keeping away contributors who are not willing to use a command line, but I think maybe asking everyone to learn to configure a MTA is a bit too much and probably adds nothing in terms of skills. Unless the project in question is a MTA :)

Pulling GitHub into the kernel process

Posted Jun 24, 2021 16:05 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (7 responses)

Wow, such inclusiveness.

In reality, it's not just command line. You need to set up a whole parallel email infrastructure, because you can't just send patches using Outlook or Gmail. And "blessed" email clients like mutt don't support Exchange mailbox protocol.

In my previous company I spent something like 3 days just setting up the email stuff to submit a kernel patch.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 16:10 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (4 responses)

Gmail does support it (I send patches through it). But with XOAUTH2 being a thing, app-specific passwords being sunset, and not knowing how msmtp and offlineimap are going to support it[1], that's not something I'd expect to be true for too much longer. Though for those poor souls trying to send via the web interface, yeah, that's a lost cause at this point and has been for at least a few years.

[1] My $DAYJOB email is fine because I can register apps and make offlineimap work at least, but my personal account doesn't seem to have the ability to register an application, so I'm probably stuck in the long term, but at least I have been migrating away for personal usage.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 16:36 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

I think there's a project that works as a local XOAUTH2 proxy for IMAP. So technically it should be still doable, but it's adding a new layer of brokenness.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 17:03 UTC (Thu) by mathstuf (subscriber, #69389) [Link] (2 responses)

That works if the app itself is registered, but since the app secret key is necessarily visible to me since I'd only really trust it if it were FOSS, that is destined to be revoked if/when the attention of the Eye of Sauron^WGoogle comes across it. OAuth is really good at locking out custom builds because now the service providers can deny arbitrary apps based on whatever whims they have that day. It is useful for blocking copycat spamware, but the knock-on effects for those of us who cobble together our setups is quite unfortunate :( .

Pulling GitHub into the kernel process

Posted Jun 24, 2021 17:06 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (1 responses)

Yes, OAuth has this problem. I'm not even sure what the solution would be. WebAuthn over SASL?

Pulling GitHub into the kernel process

Posted Jun 24, 2021 17:54 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

AFAIU, OAuth is supposed to add in a "this app is authorized to use this service" step to using the service through my account. This usually does come with real benefits:

- apps are limited in the API they access (and is curated by the developer rather than users not knowing what is actually necessary)
- additional permissions can be intercepted and requested at application update time (when refreshing their active token)
- dropping permissions doesn't require users to go and do it manually
- if my account secret token is stolen from app A, app B can't use it to access my account because it isn't authorized to do so
- services can pinpoint misbehaving applications or use of deprecated APIs and contact application developers directly

Of course, Fastmail's application-specific passwords allow you to limit which service(s) are available, but since there's no application authentication, stealing the password from offlineimap does grant IMAP access which is…substantial.

Honestly, I think I'd be OK with service-specific passwords that can be authorized every N days through WebAuthn or some other hardware token mediated thing. Though this would mean my automated backups would require some more maintenance though since I'd need to go and touch a Yubikey or whatever to keep its authorization token alive.

Pulling GitHub into the kernel process

Posted Jun 25, 2021 22:53 UTC (Fri) by LtWorf (subscriber, #124958) [Link] (1 responses)

That's what I was saying… I think the MTA is a serious barrier for people, while figuring out how to use git-send-email is (to me) acceptable.

Pulling GitHub into the kernel process

Posted Jun 26, 2021 1:02 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link]

git-send-email doesn't work with Exchange either. At my previous job I had to ask to enable IMAP-based mailboxes.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 8:38 UTC (Thu) by marcH (subscriber, #57642) [Link]

> Similarly, Drew DeVault has been working on the reverse direction, from a mailing list to a project forge, as well. Patchwork is a longstanding code-review project that also collects up patches from mailing lists to populate a web application. It would seem that much of the focus is on getting patches out of mailing lists, though, which is not where Ryabitsev is headed.

A forge has more structured metadata and more information in general, so when trying to bridge the two approaches it's of course more logical to want the forge to be the primary and derive the more basic email format from the forge as opposed to the other way round. No surprise many people try this. I understand there are other, less technical considerations.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 8:48 UTC (Thu) by marcH (subscriber, #57642) [Link] (7 responses)

> Both Mark Brown and Laurent Pinchart suggested that there are mismatches between GitHub-normal practices and those of the kernel community.

That's an understatement. Github's preferred review model does not involve rewriting commits! In other words forget "git range-diff". Review comments are shared as fixups instead and many projects don't even squash them.
https://github.com/zephyrproject-rtos/zephyr/pull/14444#i...

Github is basically "git as used by the creator of Fossil"
https://fossil-scm.org/home/doc/trunk/www/rebaseharm.md

Gerrit is much closer to "traditional" git usage and gitlab is somewhere in between.

And yes there is also the problem of vendor lock-in: good luck exporting code review metadata from github.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 9:47 UTC (Thu) by esmil (subscriber, #82064) [Link] (3 responses)

> Gerrit is much closer to "traditional" git usage and gitlab is somewhere in between.

I don't know if Gerrit has changed since I last looked at it, but back then it didn't support "1 review, multiple patches". You'd either have to squash everything into one big patch which is obviously bad, or you'd have to tell all your colleagues to please review these patches in this order :/

Pulling GitHub into the kernel process

Posted Jun 24, 2021 10:10 UTC (Thu) by Conan_Kudo (subscriber, #103240) [Link] (1 responses)

But that's how it works in the email flow. What makes Gerrit awful (aside from the horribly busy UI) is that there's no concept of a connected patch series to review together. Having contributed to Chromium, that was the worst part of the experience. Refreshing patches or updating them based on review feedback was confusing and it was difficult to see whether anything I did was "correct".

Pulling GitHub into the kernel process

Posted Jun 24, 2021 10:25 UTC (Thu) by esmil (subscriber, #82064) [Link]

Yeah, except the tooling now understands "[PATCH vX Y/Z]" prefixes and git send-email groups the patches by making them replies to the first in the series.

At least on GitHub/GitLab the basic unit of review is a pull/merge request that may contain multiple commits. The drawback is of course that the submitter needs to do a "scary" force push to update the series.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 11:53 UTC (Thu) by pizza (subscriber, #46) [Link]

Gerrit has understood the notion of a dependent patch series for at least as long as I've been using it for real work (2015ish).

I don't believe there is the concept of a single atomic review for an entire patch series, as reviews are on a per-patch basis, with comments attached to specific code sections as well as independent comments on the commit message. That's all what one generally gets via "traditional" email review anyway. That said, gerrit does provide mechansims for ensuring the entire series is applied/submitted in one go.

Also, the Gerrit UI was considerably revamped ("polygerrit") with the 3.0 release and has been continually refined since; I personally like it a lot better than than previous ("GWT") UI.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 20:47 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

> And yes there is also the problem of vendor lock-in: good luck exporting code review metadata from github.

File/line information is available (certainly more easily than in email threads) via the API. We certainly extract out "review evidence" with ghostflow so that it can all be added to the merge commit from comments made on the PR (including from those reactions one can make). Yes, it's in a different format, but at least it is structured, unlike email review feedback.

Pulling GitHub into the kernel process

Posted Jun 25, 2021 13:01 UTC (Fri) by bluca (subscriber, #118303) [Link] (1 responses)

That's not Github's preferred model, it's the maintainer's preferred model for the projects you encountered. I work on literally dozens of OSS projects on Github, and none of them do that. There's nothing forcing anybody to do "well-formatted chains of commits" via email either, it's just the maintainers (rightly) enforcing that via reviews. That's because it's the maintainers who are enforcing determinate styles. Of course forges could always make these workflows easier and better integrated - it's not very obvious where to see diff-of-diffs on force pushes on Github and it wasn't possible at all until recently, and commit-by-commit reviews interface used to be much worse with comments getting lost (but that's no longer the case today).

Pulling GitHub into the kernel process

Posted Jun 25, 2021 15:31 UTC (Fri) by marcH (subscriber, #57642) [Link]

It's github's preferred model because it supports very well while supporting force pushes very poorly. Just read the link I posted if you're interested.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 9:04 UTC (Thu) by Otus (subscriber, #67685) [Link] (4 responses)

IME the key to a pull request process that works with outsider and drive-by contributions is automation.

There *must* be automatic sanity checks for PRs, including that the change builds, that no new tests are broken, and in the case of the kernel likely that checkpatch is ok with it.

There also needs to be automation (i.e. a bot) for prodding the PR opener to fix such issues (with pointers to clear contributor guides) and to close PRs that remain unfixed too long.

Finally it needs to be easy for developers/maintainers to only see the PRs that are review/merge ready. And not just in the opinion of the contributor, but where all the checks pass.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 9:24 UTC (Thu) by fratti (subscriber, #105722) [Link] (3 responses)

> There *must* be automatic sanity checks for PRs, including that the change builds, that no new tests are broken

Do you want cryptocurrency miner spam? Because that is how you get cryptocurrency miner spam. Anything that builds and executes some random's submission will be open for abuse of compute resources.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 9:34 UTC (Thu) by Otus (subscriber, #67685) [Link] (2 responses)

GitHub is full of repositories that already do that. Crypto-scammers would be stupid to target such a visible project as the kernel.

That aside: I think it's a problem for GitHub (or whichever alternative) to solve.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 9:58 UTC (Thu) by ilammy (subscriber, #145312) [Link] (1 responses)

They already kinda solved it—for their own GitHub Actions—by requiring manual approval of CI runs for first-time contributors.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 13:07 UTC (Thu) by bluss (guest, #47454) [Link]

And this made first-time contributors not have immediate feedback on their contributions anymore unfortunately - they have to wait until someone comes around and locks it up for them.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 9:20 UTC (Thu) by fratti (subscriber, #105722) [Link] (4 responses)

> "Please opt all subsystems I maintain out of this crap. The last thing I need is patches from people that can't deal with a sane workflow."

Ah yes, the very sane workflow which leads to 5% of all bugfixes being lost, and which cannot correlate a patch that has been merged to the one that was submitted. This isn't really a matter of taste anymore; the kernel is missing important bugfixes because the current workflow allows patches to get lost on some mailing list way too easily, and heavily relies on people resubmitting patches they feel strongly about.

Anyone who has ever tried tracking patches as new versions come in and get merged in various branches across subsystem trees knows how inconvenient it is compared to a pull request with feature branch based system.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 16:45 UTC (Thu) by syrjala (subscriber, #47399) [Link] (3 responses)

I don't really see how github/gitlab/etc. would solve the problem of patches falling through the cracks. The PRs/MRs can be ignored just as easily as emailed patches.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 18:00 UTC (Thu) by mathstuf (subscriber, #69389) [Link]

Yes, but there's a difference between "actually ignored", "ignored a week after the Ack", and "silently merged a week after the Ack". Being able to tell the difference between the last two is where my initial interest lies.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 18:13 UTC (Thu) by fratti (subscriber, #105722) [Link] (1 responses)

It has a built-in mechanism to update a patch series (i.e. a branch) and to mark it as closed or merged. The latter can technically be done with patchwork from what I've heard, but I've never seen it done consistently. This would give you a view of what still hasn't been looked at enough: open PRs with no reviews and nothing explicitly blocking their merge.

Just to be clear, I'm not advocating for use of GitHub specifically, I'm pointing out how the current workflow's lack of state tracking for patches is causing real problems, and forges are the only ones who have been innovating in this direction.

Pulling GitHub into the kernel process

Posted Jul 1, 2021 5:09 UTC (Thu) by pclouds (guest, #76590) [Link]

I believe the common way to deal with patches getting lost is resend, if the submitter is still interested in getting it merged. And it might be better that way: if you found an old "open" PR and reviewed it but the submitter already lost the interest to follow it up, it would be a waste of time.

Pulling GitHub into the kernel process

Posted Jun 24, 2021 22:00 UTC (Thu) by amarao (guest, #87073) [Link]

Given the terrible uptime gihub actions has, I would add 'often broken' to the list of considerations.

Pulling GitHub into the kernel process

Posted Jun 25, 2021 9:47 UTC (Fri) by Wladmis (guest, #152006) [Link]

> But that "email-based" part has proven to be problematic for some potential contributors, especially those who might want to simply submit a small bug fix

For small bug fix email is the best solution, Github and its clones are pain.

submitGit and GitGitGadget

Posted Jun 25, 2021 9:48 UTC (Fri) by jnareb (subscriber, #46500) [Link]

I wonder if porting existing *GitGitGadget* (https://gitgitgadget.github.io/) GitHub app, that is used for specifically git.git development via GitHub Pull Requests, to be used for LKML and Linux kernel is feasible. Like the Linux kernel, Git itself does not accept code contributions via Pull Requests, but uses patch submission via git mailing list.

There also is submitGit (https://submitgit.herokuapp.com/), but I think nowadays GitGitGadget is the preferred tool.

Pulling GitHub into the kernel process

Posted Jun 25, 2021 17:14 UTC (Fri) by bartoc (guest, #124262) [Link]

I hope github grows support for just like, dropping (or, god forbid, emailing) a textual patch file onto a repository to create a pull request, without having to go through the trouble of forking. This would be quite nice for package maintainers trying to get any patches upstream.

On another note I read the headline as "Github is moving their web framework into kernel space" and had a double take!

Pulling GitHub into the kernel process

Posted Jul 1, 2021 17:23 UTC (Thu) by andy_shev (subscriber, #75870) [Link]

The curve of quality over time is on its falling phase, integration with GitHub and similar will speed up downfall even more. Agree with Laurent's opinion.

Pulling GitHub into the kernel process

Posted Jul 1, 2021 22:07 UTC (Thu) by mblinov (guest, #135953) [Link] (1 responses)

I just had a "great" idea: Why doesn't the Linux Foundation provide a Docker container, which will contain a small email server that forwards your email to the "real" kernel mailing list, but before doing so will run all the checks and do all the formatting for you?

So I can do "git format-patch", send it to localhost:14121 (or wherever I launch the Docker instance), and then the Docker instance, upon receipt of the patch, will:
- Check the formatting,
- Check the "CC" list,
- Forward to lkml, etc.

And if anything isn't quite right, it just sends it right back to you and documents why and what needs changing, e.g. "Hey, your patch touched subsystem X but you didn't add X@so-and-so.org in the CC", and if you know what you're doing, I suppose you could have some mechanism for passing "flags" to the server to override warnings...

And then if you're really paranoid, it could contain a mirror of the linux mailing list, and you could choose to "emulate" sending your patch, which would really just send it to this internal mirrored mailing list. That way you can convince yourself that it really does get sent the way you want and expect it to.

Come to think of it, you could put a mini-webserver in it with all the clicky buttons anyone could want.

Too crazy?

Pulling GitHub into the kernel process

Posted Jul 3, 2021 2:37 UTC (Sat) by mathstuf (subscriber, #69389) [Link]

FWIW, adding these kinds of things to ghostflow[1] seems like something within its wheelhouse. Willing to discuss if anyone is interested in implementing these actions (which can then be exposed through the `ghostflow-cli` tool). FWIW, format checking is already supported on a "check the whole file" level, but I suppose it can be checked if it conflicts with the diff to know if the modified lines need updated (rather than forcing the whole file to be formatted all the time; something not really feasible with the kernel).

[1]https://gitlab.kitware.com/utils/rust-ghostflow

Pulling GitHub into the kernel process

Posted Jul 3, 2021 10:26 UTC (Sat) by Hi-Angel (guest, #110915) [Link] (1 responses)

So, Konstantin didn't mention Radicle? I find this interesting, it should be a nice fit for kernel development once mature. I wonder if their project didn't work out, or is there some other reason.

Pulling GitHub into the kernel process

Posted Jul 3, 2021 14:31 UTC (Sat) by mathstuf (subscriber, #69389) [Link]

I see this mentioned in various places, then I get to the blockchain part of the webpage and remember why I end up closing the tab. It seems like an interesting project, but I feel like it has some of that "add the blockchain and it gets better" mentality. I'd really like some details as to what adding blockchain magic pixie dust would improve here for distributed hosting or software processes and why the alternatives to whatever it is doing aren't sufficient.