How Red Hat uses GitLab for kernel development

How Red Hat uses GitLab for kernel development

As a Red Hatter and an upstream developer, it's a bit more complicated than this. Despite being on a nominally old version, lots of subsystems in the RHEL kernel are rebased every 4-8 months to the latest version in the upstream kernel. For example, if you compare arch/x86/kvm/ between kernel-4.18.0-333.el8 and upstream 4.18, you get

 72 files changed, 45945 insertions(+), 26903 deletions(-)

If you do the same with 5.13, you get

 36 files changed, 985 insertions(+), 927 deletions(-)

These differences in turn are mostly because the kernel version backports the bugfix parts of what went into the 5.14 merge window. If it weren't for that, it would be even smaller.

As a result, there are two typical kinds of pull request in the RHEL kernel: bugfixes that are verbatim copies of 1-8 upstream commits, and large backports consisting of 200 to 1000 patches. The former are almost entirely trivial to review; often the developer who posts them is the same person who did the work upstream. The latter account for most of the 15000 patches mentioned in the article; of those, about 3-5% patches will have some difference with the corresponding upstream commit, typically due to bugfixes being backported out of order, and even fewer will be exclusive to RHEL. Of the 15000 patches, perhaps 1000 are going to be ever "clicked on" in the Gitlab merge request web pages.

As a result, review used to be 95+% the menial work of finding which patches have differences with the corresponding upstream. Such work is perfect for bots: they check the correspondance between the RHEL commit and the one in the "cherry picked from ..." lines (and the presence of those lines), signal differences, and let humans actually do the interesting parts of the review.

So, on one hand I absolutely love the GitLab-based development process that Prarit and Donald and many others have implemented. On the other hand, I would be wary to suggest that it could be applied to anything but the stable kernel process. In particular, code review on the web at the scale of Linux development is a complete non-starter until GitLab or GitHub implement a UI that maps to "git range-diff".