|
|
Subscribe / Log in / New account

Statistics from the 5.18 development cycle

By Jonathan Corbet
May 23, 2022
The 5.18 kernel was released on May 22 after a nine-week development cycle. That can only mean that the time has come to look at some of the statistics behind this release, which was one of the busiest in a while. Read on for a look at the 5.18 kernel, where the code in this release came from, and how it found its way into the mainline.

The 5.18 development cycle saw the addition of 14,954 non-merge changesets from 2,024 developers, 289 of whom made their first kernel contribution during this time. None of these numbers are records, though the number of developers came close to the maximum seen so far (2,062 for 5.13). This work resulted in the addition of 756,00 lines of code to the kernel.

The top contributors to 5.18 were:

Most active 5.18 developers
By changesets
Krzysztof Kozlowski 2141.4%
Matthew Wilcox1641.1%
Christoph Hellwig 1541.0%
Geert Uytterhoeven 1400.9%
Ville Syrjälä 1350.9%
Jonathan Cameron 1190.8%
Andy Shevchenko 1180.8%
Lorenzo Bianconi 1170.8%
Vladimir Oltean 1110.7%
Hans de Goede 1100.7%
Martin Kaiser 1100.7%
Colin Ian King 1040.7%
Sean Christopherson 1000.7%
Jakub Kicinski 1000.7%
Christophe JAILLET 890.6%
Michael Straube 870.6%
Jani Nikula 860.6%
Trond Myklebust 810.5%
Eric Dumazet 800.5%
Christophe Leroy 800.5%
By changed lines
Leo Li 22767619.4%
Qingqing Zhuo 19775716.9%
Ian Rogers 720086.1%
Alan Kao 158141.3%
Ming Qian 121761.0%
Linus Walleij 88810.8%
Krzysztof Kozlowski 88440.8%
Dimitris Michailidis 87910.7%
Christoph Hellwig 71650.6%
Matt Roper 71140.6%
Jakub Kicinski 70400.6%
Jacob Keller 68770.6%
Geert Uytterhoeven 60390.5%
Ranjani Sridharan 57680.5%
Evan Quan 52320.4%
Guodong Liu 49440.4%
Mauro Carvalho Chehab 48160.4%
Vladimir Oltean 47760.4%
Brett Creeley 46600.4%
Adrian Hunter 46510.4%

Krzysztof Kozlowski is the developer who contributed the most patches to 5.18; this work consisted mainly of device-tree updates. Matthew Wilcox managed to get another set of folio patches merged. Christoph Hellwig continues to massively refactor the block and filesystem layers. Geert Uytterhoeven contributed a large set of Renesas pin-control improvements, and Ville Syrjälä did a lot of work on the Intel i915 graphics driver.

In the "changed lines" column, Leo Li added over 200,000 lines with just five patches adding register definitions for the AMD graphics driver — and Qingqing Zhuo added nearly 200,000 more. Ian Rogers made a number of improvements to the perf tool, Alan Kao contributed a single patch removing the nds32 architecture, and Ming Qian contributed a set of Amphion media drivers.

The top testers and reviewers of patches were:

Test and review credits in 5.18
Tested-by
Daniel Wheeler 15511.7%
Damien Le Moal 775.8%
Konrad Jankowski 544.1%
David Howells 534.0%
Mike Marshall 534.0%
Gurucharan 382.9%
Marc Zyngier 322.4%
Vladimir Murzin 322.4%
Randy Dunlap 211.6%
Jiri Olsa 171.3%
Julian Grahsl 161.2%
Yihang Li 151.1%
Reviewed-by
Rob Herring 2172.7%
Christoph Hellwig 2042.6%
Andy Shevchenko 1431.1%
AngeloGioacchino Del Regno 1101.4%
Stephen Boyd 1031.3%
Pierre-Louis Bossart 1031.3%
Alex Deucher 981.2%
Krzysztof Kozlowski 961.2%
Hans de Goede 911.1%
Péter Ujfalusi 881.1%
Jani Nikula 861.1%
Himanshu Madhani 851.1%

Daniel Wheeler continues to receive the most test credits, having applied Tested-by tags to many AMD graphics-driver patches. It's worth noting that Wheeler posts occasional summaries describing the testing that has been done. Damien Le Moal tested many of the folio patches. and Konrad Jankowski regularly tests Intel network-driver patches.

Turning to the review column, Rob Herring routinely reviews device-tree patches. Christoph Hellwig reviewed patches in the block and filesystem subsystems — and a number of the folio patches as well. Andy Shevchenko reviewed many driver patches, mostly in the I2C, GPIO, and pin-control subsystems.

In the past it has been easy to be cynical about these numbers; they didn't capture much of the test and review activity happening in the community and were easily gamed. There is still surely a lot of work going on that is not reflected above, but it would be hard to argue that the testers and reviewers on these lists don't belong there. Perhaps this reflects a greater understanding of the value of these activities on the part of developers and (especially) their employers.

Whether the same can be said for bug reporting will be left for the reader to decide:

Top bug-report credits for 5.18
kernel test robot 23219.3%
Zeal Robot 766.3%
Syzbot726.0%
Abaci 625.2%
Dan Carpenter 292.4%
Hulk Robot 272.2%
Stephen Rothwell 262.2%
Igor Zhbanov 191.6%
Randy Dunlap 121.0%
Rob Herring 90.7%

Bug reporting is clearly a job for robots these days. But note that, while 2,249 5.18 patches were backported to the 5.17 stable updates (so far), only 1,075 contained Reported-by tags. That would suggest that that just over half of the fixes being applied do not carry those tags and that, probably, a number of bug reports are going without credit.

The employers contributing most actively to this development cycle were:

Most active 5.18 employers
By changesets
Intel170811.4%
(Unknown)11557.7%
Red Hat9586.4%
Google8865.9%
(None)8185.5%
AMD7815.2%
Linaro5603.7%
Huawei Technologies4713.1%
Facebook4463.0%
NVIDIA3962.6%
(Consultant)3632.4%
SUSE3442.3%
IBM3342.2%
Oracle3252.2%
Arm2942.0%
Renesas Electronics2621.8%
MediaTek2491.7%
NXP Semiconductors2361.6%
Canonical2271.5%
Microchip Technology2011.3%
By lines changed
AMD46764239.9%
Intel1070819.1%
Google1038018.8%
(Unknown)496694.2%
Linaro296312.5%
Red Hat288072.5%
(None)279892.4%
NXP Semiconductors214181.8%
NVIDIA192031.6%
MediaTek189801.6%
Facebook160361.4%
Andes Technology158141.3%
(Consultant)143141.2%
Huawei Technologies134831.1%
IBM119601.0%
Microchip Technology118531.0%
Renesas Electronics114271.0%
SUSE101280.9%
Canonical89840.8%
Fungible87910.7%

As usual, there are few surprises here.

[Patch-flow plot]

Patch flow and signed tags

The illegible plot on the right (click to be able to actually read it) shows the paths taken by patches into the mainline kernel. Each box represents a Git repository, with the vectors showing the movement of patches from one repository to the next. This plot, which was generated by the treeplot utility from the gitdm collection of hacks (available from git://git.lwn.net/gitdm.git), provides an overall picture of how code moves through the maintainer community.

That picture remains relatively flat; most maintainers push their changes directly to Linus Torvalds. There is, however, a steady growth in the role of intermediate repositories, with the biggest ones handling the networking, graphics, system-on-chip, and character driver subsystems. The plot is a schematic diagram of the machine that has allowed the kernel process to scale to its current size — and, presumably, beyond.

The color of each vector indicates whether that repository is using signed tags on patches being pushed to the next level in the hierarchy; red lines indicate the lack of such a tag. The use of GPG signatures on tags allows a receiving maintainer to verify that a pull request was created by the person it claims to be from. If all pull requests include signed tags, it will be significantly harder for an attacker to convince a maintainer to pull from a malicious branch.

As has been documented here over the years, that universal use of signed tags has been slow to happen. Recently, though, Torvalds has become more insistent, with explicit requests to recalcitrant maintainers to get with the program. The end result is that, for 5.18, only 714 patches did not come from a signed tag — and 565 of those were directly applied by Torvalds and didn't arrive via a Git repository at all. So, at the top level of the tree, the switch to using signed tags is nearly complete — a mere 11 years after the practice was adopted. Some of the mid-level maintainers are still clearly not requiring signed tags on pull requests, though, so there are still some holes in the process.

Older bugs

Many of the patches applied to 5.18 fix bugs; how old are those bugs? One way of approximating an answer to that question is to look at how many fixes showing up in the stable updates were first applied to 5.18. A bug fix, one would expect, will not be backported beyond the release that introduced the bug in the first place. The results for 5.18 are:

ReleaseBackports
5.17 (Mar 2022) 2,249
5.15 (Oct 2021) 1,762
5.10 (Dec 2020) 1,185
5.4 (Nov 2019) 756
4.19 (Oct 2018) 532
4.14 (Nov 2017) 422
4.9 (Dec 2016) 331

As can be seen above, 331 fixes (so far) have been ported from 5.18 all the way back to the 4.9 kernel, which was released over five years ago. In other words, after more than five years of intensive fixing (stable updates to 4.9 have added nearly 22,000 fixes), we are still fixing nearly five bugs in 4.9 every day. We'll get that kernel right one of these years, probably just before its end of life date.

To summarize, the kernel machine continues to move at high speed. Lots of bugs are being fixed and, beyond doubt, lots more are being introduced. The end result continues to be the kernel that we all rely on.

Index entries for this article
KernelReleases/5.18


to post comments

Statistics from the 5.18 development cycle

Posted May 25, 2022 11:44 UTC (Wed) by blackwood (guest, #44174) [Link] (2 responses)

Is there a way to make the number of patches/commits flowing through a tree in the big ineligble graph more readable? Even at full resolution there's many nodes where that number is blacked out by all the arrows going around.

Legible numbers

Posted May 25, 2022 13:30 UTC (Wed) by corbet (editor, #1) [Link] (1 responses)

There's probably some way to make graphviz produce something more legible...but I've always found my experiences with graphviz to involve a lot of head-pounding.

What I can do is give you the info in text form, which treeplot also puts out. Here we see each repo with a number showing how many commits it sent to the next upstream repo... The ** annotation indicates use of signed tags, $KORG is a kernel.org repo. Hope this helps.

2587 ** $KORG/netdev/net-next
   523 ** $KORG/wireless/wireless-next
     171 ** github.com/nbd168/wireless
      97 ** $KORG/iwlwifi/iwlwifi-next
   407 $KORG/bpf/bpf-next
       1 gitolite.kernel.org/pub/scm/linux/kernel/git/arm64/linux
   115 ** $KORG/saeed/linux
   109 $KORG/tnguy/next-queue
    87 ** $KORG/mkl/linux-can-next
    78 ** $KORG/bluetooth/bluetooth-next
    42 $KORG/netfilter/nf-next
    34 ** $KORG/netdev/net
      35 $KORG/bpf/bpf
      25 ** $KORG/wireless/wireless
      25 $KORG/netfilter/nf
      21 $KORG/tnguy/net-queue
      17 ** $KORG/mkl/linux-can
      16 ** $KORG/kernel/g
       6 ** $KORG/bluetooth/bluetooth
       5 $KORG/tnguy/net
       3 $KORG/klassert/ipsec
       2 ** git.open-mesh.org/linux-merge
    20 $KORG/tnguy/next
    18 ** $KORG/mellanox/linux
     6 $KORG/klassert/
     6 $KORG/sschmidt/wpan-next
     5 ** git.open-mesh.org/linux-merge
     2 $KORG/tnguy/linux
     1 ** $KORG/broonie/spi
1723 ** anongit.freedesktop.org/drm/drm
   718 gitlab.freedesktop.org/agd5f/linux
   444 ** anongit.freedesktop.org/drm/drm-intel
      68 ** anongit.freedesktop.org/drm/drm-intel
        14 github.com/intel/gvt-linux
      14 github.com/intel/gvt-linux
   406 ** anongit.freedesktop.org/drm/drm-misc
   101 gitlab.freedesktop.org/drm/msm
     9 ** $KORG/daeinki/drm-exynos
     9 ** gitlab.freedesktop.org/drm/tegra
     9 ** $KORG/chunkuang.hu/linux
     6 ** linuxtv.org/pinchartl/media
     5 ** git.pengutronix.de/pza/linux
     4 gitlab.freedesktop.org/zack/vmwgfx
1040 ** $KORG/soc/soc
   169 ** $KORG/qcom/linux
   118 ** $KORG/shawnguo/linux
    93 ** $KORG/krzk/linux
       1 ** $KORG/pinctrl/samsung
    56 ** $KORG/geert/renesas-devel
    46 ** $KORG/at91/linux
    44 ** $KORG/atorgue/stm32
    42 ** $KORG/tegra/linux
    42 ** $KORG/matthias.bgg/linux
    41 ** $KORG/sudeep.holla/linux
      20 ** $KORG/sudeep.holla/linux
    41 ** $KORG/mmind/linux-rockchip
    37 ** $KORG/joel/bmc
    36 ** $KORG/krzk/linux-mem-ctrl
    31 ** $KORG/dinguyen/linux
    27 ** $KORG/linusw/linux-nomadik
    26 ** $KORG/amlogic/linux
    25 ** $KORG/ti/linux
    23 ** github.com/Broadcom/stblinux
    17 ** github.com/linux-chenxing/linux
    13 ** git.linaro.org:/people/jens.wiklander/linux-tee
    12 ** $KORG/tmlind/linux-omap
    11 ** $KORG/sunxi/linux
    10 ** $KORG/gclement/mvebu
     8 ** $KORG/pchotard/sti
     5 ** git.pengutronix.de/pza/linux
     4 ** github.com/Xilinx/linux-xlnx
     2 ** git.linaro.org/people/jens.wiklander/linux-tee
     2 ** $KORG/narmstrong/linux-oxnas
 621 ** $KORG/tiwai/sound
   481 ** $KORG/broonie/sound
     1 ** $KORG/pdx86/platform-drivers-x86
 607 ** $KORG/gregkh/char-misc
   262 ** $KORG/jic23/iio
    64 ** $KORG/phy/linux-phy
    51 ** $KORG/ogabbay/linux
    28 ** $KORG/vkoul/soundwire
    14 ** $KORG/coresight/linux
    10 ** $KORG/djakov/icc
     7 ** $KORG/joel/fsi
     3 ** $KORG/mani/mhi
     2 ** $KORG/mdf/linux-fpga
 565 Applied by Linus
 468 ** $KORG/netdev/net
    35 $KORG/bpf/bpf
    25 ** $KORG/wireless/wireless
    25 $KORG/netfilter/nf
    21 $KORG/tnguy/net-queue
    17 ** $KORG/mkl/linux-can
    16 ** $KORG/kernel/g
     6 ** $KORG/bluetooth/bluetooth
     5 $KORG/tnguy/net
     3 $KORG/klassert/ipsec
     2 ** git.open-mesh.org/linux-merge
 458 ** $KORG/gregkh/staging
 442 ** $KORG/tip/tip
    61 ** $KORG/maz/arm-platforms
    19 ** git.linaro.org/people/daniel.lezcano/linux
     5 $KORG/frederic/linux-dynticks
     2 tip:locking/core
     2 ** $KORG/paulmck/linux-rcu
 427 ** $KORG/mchehab/linux-media
    76 ** linuxtv.org/hverkuil/media_tree
    19 ** linuxtv.org/sailus/media_tree
 360 ** $KORG/kvm/kvm
    56 ** $KORG/kvmarm/kvmarm
    22 ** $KORG/kvms390/linux
    18 ** github.com/kvm-riscv/linux
 342 ** $KORG/jejb/scsi
 322 ** git.kernel.dk/linux-block
    57 ** git.infradead.org/nvme
    11 $KORG/song/md
     3 $KORG/colyli/linux-bcache
 253 ** $KORG/clk/linux
    60 ** $KORG/qcom/linux
    25 ** $KORG/abelvesa/linux
    12 ** $KORG/geert/renesas-drivers
     6 ** $KORG/mmind/linux-rockchip
     4 ** $KORG/sunxi/linux
     3 ** $KORG/at91/linux
 221 ** $KORG/gregkh/usb
    15 ** $KORG/johan/usb-serial
    11 ** $KORG/westeri/thunderbolt
 214 ** $KORG/linusw/linux-pinctrl
    99 ** $KORG/geert/renesas-drivers
    13 ** $KORG/pinctrl/samsung
     8 ** gitolite.kernel.org:pub/scm/linux/kernel/git/pinctrl/intel
 210 ** $KORG/rafael/linux-pm
    32 ** $KORG/vireshk/pm
    14 ** $KORG/shuah/linux
    13 ** git.linaro.org/people/daniel.lezcano/linux
    12 ** $KORG/thermal/linux
     4 github.com/spandruvada/linux-kernel
     3 $KORG/wsa/linux
       2 ** $KORG/tip/tip
        61 ** $KORG/maz/arm-platforms
        19 ** git.linaro.org/people/daniel.lezcano/linux
         5 $KORG/frederic/linux-dynticks
         2 tip:locking/core
         2 ** $KORG/paulmck/linux-rcu
 203 ** $KORG/powerpc/linux
     1 ** $KORG/kvm/kvm
      56 ** $KORG/kvmarm/kvmarm
      22 ** $KORG/kvms390/linux
      18 ** github.com/kvm-riscv/linux
 203 ** $KORG/acme/linux
 195 ** git.infradead.org/users/willy/pagecache
 145 ** $KORG/herbert/crypto-2.6
 144 ** $KORG/gregkh/tty
 137 ** $KORG/kdave/linux
 134 ** $KORG/device-mapper/linux-dm
 123 ** $KORG/broonie/spi
 121 ** $KORG/robh/linux
 117 ** git.linux-nfs.org/projects/trondmy/linux-nfs
 112 ** $KORG/rdma/rdma
 110 ** $KORG/mtd/linux
 108 ** $KORG/s390/linux
 103 ** $KORG/arm64/linux
     1 ** $KORG/maz/arm-platforms
  98 ** $KORG/helgaas/pci
  98 ** $KORG/groeck/linux-staging
  98 ** $KORG/sre/linux-power-supply
  93 ** $KORG/pdx86/platform-drivers-x86
  93 ** git.lwn.net/linux
  82 ** $KORG/crng/random
  81 ** $KORG/deller/linux-fbdev
  78 ** git.armlinux.org.uk/~rmk/linux-arm
    56 ** $KORG/ardb/linux
  72 ** $KORG/kees/linux
     7 ** $KORG/ebiederm/user-namespace
  69 ** $KORG/gregkh/driver-core
  69 ** $KORG/riscv/linux
  69 $KORG/wsa/linux
     2 ** $KORG/tip/tip
      61 ** $KORG/maz/arm-platforms
      19 ** git.linaro.org/people/daniel.lezcano/linux
       5 $KORG/frederic/linux-dynticks
       2 tip:locking/core
       2 ** $KORG/paulmck/linux-rcu
  64 ** $KORG/lee/mfd
  60 ** $KORG/joro/iommu
    10 ** $KORG/will/linux
  57 ** $KORG/rostedt/linux-trace
  57 ** $KORG/abelloni/linux
  56 $KORG/hid/hid
  55 ** $KORG/shuah/linux-kselftest
  54 ** $KORG/deller/parisc-linux
  52 ** $KORG/dhowells/linux-fs
  52 ** $KORG/paulmck/linux-rcu
  50 ** $KORG/ulfh/mmc
  49 ** $KORG/cxl/cxl
  47 ** $KORG/tytso/ext4
  46 ** $KORG/dlemoal/libata
  46 ** $KORG/thierry.reding/linux-pwm
  42 ** $KORG/mst/vhost
  41 ** $KORG/remoteproc/linux
  39 ** $KORG/mips/linux
  38 ** $KORG/brgl/linux
     9 ** gitolite.kernel.org:pub/scm/linux/kernel/git/andy/linux-gpio-intel
  38 ** $KORG/vkoul/dmaengine
  37 ** $KORG/broonie/regulator
  37 ** $KORG/xfs/xfs-linux
  37 ** git.samba.org/sfrench/cifs-2.6
  37 ** $KORG/cel/linux
  36 ** $KORG/dtor/input
  35 ** github.com/awilliam/linux-vfio
    16 ** $KORG/mellanox/linux
  33 ** $KORG/jaegeuk/f2fs
  30 ** $KORG/gfs2/linux-gfs2
  29 ** $KORG/arnd/asm-generic
    20 ** $KORG/arnd/asm-generic
Skip $KORG/arnd/asm-generic 20 ['mainline', '$KORG/arnd/asm-generic']
  28 ** $KORG/rppt/memblock
  25 ** $KORG/pcmoore/selinux
  25 ** $KORG/masahiroy/linux-kbuild
  25 ** $KORG/jarkko/linux-tpmdd
  23 ** github.com/jcmvbkbc/linux-xtensa
  23 ** $KORG/ebiederm/user-namespace
  22 ** $KORG/nvdimm/nvdimm
  20 ** git.infradead.org/users/hch/dma-mapping
  20 ** git.linaro.org/landing-teams/working/fujitsu/integration
  19 ** $KORG/rw/uml
  18 ** $KORG/hyperv/linux
  18 ** $KORG/rw/ubifs
  17 ** github.com/ceph/ceph-client
  15 ** $KORG/xen/tip
  15 ** $KORG/xiang/erofs
  14 ** www.linux-watchdog.org/linux-watchdog
  14 ** $KORG/printk/linux
  13 ** $KORG/jack/linux-fs
  12 ** github.com/cminyard/linux-ipmi
  11 ** $KORG/akpm/mm
  11 ** git.samba.org/ksmbd
  10 ** $KORG/brauner/linux
  10 ** $KORG/chrome-platform/linux
   9 $KORG/viro/vfs
   9 ** $KORG/ras/ras
   9 ** $KORG/pavel/linux-leds
   9 ** $KORG/vbabka/slab
   9 ** $KORG/zohar/linux-integrity
   8 ** $KORG/vgupta/arc
   8 ** git.infradead.org/users/willy/xarray
   7 $KORG/tj/cgroup
   6 ** github.com/ojeda/linux
   6 ** $KORG/gerg/m68knommu
   6 ** $KORG/fscrypt/fscrypt
   5 ** $KORG/lee/backlight
   5 $KORG/tj/wq
   5 ** $KORG/broonie/regmap
   5 ** $KORG/geert/linux-m68k
   4 ** $KORG/i3c/linux
   4 ** git.monstr.eu/linux-2.6-microblaze
   4 ** $KORG/livepatching/livepatching
   3 ** $KORG/pcmoore/audit
   3 ** $KORG/dlemoal/zonefs
   3 ** $KORG/linkinjeon/exfat
   3 ** github.com/kleikamp/linux-shaggy
   3 ** $KORG/mic/linux
   2 ** $KORG/mcgrof/linux
   2 ** $KORG/danielt/linux
   2 ** $KORG/bcain/linux
   2 ** github.com/openrisc/linux
   2 ** git.osdn.net/gitroot/tomoyo/tomoyo-test1
   2 ** $KORG/gustavoars/linux
   2 ** github.com/cschaufler/smack-next
   1 $KORG/netfilter/nf
   1 $KORG/tnguy/net-queue
   1 $KORG/klassert/ipsec
   1 ** $KORG/saeed/linux
   1 ** $KORG/mkl/linux-can
   1 ** $KORG/mmind/linux-rockchip
   1 ** $KORG/qcom/linux
   1 ** $KORG/joel/bmc
   1 ** $KORG/tmlind/linux-omap
133 repos total, 124 signed, 9 unsigned
   14919 commits from signed, 714 from unsigned

Legible numbers

Posted May 26, 2022 4:04 UTC (Thu) by akiyks (guest, #115362) [Link]

The plot is provided in SVG at https://lwn.net/Articles/895911/.

You can download it and open it in a SVG editor such as Inkscape and move
those illegible numbers. You will need to do some ungrouping operations before
moving them.


Copyright © 2022, Eklektix, Inc.
This article may be redistributed under the terms of the Creative Commons CC BY-SA 4.0 license
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds