Some 5.5 kernel development statistics

The 5.5 kernel was released on January 26. Over the course of this development cycle, it was occasionally said that the holidays were slowing contributions. At the end, though, 5.5 saw the merging of 14,350 non-merge changesets from 1,885 developers — not exactly a slow-moving cycle. Indeed, 5.5 just barely edged out 5.4 as the kernel with the most developers ever. Read on for our traditional look at where the contributions to 5.5 came from, along with a digression into the stable-update process.

Just under 590,000 lines of code were added for 5.5, while almost 272,000 were removed, for a net growth of 318,000 lines of code. Of the developers contributing to 5.5, 285 were contributing for the first time. The most active developers working on 5.5 were:

Most active 5.5 developers

By changesets Chris Wilson 264 1.8% Christoph Hellwig 221 1.5% Yue Haibing 197 1.4% Colin Ian King 144 1.0% Thierry Reding 139 1.0% Krzysztof Kozlowski 130 0.9% Jens Axboe 124 0.9% Arnaldo Carvalho de Melo 121 0.8% Arnd Bergmann 120 0.8% Geert Uytterhoeven 120 0.8% Ville Syrjälä 109 0.8% Kuninori Morimoto 106 0.7% Alex Deucher 91 0.6% Takashi Iwai 90 0.6% Andy Shevchenko 89 0.6% Tony Lindgren 86 0.6% Andrii Nakryiko 85 0.6% zhengbin 83 0.6% Ben Dooks 78 0.5% Dmitry Torokhov 76 0.5%

By changed lines Ard Biesheuvel 24006 3.6% Haiyan Song 20182 3.0% Chris Wilson 13598 2.0% Dmitry Osipenko 12745 1.9% Hao Zheng 11252 1.7% Christoph Hellwig 10652 1.6% Jérôme Pouiller 10605 1.6% Potnuri Bharat Teja 9746 1.5% Jason A. Donenfeld 8656 1.3% Jiaxun Yang 6554 1.0% Mauro Carvalho Chehab 6240 0.9% Bhawanpreet Lakha 5908 0.9% Jens Axboe 5709 0.8% Thierry Reding 5208 0.8% Vladimir Oltean 4960 0.7% Zaibo Xu 4849 0.7% Adrian Hunter 4668 0.7% Andrii Nakryiko 4571 0.7% Nuno Sá 4516 0.7% Brendan Higgins 4402 0.7%

One of the most reliable ways to get into the list of top contributors, it seems, is to work on a graphics driver; it is thus not surprising that Chris Wilson contributed the most changesets entirely through work on the Intel i915 driver. Christoph Hellwig did a lot of work in the XFS filesystem, the block layer, and the RISC-V architecture code. Yue Haibing and Colin Ian King both contributed cleanup patches all over the tree, while Thierry Reding worked mostly on the Tegra graphics driver.

In the "lines changed" column, Ard Biesheuvel worked almost entirely in the crypto subsystem; much of that work was aimed at enabling the merging of the WireGuard VPN code into 5.6. Haiyan Song contributed exactly two patches updating perf events data for Intel CPUs. Dmitry Osipenko worked on Tegra hardware support, and Hao Zheng contributed one big patch to the Marvell octeontx2 network driver.

The testing and reviewing numbers this time around look like this:

Test and review credits in 5.5
Tested-by Andrew Bowers 73 8.2% Arnaldo Carvalho de Melo 46 5.2% Keerthy 21 2.4% Adam Ford 21 2.4% Yoshihiro Shimoda 17 1.9% Peter Geis 14 1.6% Hannes Reinecke 12 1.3% Stan Johnson 12 1.3% Aaron Brown 12 1.3% Sean Nyekjaer 11 1.2% Randy Dunlap 11 1.2%	Reviewed-by Darrick J. Wong 205 3.5% Rob Herring 176 3.0% Chris Wilson 142 2.4% Christoph Hellwig 115 1.9% Tvrtko Ursulin 109 1.8% Alex Deucher 94 1.6% David Sterba 87 1.5% Andrew Lunn 84 1.4% Daniel Vetter 73 1.2% Christian König 62 1.0% Greg Kroah-Hartman 61 1.0%

Only 797 changesets (5.5% of the total) carried Tested-by tags, while 4,939 changesets (34% of the total) had Reviewed-by tags. Two of the top testers, Andrew Bowers and Keerthy focus on testing Intel and TI-specific driver patches, respectively. The top reviewer, Darrick Wong, applied the Reviewed-by tag to XFS patches that, as the XFS maintainer, he also signed off on; most other subsystem maintainers do not follow that practice.

The most prolific bug reporters (and those who credited them) in this cycle were:

Reported-by credits in 5.5
Recipients Hulk Robot 164 15.7% Syzbot 125 12.0% kbuild test robot 102 9.8% Dan Carpenter 32 3.1% Linus Torvalds 14 1.3% Stephen Rothwell 12 1.2% Geert Uytterhoeven 8 0.8% Randy Dunlap 8 0.8% kernel test robot 8 0.8% Qian Cai 8 0.8% Yauheni Kaliuta 8 0.8% Arnaldo Carvalho de Melo 7 0.7% Johan Hovold 7 0.7% Christophe Leroy 7 0.7% coverity-bot 7 0.7%	Creditors zhengbin 83 8.0% YueHaibing 69 6.6% Eric Dumazet 32 3.1% Jens Axboe 26 2.5% Chris Wilson 22 2.1% Jérôme Pouiller 18 1.7% Paul E. McKenney 16 1.5% Takashi Iwai 12 1.2% Florian Westphal 11 1.1% Frederic Weisbecker 11 1.1% Andrii Nakryiko 10 1.0% Linus Torvalds 9 0.9%

These numbers would indicate that over 1/3 of the bug reports for the kernel (of which 934 were credited in 5.5) are now coming from automated testing systems. Some of the reported bugs are more severe than others, but there is little doubt that having automated systems finding hundreds of bugs (that are subsequently fixed) each development cycle is good for the kernel as a whole.

A total of 231 companies (that we know about) support work on the 5.5 kernel. The most active employers this time around were:

Most active 5.5 employers

By changesets Intel 1655 11.5% (Unknown) 999 7.0% Red Hat 945 6.6% (None) 804 5.6% Google 780 5.4% AMD 709 4.9% Huawei Technologies 586 4.1% SUSE 549 3.8% Linaro 500 3.5% IBM 458 3.2% (Consultant) 395 2.8% Renesas Electronics 384 2.7% Facebook 359 2.5% NXP Semiconductors 336 2.3% Mellanox 304 2.1% Samsung 240 1.7% Arm 234 1.6% Texas Instruments 218 1.5% Canonical 197 1.4% NVIDIA 193 1.3%

By lines changed Intel 95531 14.2% (Unknown) 47298 7.0% Red Hat 34064 5.1% Arm 33500 5.0% Google 31617 4.7% (None) 27912 4.2% AMD 26259 3.9% Linaro 25838 3.8% (Consultant) 23358 3.5% Marvell 20777 3.1% SUSE 18321 2.7% IBM 17923 2.7% Facebook 17628 2.6% Samsung 14863 2.2% NXP Semiconductors 14197 2.1% Chelsio 13322 2.0% Renesas Electronics 12943 1.9% Huawei Technologies 11292 1.7% NVIDIA 10899 1.6% Mellanox 10704 1.6%

The employer table is generally unsurprising, and this month is no exception.

A walk on the stable side

The 4.9 kernel was released on December 11, 2016, just over three years ago. Some 16,214 non-merge changesets went into the 4.9 release. Since then, as of this writing, there have been 210 stable updates to 4.9, adding another 15,210 changesets — enough to make up another large development cycle. That is a lot of fixes, and they are not all small: 4.9.210 is 80,000 lines larger than 4.9 was.

The contributor picture for this long-term stable kernel is somewhat different than it was at release time. The following are the most active contributors to the updates — the releases that came out after 4.9:

Most active 4.9-stable contributors

Individuals Greg Kroah-Hartman 301 2.0% Eric Dumazet 285 1.9% Johan Hovold 224 1.5% Arnd Bergmann 216 1.4% Takashi Iwai 184 1.2% Dan Carpenter 174 1.1% Thomas Gleixner 141 0.9% Eric Biggers 121 0.8% Xin Long 93 0.6% Hans de Goede 81 0.5% Geert Uytterhoeven 80 0.5% Mark Rutland 78 0.5% Colin Ian King 70 0.5% Will Deacon 69 0.5% Cong Wang 69 0.5% Bart Van Assche 68 0.4% Dan Williams 68 0.4% Gustavo A. R. Silva 65 0.4% Peter Zijlstra 64 0.4% Theodore Ts'o 64 0.4%

Companies Red Hat 1353 8.9% Google 1344 8.8% (None) 1243 8.2% (Unknown) 1135 7.5% Intel 1044 6.9% SUSE 745 4.9% IBM 669 4.4% Oracle 462 3.0% Linaro 452 3.0% (Consultant) 425 2.8% Linux Foundation 418 2.7% Huawei Technologies 301 2.0% Mellanox 284 1.9% Arm 258 1.7% Broadcom 189 1.2% Samsung 187 1.2% Canonical 181 1.2% Linutronix 165 1.1% Renesas Electronics 149 1.0% AMD 147 1.0%

Greg Kroah-Hartman's position at the top of the individual list is a bit deceptive; over 200 of those commits are simply setting the version numbers for each stable release. Of the remaining commits, 50 are reverts of "stable" changes that proved to be not such a good idea. Take those commits out, and he would not have made it into the top 20 changeset contributors.

The picture that emerges in general is one containing many long-term contributors to the core kernel; they may not be the top contributors from one release to the next, but they are creating the important fixes that we all depend on.

Only 2,775 of the fixes merged between 4.9 and 4.9.210 contain a Reported-by tag; that is about 18% of the total. Given that most of the commits in this series are meant to be bug fixes, that suggests that a lot of bug reports are still going without credit. The reporters who were credited for fixes going into 4.9.x were:

Top 4.9.x bug reporters
Syzbot	471
Dmitry Vyukov	98
Andrey Konovalov	79
Dan Carpenter	52
kbuild test robot	32
Hulk Robot	30
Fengguang Wu	26
Jianlin Shi	26
Ben Hutchings	24
Jann Horn	23
Al Viro	18
Guenter Roeck	18
Wen Xu	17
Arnd Bergmann	14
Eric Biggers	14
Anatoly Trosinenko	11
Alexander Potapenko	10
Li Shuang	10
Eric Dumazet	9
Tetsuo Handa	9
Pali Rohár	9

Bearing in mind that Syzbot and Dmitry Vyukov are one and the same in this context, it is clear that the Syzbot robot, in particular, has been highly effective in finding bugs that have been in the kernel for a long time. The same is true, to a lesser degree, for Fengguang Wu and his "kbuild test robot".

The process of developing and stabilizing a kernel clearly does not abruptly end the day that Linus Torvalds releases it; one might argue that much of the real work has just begun. Getting developers to focus on fixing bugs rather than developing new ones has been a struggle for almost as long as there has been software; the good news is that, over the years, the development community has gotten better at doing that work. The development process now continues at a high speed, even after a new kernel is released and the focus moves on to the next development cycle.

Index entries for this article
Kernel	Releases/5.5

---

to post comments