Hefty stable kernel updates for Friday [LWN.net]

Greg Kroah-Hartman has announced the release of the 7.1.5, 6.18.40, 6.12.97, 6.6.145, 6.1.178, 5.15.212, and 5.10.261 stable Linux kernels.

This batch of kernels includes a hefty set of updates, possibly the largest ever. 7.1.5-rc1, for example, included more than 2,000 patches, 6.18.40-rc1 included 1,611 patches, and so forth. Users are advised to upgrade.

to post comments

Regressions?

Posted Jul 24, 2026 18:02 UTC (Fri) by abatters (✭ supporter ✭, #6932) [Link] (6 responses)

Is there any word on recent trends on -stable regressions? Just wondering how the flood of AI-found bugs is affecting stability.

Regressions?

Posted Jul 25, 2026 1:24 UTC (Sat) by gerdesj (subscriber, #5446) [Link] (4 responses)

"-stable regressions" - I have no idea what that is but I suggest you forget about it. The world has spun and changed somewhat. GKH and co will do their best but you are ultimately responsible for your stuff:

"This batch of kernels includes a hefty set of updates, possibly the the largest ever. 7.1.5-rc1, for example, included more than 2,000 patches, 6.18.40-rc1 included 1,611 patches, and so forth. Users are advised to upgrade."

We will have to change our work practices. If your gear is pets and not cattle then you will have to consider converting to cattle (that's my dilemma too).

I cannot keep up with patching systems by hand any more. I am going to have to automate (ansible obvs) a lot of things which don't want to be automated Have you seen how awful MS updates are for automation? apt is a breeze!

Regressions?

Posted Jul 25, 2026 8:08 UTC (Sat) by ballombe (subscriber, #9523) [Link] (2 responses)

If every kernel bug is a security bug, then every kernel regression is a security bug too, so the OP question is very relevant.

Regressions?

Posted Jul 25, 2026 9:40 UTC (Sat) by hailfinger (subscriber, #76962) [Link] (1 responses)

Ah yes, the problem of quickly estimating the risk of moving from a known-vulnerable but stable kernel to a no-known-vulnerable but untested kernel. Being an admin is definitely less fun today.

Side note: I have seen the same problem in application software as well. A security update comes with additional changes breaking production, and you either have to work around the breakage (if possible) or revert to the vulnerable version. The Linux kernel is one of the better managed "stable" pieces of software. I regularly see "update to latest git of component ABC" in updates for stable branches of application software, followed by the same comment a few weeks later because the previous update broke things.

Regressions?

Posted Jul 27, 2026 7:27 UTC (Mon) by taladar (subscriber, #68407) [Link]

What people fail to understand is that "stable" versions + backports are also untested versions and the pool of people testing those before you is actually much smaller than for the upstream versions.

Regressions?

Posted Jul 26, 2026 20:31 UTC (Sun) by geofft (subscriber, #59789) [Link]

I don't understand what you are trying to say.

Of course your infrastructure should be cattle and not pets to the extent possible. But if there is a bug introduced in 7.1.5 and 7.1.4, then all your excellent automation will immediately deploy that bug fleet-wide. Is that not a bad outcome?

The question here is not about the work of applying the patches but about the contents of the patches. How are we confident that the patches bring benefits and not problems?

Of course you are ultimately responsible for your stuff. The question is how does one exercise that responsibility? Is the expectation to have an LLM read through all the patches and identify those that might be incorrect, perhaps biased by knowledge of your workload and kernel config (there's no point in looking carefully at a fix to a driver for hardware you don't have in a module that is never loaded), and to have a second pair of local eyes on them before pressing the Ansible button?

For that matter, has the world spun and changed enough that we shouldn't bother with stable kernels? Stable tagging has been LLM-driven for quite a while. Backporting a patch is a relatively mechanical and uncreative process, one that definitely benefits from domain expertise and an understanding of what the original patch is intended to do and how, but a rather different kind of work from actually writing the patch in the first place, more deterministic both in the work and in reviewing it for correctness. Is it still worth waiting for the stable team or should we exercise responsibility for our own stuff by following mainline and automatically backporting everything relevant?

Or, in the other direction, given the sort of automation and testabilityneeded to operate machines at scale, why stick with a particular minor version? Why not run the tip of linux.git, again reviewing all the patches for problems that may arise in the local environment?

Re: Regressions?

Posted Jul 30, 2026 12:13 UTC (Thu) by fratti (subscriber, #105722) [Link]

Answering this requires some effort, since at best people will be giving you anecdotal evidence (i.e. not representative), at worst they'll give you second-hand anecdotal evidence or completely irrelevant soapboxing as you already got.

I think the best barometers are KernelCI for automated testing, and Debian bug reports for user-reported regressions.

KernelCI doesn't seem to show historical data over time for branches on its dashboard. However, it's what stable kernel releases already use as a barometer for whether a kernel is good to be released. There's some pretty huge blind spots though.

KernelCI only receives test reports for what someone wrote a test for, and for hardware specific code (like drivers!), KernelCI can only receive tests from labs that have that hardware and exercise it. Automating that is not easy to do; it's already a bit of an adventure to get consumer hardware to automate clean boots of fresh kernels over the network regardless of prior device state (like hard lockups) while collecting kernel logs, and it's an even bigger hassle to then test things like hot-plugging USB-C display outputs or connecting to wireless APs or bluetooth devices, so I doubt anyone does this.

Most automated testing also doesn't have infinite time to burn, so whether a Wi-Fi chipset now drops connection after 4 hours due to an obscure new interaction between driver and blob firmware that regressed isn't something that can really be covered either.

Debian ships stable releases and user-reported regressions will have more luck catching this stuff, but it'll suffer from different insufficient coverage and harder to identify causes.

So the data is out there, someone just needs to do some data collection to compare pre-LLM-reports to current.