Stable kernel 5.15.5 [LWN.net]

Stable kernel 5.15.5

Posted Nov 29, 2021 6:33 UTC (Mon) by roc (subscriber, #30627)
In reply to: Stable kernel 5.15.5 by birdie
Parent article: Stable kernel 5.15.5

Really, "users" shouldn't be using upstream "stable" kernels at all. Fixes are pulled from master into "stable" at high rates with minimal testing. Regular severe regressions can only be expected.

E.g., Linus pulled the SA_IMMUTABLE changes that broke debugging into 5.16 master on Nov 10. Those changes pulled into 5.15.3 and released just 8 days later. How much testing or other scrutiny could there have been? It's sheer luck that 5.16rc1 was released on Nov 14, Kyle ran the rr test suite against it a few days later and discovered everything was broken.

I suggest that the upstream "stable" kernels be relabeled "maintenance" and disclaimers attached to indicate that they are not suitable for production use. I think making quality downstream's problem is a mistake, but since that's how it is, better be honest about it.

Stable kernel 5.15.5

Posted Nov 29, 2021 16:50 UTC (Mon) by mat2 (guest, #100235) [Link] (2 responses)

> Those changes pulled into 5.15.3 and released just 8 days later.

The problem is that patches Cced stable are sent to the stable review process the moment they appear in Linus git tree. There should be some way of adding a configurable delay, for example like this:

> Cc: stable@vger.kernel.org #delay: 2 weeks

so that they are considered for inclusion into stable 2 weeks after being merged in master.

I have a similar problem now: I have submitted a patch that fixes a regression (since 5.11) in the CMOS RTC code ( https://lkml.org/lkml/2021/11/19/796 ), but this area looks tricky and I am not 100% sure it will not cause problems on other systems. So I opted not to Cc it stable, but will likely sent it to stable manually some time later.

Stable kernel 5.15.5

Posted Nov 29, 2021 18:27 UTC (Mon) by Wol (subscriber, #4433) [Link]

The other problem is the bots that cc stable. There have been cases where EXPERIMENTAL code has been cc'd stable by a bot and the author has been horrified when they found out.

Cheers,
Wol

Stable kernel 5.15.5

Posted Dec 1, 2021 15:47 UTC (Wed) by Wol (subscriber, #4433) [Link]

> > Those changes pulled into 5.15.3 and released just 8 days later.

> The problem is that patches Cced stable are sent to the stable review process the moment they appear in Linus git tree. There should be some way of adding a configurable delay, for example like this:

But that sounds like those were changes pulled in the merge window. There's NO WAY those changes should be fed straight to stable. There's a good chance they're new features, which are explicitly not supposed to go to stable. When they went to Linus, were they flagged as cc-stable by the AUTHOR? And if not, why the hell were they picked up like that?

If stuff hasn't been cc'ed stable by the author, it shouldn't go there for at least 2 rc cycles, imho.

Cheers,
Wol

Stable kernel 5.15.5

Posted Nov 29, 2021 17:35 UTC (Mon) by andresfreund (subscriber, #69562) [Link]

> E.g., Linus pulled the SA_IMMUTABLE changes that broke debugging into 5.16 master on Nov 10. Those changes pulled into 5.15.3 and released just 8 days later. How much testing or other scrutiny could there have been? It's sheer luck that 5.16rc1 was released on Nov 14, Kyle ran the rr test suite against it a few days later and discovered everything was broken.

For my/our Postgres AIO work I have set up a script to build VM images that use the tip of Linus and Axboe trees (daily), as kernel which then are used for the actual CI runs (daily + changes). Might make sense to do something similar for rr? Happy to provide a few pointers.