LWN: Comments on "Identifying buggy patches with machine learning"

Identifying buggy patches with machine learning

Smon — Sat, 08 May 2021 17:21:58 +0000

Arch does not push mainline kernels.
They wait for x.x.1 and according to kernel.org, x.x.1 is stable. (x.x.0 is mainline)

Identifying buggy patches with machine learning

mina86 — Mon, 11 Nov 2019 00:07:10 +0000

One possible solution for late-rc patches problem is to prefer reverts over fixes during the -rc period.

Identifying buggy patches with machine learning

jezuch — Sun, 10 Nov 2019 16:56:07 +0000

It's a known problem that most of "machine learning" models are crap: https://thegradient.pub/nlps-clever-hans-moment-has-arrived/

Identifying buggy patches with machine learning

gerdesj — Sat, 09 Nov 2019 17:31:47 +0000

I'll provide a counter example: me.

Identifying buggy patches with machine learning

ajdlinux — Thu, 07 Nov 2019 01:52:09 +0000

Could we hook up autosel with the Patchwork API and see whether it's useful for identifying patches that are missing a Cc: stable before patches are applied?

Identifying buggy patches with machine learning

rweikusat2 — Wed, 06 Nov 2019 19:52:19 +0000

This still rests on the assumption that the "machine-learning model" will at least find all cases which possibly need "more review".

Which made me recall a nice (online) newspaper story of some weeks ago: Someone was using a "machine learning system" in order to help with identifying children which may end up be sexually exploited. It was fed with data of about 7000 people and produced an ordered list supposedly ranking them in order of most-to-least-likely. Of this 7000 people, 5 actually ended up being sexually exploited. 3 of these 5 were among the first 100 on the list, but considering the way this was worded, certainly not among the first 10 and very likely not even among the first 50. A fourth was among the first 200, ie, probably somewhere between 150 - 200. No information about classification of the fifth was mentioned in the article, presumably, because it was so outragously wrong that not even the developers could spin this as something positive anymore.

Put in other words: The output of the computer program was completely wrong and correlations with observable reality are probably happenstance.

Identifying buggy patches with machine learning

riking — Wed, 06 Nov 2019 16:46:29 +0000

> he said that he is currently using it to flag patches that need more careful review

IMO, this is the best use of machine learning systems: using them to sift through bunches of data and raise anomalies for human review. Systems without a human in the loop at the end are prone to silent and undetected bad biases.

Identifying buggy patches with machine learning

zdzichu — Tue, 05 Nov 2019 19:11:21 +0000

I must have misunderstood. For me, everything coming from kernel.org is "mainline", but in this discussion this adjective seem to only be used to mean 5.x.0 releases, not even -stable releases.
In the opposition of "mainline" I see so called "distro" kernels with hundreds/thousands of patches and backports.

Identifying buggy patches with machine learning

zblaxell — Tue, 05 Nov 2019 17:11:22 +0000

$ git log --oneline v5.3..v5.3.8 | wc -l
1018

"mainline + 1018 patches + whatever cherries Fedora puts on top" is not, in any literal or practical sense, "mainline". It's not "production" either. At best, it's a late-stage CI artifact, an input to downstream integration and verification.

Identifying buggy patches with machine learning

sashal — Tue, 05 Nov 2019 12:22:17 +0000

5.3.8 is a stable kernel, which is exactly the point I was trying to make: people use stable kernels in production, not Linus's tree.

Identifying buggy patches with machine learning

error27 — Tue, 05 Nov 2019 10:40:57 +0000

Reviewed-by means you reviewed the whole patch. It doesn't necessarily mean anything. If you see a Reviewed-by from me, that means it was a thank you to someone who redid their patch like I asked them to. Reviewed-by tags are not required for most of staging but I think people appreciate a little thank you note for their hard work.

Identifying buggy patches with machine learning

mst@redhat.com — Tue, 05 Nov 2019 10:01:27 +0000

There's Reviewed-by - if it's a one liner then what matters is the review not who coded it up, right?

Identifying buggy patches with machine learning

zdzichu — Tue, 05 Nov 2019 09:03:27 +0000

Fedora kernel is almost mainline (no big scary patches, mostly fixes cherrypicked for mainline), provided without a lag - at the moment of writing 5.3.8 is ready for stable F31.
And Fedora is certainly intended for production.

Identifying buggy patches with machine learning

sashal — Mon, 04 Nov 2019 19:55:47 +0000

Sure, distros provide -rc (or even "git") kernels to users, but no one actually deploys them in production.

Users *can* deploy it in production, but I believe that if you have -rc kernels deployed at scale for reasons other than testing and validating the upcoming release, you're doing something wrong.

Identifying buggy patches with machine learning

error27 — Mon, 04 Nov 2019 19:52:20 +0000

One difference between early and late patches is if you send a fix to an early patch that gets folded in there is no record of it in the git log. I've argued before that reviewer who notice a real bug should get credit.

Then the other question is for the late patches do the fixes come before or after the kernel release? If it comes before then that's fine and the system is working as designed. It's better to push those fixes to the Linus tree quite quickly so they get as much testing as possible before the release.

Identifying buggy patches with machine learning

zblaxell — Mon, 04 Nov 2019 19:27:37 +0000

Corollary: nobody runs Arch in production.

Identifying buggy patches with machine learning

darwi — Mon, 04 Nov 2019 19:19:47 +0000

> nobody runs mainline kernels in production anyway.

Arch *does* push latest kernel.org releases to users.