Proprietary vs open source models [LWN.net]

Proprietary vs open source models

Posted May 25, 2026 23:36 UTC (Mon) by mpsk (subscriber, #183534)
Parent article: Reviewing kernel patches with LLMs

I'm wondering, if there's any discussion ongoing regarding usage of open source (and perhaps open weight) LLMs vs the closed ones?

Linux is a FOSS project itself; one could think it'd naturally prefer open solution over closed ones. Yet, in the most of Linux-related news (here and elsewhere), if any specific model name appears, it's usually one of the proprietary ones (that's my impression, may be wrong, I can't read everything).

Sure, I'm aware that open source LLMs tend to lag behind (due to training costs) and that employing proprietary tools in the kernel development isn't anything new. But still, I'm curious whether this topic surfaces anywhere in the developers' discussions?

to post comments

Proprietary vs open source models

Posted May 26, 2026 0:02 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (12 responses)

Open models are significantly inferior to proprietary ones. The lag is probably around a year, which is an eternity for AI right now.

But I think it's just a case of Linux developers not really following the AI field closely. Unless you're deep in the field, it's just looks like total chaos from outside.

Proprietary vs open source models

Posted May 26, 2026 1:45 UTC (Tue) by gmprice (subscriber, #167884) [Link] (9 responses)

Looks like total chaos from the inside too, to be clear.

It's not worth dumping excessive amount of effort into stabilizing systems at this point because every 6 months there's a new frontier model update that outperforms the last one and the old prompts are just bad and not optimized for the new model.

Feels like negative value to dump a ton of time into developing prompts that won't be any good a few months from now.

Proprietary vs open source models

Posted May 26, 2026 3:00 UTC (Tue) by marcH (subscriber, #57642) [Link] (4 responses)

It's good to start somewhere. Also, it sounds like what already exists is good and valuable enough to be shared and used, even if that's just for a few months.

> Feels like negative value to dump a ton of time into developing prompts that won't be any good a few months from now.

A ton of time: no. But merely sharing what already exists to get the ball rolling: yes.

Proprietary vs open source models

Posted May 26, 2026 7:28 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]

The current open models are good but still require quite a bit of hand-holding and/or expensive hardware. E.g. you need something like $80k to buy hardware to run Kimi locally with adequate performance. Qwen 3.6 runs well on a laptop but requires a lot of hand-holding and time.

But even with all that, there's really no lack of enthusiasts experimenting with open models. The whole field reminds me of the mid-2000s explosion of open source and Internet-related projects. But faster moving and with a lot more people.

It just takes some time to filter down to the level of kernel development. It's by nature very conservative.

Proprietary vs open source models

Posted May 26, 2026 12:48 UTC (Tue) by csamuel (✭ supporter ✭, #2624) [Link] (2 responses)

> It's good to start somewhere. Also, it sounds like what already exists is good and valuable enough to be shared and used, even if that's just for a few months.

Strong agree, and I think one definite advantage to getting things in now is made by Jon here:

> Kernel documentation maintainer, Jonathan Corbet, said that the prompts looked like ""a whole lot of very useful documentation on how to understand and review kernel patches"", though it is ""really sad that we couldn't write it until we were writing it for a machine"". He wondered what that kind of documentation might have enabled had it been added to the kernel long before now.

If what is written helps humans do this work, then we are benefiting the community, not (just) the machines.

All the best,
Chris

Proprietary vs open source models

Posted May 26, 2026 15:23 UTC (Tue) by kleptog (subscriber, #1183) [Link] (1 responses)

> "really sad that we couldn't write it until we were writing it for a machine"

To some extent I can understand this. Writing documentation is also a process that requires feedback. You write documentation, you want to know from users whether this helped them or not, you improve it. But the feedback process for documentation is not easy to arrange and slow.

If you write the documentation for an LLM you can get direct feedback as to whether it understands it or not, whether important things are missing. This makes it way more interesting to write. I'm not sure we could have done it earlier.

Proprietary vs open source models

Posted May 26, 2026 18:12 UTC (Tue) by iabervon (subscriber, #722) [Link]

Also, humans tend to be affected by the previous version of the documentation and your reactions to the inaccurate statements they made about your code after reading it, in a way that prevents really checking whether it will work to educate new humans yet. With LLMs, you can just start a new session, and you don't need to worry about having confused and then discarded a potential contributor last time.

Proprietary vs open source models

Posted May 26, 2026 7:32 UTC (Tue) by jepsis (subscriber, #130218) [Link] (3 responses)

New LLM models do not inherently require new prompts. Prompts are natural language instructions, and language itself does not change significantly within a few months. The core meaning of a well-written prompt remains the same across model updates. Even language itself is not the key factor, since modern LLMs are multilingual and a few months of English evolution is hardly enough to make old prompts suddenly incomprehensible. Differences between models come from improved instruction-following and reasoning, not from any need to rewrite prompts for each new release.

Proprietary vs open source models

Posted May 26, 2026 12:32 UTC (Tue) by foom (subscriber, #14868) [Link] (2 responses)

> Differences between models come from improved instruction-following and reasoning, not from any need to rewrite prompts for each new release.

But that is why you need to rewrite prompts.

For model A, you had to say something stronger like "absolutely do not under any circumstances write X", in order to approach the desired outcome of only writing X when really needed. Because model A isn't good at following instructions.

But then you upgrade to model Y, which is better at following instructions, and now it interprets your existing prompt to _actually_ never do X. Even when it "should".

Proprietary vs open source models

Posted May 26, 2026 13:56 UTC (Tue) by paulj (subscriber, #341) [Link]

These new programming languages are so awesome. The future is bright.

Proprietary vs open source models

Posted May 26, 2026 15:17 UTC (Tue) by gmprice (subscriber, #167884) [Link]

Not only that, having "X" anywhere in the prompt at all can actually cause issues if you don't want "X".

Compaction occurs, summary messes up, "X" was in the prompt but you lose the emphasis on not doing "X", and suddenly it starts doing "X" because X is in the context.

Proprietary vs open source models

Posted Jun 3, 2026 14:28 UTC (Wed) by re:fi.64 (subscriber, #132628) [Link] (1 responses)

I don't think it's really a full year? This time last year we had just gotten Opus / Sonnet *4*, which I feel have actually been surpassed on average. A lot of the 12-18mo timelines I've seen seem pretty tied to only evaluating DSv4 Pro, which seems rather undertrained (probably spent a bit too much time on their architecture?).

Proprietary vs open source models

Posted Jun 3, 2026 23:59 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

I'm being a bit conservative here. There's also Mythos which is more capable than Opus, and there are rumors that OpenAI has something similar almost ready.

In any case, if you're not really interested in the field itself, it just makes sense to wait for it to settle at least a bit.

Proprietary vs open source models

Posted May 26, 2026 4:15 UTC (Tue) by mrchuck (subscriber, #62450) [Link] (1 responses)

I don’t recall any discussion of the “open v. proprietary model” issue during the sessions. The same category of discussion came up 20 years ago when git was invented to replace a proprietary source control system (BitKeeper) to track kernel changes, and it still makes sense now to have this conversation about LLM coding tools.

I’m aware of a few kernel developers looking into the idea of using open models, however. Certainly available open models are good enough to handle simpler coding tasks, but currently the proprietary models are unmatched at handling complex problems and analysis that is sometimes needed for kernel development. As the economics of purchasing and running hardware at home with sufficiently capable models evolves, we should see people showing up with recipes and success stories. I recently saw a video that demonstrated using Hermes to automatically route prompts to a local open models or a fee-based API, based on prompt complexity — but this was an AI special-interest demonstration, not a kernel development set-up.

Proprietary vs open source models

Posted May 26, 2026 12:01 UTC (Tue) by SLi (subscriber, #53131) [Link]

I feel the difference between Bitkeeper and proprietary LLMs is that there's realistically no lock-in, especially at the pace where things are moving now where you can speculate that a random large enough model, proprietary or open, a year or two from now will neatly run "the same program" and do it better. I find this structurally very different from making a development workflow fundamentally locked to a proprietary version control solution.

Proprietary vs open source models

Posted May 26, 2026 12:53 UTC (Tue) by daenzer (subscriber, #7050) [Link] (1 responses)

> Linux is a FOSS project itself; one could think it'd naturally prefer open solution over closed ones.

I for one would hope that should go without saying.

> [...] employing proprietary tools in the kernel development isn't anything new.

The only obvious thing that comes to mind is BitKeeper, which was a rather unpleasant experience. (One might argue that getting Git out of it was worth it, not sure we can expect something similar this time around though)

This quote from the article seems a propos:

> He said that Anthropic is willing to give tokens to kernel maintainers and, he believes, Google is also willing to do that.

Similarly, BitKeeper was giving away free licences to kernel developers for some time.

Proprietary vs open source models

Posted May 26, 2026 13:16 UTC (Tue) by bluca (subscriber, #118303) [Link]

The difference here is that there isn't really much specific to one model or another, besides the glue in the tool to interact with each, that users shouldn't even be aware of. The fact that one can already switch from one model to the next relatively seamlessly should be sufficient proof. The prompts are just prompts.

Proprietary vs open source models

Posted May 26, 2026 13:48 UTC (Tue) by Wol (subscriber, #4433) [Link]

Would we be better off using an SLM and training it on the kernel source, documentation, and the C standard?

Certainly with minority languages (DataBASIC cough cough) I've heard a fair few reports that that approach actually gets pretty good results. An LLM just gets terminally confused with all the variants of BASIC out there ...

Cheers,
Wol