Prompt quality? [LWN.net]

Prompt quality?

Posted Apr 23, 2026 17:30 UTC (Thu) by Curan (subscriber, #66186)
Parent article: Firefox: The zero-days are numbered

The main question here is: what are the prompts and would a "normal" developer/project be able to have similarly good prompts? Looking at the whole LLM-aided patch review process for the kernel, I seriously doubt, that normal projects could manage the same results – unless, of course, Anthropic is providing those input modifiers alongside their model. So far there seems to be a "thriving" (personally I would tend to say "profiteering") ecosystem of resellers of access to these LLMs, that offer specialised prompts.

Apart from that: none of this is deterministic. That is a really big issue. Though admittedly worse on the generation side, I think.

to post comments

Prompt quality?

Posted Apr 23, 2026 22:38 UTC (Thu) by Paf (subscriber, #91811) [Link] (1 responses)

But human security research is highly non-deterministic too?

Prompt quality?

Posted Apr 23, 2026 23:10 UTC (Thu) by Curan (subscriber, #66186) [Link]

I think, you focus on the wrong part of my comment, since I explicitly stated, that this is mostly an issue on the generation side.

Though, I would say, that having an automated system, that generates ten different answers between three runs is worse than a few humans, that actually have to think about a review. Even if several teams would find different things or the same time does. Anyway: this is the wrong focus, in my opinion.