An LLM is...

Posted Jun 27, 2025 9:17 UTC (Fri) by farnz (subscriber, #17727)
In reply to: An LLM is... by Wol
Parent article: Supporting kernel development with large language models

Also note that one of the great things that comes out when you have a program that meets the user's needs (even if it's otherwise awful - insecure, unreadable, prone to crashing off the happy path etc) is that you can write a fuzz tester to compare the two programs and tell you about differences.

If the differences are acceptable (e.g. user program crashes, yours succeeds), you can ignore them; if they're not (user program outputs a different value to yours), you can turn it into a test case, confirm that it's reasonable with the user (and not a bug in their program), and then fix this failing test in your program.

There's even people out there experimenting with using LLMs to fuzz the differences between a reimplementation and an original program.

But the key power here is having an unreliable oracle (the user's LLM-aided attempt at a program) that you can use to quickly answer questions about what the user "really" wants. That allows you to use your human intelligence to build up a reasonable set of questions to ask the user, using the oracle to answer the dumb questions.