Preferred form of modification [LWN.net]

Preferred form of modification

Posted Mar 10, 2026 16:35 UTC (Tue) by koverstreet (✭ supporter ✭, #4296)
In reply to: Preferred form of modification by neggles
Parent article: Debian decides not to decide on AI-generated contributions

No, it won't. You can run an LLM that way - temperature = 0 - but you generally don't want to. Like in many other algorithms, introducing some stochastic noise often produces better results.

to post comments

Preferred form of modification

Posted Mar 10, 2026 16:54 UTC (Tue) by geofft (subscriber, #59789) [Link]

There's a difference between setting the temperature to zero, i.e. deterministically taking the most-likely token (basically changing softmax to regular-old deterministic max), and using a PRNG with a deterministic seed with a non-zero temperature, which will sometimes take less-likely tokens but will make that decision in the same way for every re-execution of the same network (with the same hardware, resources, etc.) with the same seed. I agree that setting the temperature to zero is probably not what you want.

Preferred form of modification

Posted Mar 10, 2026 17:34 UTC (Tue) by phm (subscriber, #168918) [Link]

No, it won't. You can run an LLM that way - temperature = 0 - but you generally don't want to. Like in many other algorithms, introducing some stochastic noise often produces better results.

Given the same settings (temperature, model, RNG seed) an LLM will produce the same results. Here are some example sessions run with llama.cpp (LLM is mdradermacher's quantization i1 of Apertus 8B, abliterated) on a Thinkpad.


t420:~/llama.cpp/build/bin$ ./llama-cli --temp 20 --seed 12345 $LLM

> Hello!

Hello! It's great to connect with you. If you have a question or a [^C]

# Running the same command again:

t420:~/llama.cpp/build/bin$ ./llama-cli --temp 20 --seed 12345 -m $LLM

> Hello!

Hello! It's great to connect with you [^C]

./llama-cli --temp 20000000 --seed 12345 -m $LLM

> Hello!

Hello! Welcome to the SwissAI assistant service. What can I help [^C]