Preferred form of modification [LWN.net]

Preferred form of modification

Posted Mar 10, 2026 16:56 UTC (Tue) by excors (subscriber, #95769)
In reply to: Preferred form of modification by kleptog
Parent article: Debian decides not to decide on AI-generated contributions

> LLMs are by their nature non-deterministic (though I guess you could turn down the temperature).

I think that's not really true: the LLM basically does a deterministic computation of next-token probabilities, biases the probabilities based on the temperature parameter (low temperature exaggerates the high-probability tokens), then uses a PRNG to make a weighted selection of a single token to add to the prompt, and repeats. Some APIs (though not all) let you select the PRNG seed, in which case the output is usually reproducible with the same initial prompt and seed and other parameters. It seems they often have optimisations that can break the deterministic part, but that's just a quality-of-implementation issue, it's not part of the nature of LLMs.

But (as I understand it) coding assistants are not just an LLM with an input string and an output string. They mix multiple LLM sessions (including LLMs to generate prompts for other LLMs) with external tools (web search, filesystem access, etc) and with user input in a complex feedback loop. "The input to the tool" is not meaningful or useful for modification - that'd be like distributing an image as a list of Photoshop commands and brush strokes applied to a blank canvas.

to post comments

Preferred form of modification

Posted Mar 10, 2026 19:31 UTC (Tue) by NYKevin (subscriber, #129325) [Link] (1 responses)

Unfortunately, while this is *mostly* true, the probabilities may suffer from numerical stability issues. Each probability falls out of a lengthy sequence of matrix multiply etc. operations. Any of the following changes may invalidate existing seeds:

* Changing whether or not -ffast-math is enabled.
* Compiling with -ffast-math under a different compiler (or different version of the same compiler)
* Linking against a different BLAS library (or different version of the same BLAS).
* Compiling on any platform that fails to uphold IEEE 754 (other than as a result of -ffast-math). Cursory Googling suggests that "modern" GPUs "generally" uphold IEEE754.
* Changing whether or not the hardware can do fused multiply-add (FMA), and/or whether or not the BLAS is smart enough to take advantage of it.

Probably there are others as well, these are just the obvious ones.

TL;DR: Seeds are reproducible assuming we're talking about a specific binary running on specific hardware. In all other cases, you have to audit a lot of miscellaneous stuff to ensure reproducibility.

Preferred form of modification

Posted Mar 10, 2026 21:53 UTC (Tue) by excors (subscriber, #95769) [Link]

That seems no different to any other reproducible builds - you have to use largely the same compiler and compiler flags and hardware architecture etc. (And don't use -ffast-math in any case, because it's explicitly documented as producing incorrect output.)

I think the tricky part of non-determinism in LLMs is there's insufficient synchronisation in their parallel GPU code, so the order of some non-associative FP arithmetic depends on their dynamic load balancing and the GPU's non-deterministic thread scheduling. That's a deliberate tradeoff of performance against reproducibility, and they could choose to implement it the other way if they cared. (https://thinkingmachines.ai/blog/defeating-nondeterminism... has some plausible-sounding discussion of the changes needed to make it deterministic.)

Preferred form of modification

Posted Mar 10, 2026 21:21 UTC (Tue) by kleptog (subscriber, #1183) [Link]

> But (as I understand it) coding assistants are not just an LLM with an input string and an output string.

Right, because directly using the output of an LLM is like recording the output of someone's talking after asking them a question. You get the requests for clarification, side-tangents, ums and ahs, etc. You don't record someone talking, you ask them to write down their answer and let them refine it a few times. And you record that.

There are frameworks that do this, where you have the output stream of "talking" from the LLM and it also has an editor where it can write code, and rewrite it as required. But that's just making the LLM one cog in a larger machine, which makes the whole discussion about only LLMs kind of pointless.

For me LLMs feel like what happens in my mind between forming a thought and opening my mouth to produce grammatically correct sentences to convey a thought. They are a State -> Words transformer, that's all.

Preferred form of modification

Posted Mar 11, 2026 14:06 UTC (Wed) by udorsch (subscriber, #169676) [Link]

> "The input to the tool" is not meaningful or useful for modification - that'd be like distributing an image as a list of Photoshop commands and brush strokes applied to a blank canvas.

That's pretty much exactly how vector graphic formats work and vector graphics are highly efficient, widely used, and in many aspects superior to raster graphics (which would correspond to distributing the result not the input).

Preferred form of modification

Posted Mar 12, 2026 3:48 UTC (Thu) by gf2p8affineqb (subscriber, #124723) [Link]

> list of Photoshop commands and brush strokes applied to a blank canvas.

It is not! That would be a formally defined system like SVG. There is no formal definition for the semantics of an LLM, which make the input impossible to reason about or change in a predictable fashion.

Preferred form of modification

Posted Mar 13, 2026 5:34 UTC (Fri) by sramkrishna (subscriber, #72628) [Link]

It's non-deterministic because the training dictates the choices. If you the same LLM gets updated then the behavior of the LLM can possibly change. If you change the LLM then that will also change. Almost always, you will need extra prompting to add more guide rails. This is one of the weaknesses because you never know what the change will do without very extensive testing. You're going to need a lot of expertise here to do that. I reckon that is not going to be cheap.