Preferred form of modification [LWN.net]

Preferred form of modification

Posted Mar 10, 2026 19:31 UTC (Tue) by NYKevin (subscriber, #129325)
In reply to: Preferred form of modification by excors
Parent article: Debian decides not to decide on AI-generated contributions

Unfortunately, while this is *mostly* true, the probabilities may suffer from numerical stability issues. Each probability falls out of a lengthy sequence of matrix multiply etc. operations. Any of the following changes may invalidate existing seeds:

* Changing whether or not -ffast-math is enabled.
* Compiling with -ffast-math under a different compiler (or different version of the same compiler)
* Linking against a different BLAS library (or different version of the same BLAS).
* Compiling on any platform that fails to uphold IEEE 754 (other than as a result of -ffast-math). Cursory Googling suggests that "modern" GPUs "generally" uphold IEEE754.
* Changing whether or not the hardware can do fused multiply-add (FMA), and/or whether or not the BLAS is smart enough to take advantage of it.

Probably there are others as well, these are just the obvious ones.

TL;DR: Seeds are reproducible assuming we're talking about a specific binary running on specific hardware. In all other cases, you have to audit a lot of miscellaneous stuff to ensure reproducibility.

to post comments

Preferred form of modification

Posted Mar 10, 2026 21:53 UTC (Tue) by excors (subscriber, #95769) [Link]

That seems no different to any other reproducible builds - you have to use largely the same compiler and compiler flags and hardware architecture etc. (And don't use -ffast-math in any case, because it's explicitly documented as producing incorrect output.)

I think the tricky part of non-determinism in LLMs is there's insufficient synchronisation in their parallel GPU code, so the order of some non-associative FP arithmetic depends on their dynamic load balancing and the GPU's non-deterministic thread scheduling. That's a deliberate tradeoff of performance against reproducibility, and they could choose to implement it the other way if they cared. (https://thinkingmachines.ai/blog/defeating-nondeterminism... has some plausible-sounding discussion of the changes needed to make it deterministic.)