Good, bad and probabilistic

This is indeed both good and bad and frustrating since you cannot non-probabilistically rerun a language model with a tweaked prompt and observe the effects.

Now I think we need to separate two axis of probabilistic here. Getting different results every time you run LLMs is *not* a fundamental property of LLMs. It's just evaluation of a function. Nothing non-deterministic about that. Now, we often intentionally inject randomness (called temperature)—it tends to make the outputs better for many kinds of tasks—but even that is, in principle, captured by a single seed and rerunning the same LLM with the same temperature and the same seed gives you the same results. So it's non-deterministic in the sense of a Python program that sets the random seed to 42 and returns the same results every time, using randomness from that.

That's the theory at least. *Practically*, how we run LLMs makes them less repeatable, but this is really just a question of us choosing to optimize running them in a way that introduces noise. You can think of it as analogous to how FPUs alternate between rounding up or down, combined with how in threaded code scheduling decisions can change which rounding got used for which operation.

Again, I think we *could* choose to make it completely deterministic. There probably would be a price; I suspect it wouldn't be very large in computational terms, but it's likely there and someone would need to do the work (I'd also expect it to require deep enough knowledge of the accelerators that it may need vendor support).