Let's not exaggerate

I guess it's the second part of your original comment that appears not to fit. "Especially since neural networks often use f16 or even f8 precision." The reasons you describe for nondeterminism don't seem that they would affect smaller floating point types any worse than bigger ones.