152334H writes about machine learning internals, such as mixture-of-experts behavior in GPT-4, model training costs and AI-written kernel exploits.
HN commenters cite the post on GPT-4 non-determinism to explain why sparse mixture-of-experts models give different outputs depending on how requests are batched.
>compilers behaving in a deterministic and predictable way is an important fundamental of pipelines. LLMs are inherently unpredictable, and so using an LLM for compilation / decompilation -- even an LLM that has 99.99% accuracy
You're confusing different concepts here. An llm is technically not unpredictable by itself (at least the ones we are talking about here, there are different problems with beasts like GPT4 [1]). The "randomness" of llms you are probably experiencing stems from the autoregressive completion, which samples from probabilities for a temperature T>0 (which is very common because it makes sense in chat applications). But there is nothing that prevents you from simply choosing greedy sampling, which would make your output 100% deterministic and reproducible. That is particularly useful for disassembling/decompiling and has the chance to vastly improve over existing tools, because it is common knowledge that they are often not the sharpest tools and humans are much better at piecing together working code.
The other question here is accuracy for compiling. For that it is important whether the llm can follow a specification correctly. Because once you write unspecified behaviour, your code is fair game for other compilers as well. So the real question is how well does it follow the spec how good is it at dealing with situations where normal compilers will flounder.
[1] https://152334h.github.io/blog/non-determinism-in-gpt-4/
Nope. You have not shown how a large scale collection of neural networks irrespective of their architecture is more deterministic when compared to a 'compiler' and only repeating a known misconception of tweaking the temperature to 0 which does not bring the determinism you claim it brings with LLMs [0] [1] [2], otherwise you would not have this problem in the first place.
By even doing that, the result of the outputs are useless anyway. So this really does not help your point at all. So therefore:
> You're just using words incorrectly. Deterministic means repeatable. That's it. Predictable, verifiable, etc are tangential to deterministic.
There is nothing deteministic or predictable about an LLM even when you compare it to a compiler, unless you can guarrantee that the individual neurons through inference give a predictable output which would be useful enough for being a drop-in compiler replacement.
[0] https://152334h.github.io/blog/non-determinism-in-gpt-4/
[1] https://arxiv.org/pdf/2506.09501
[2] https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/
For Mixture of Expert models (like GPTs are), they can produce different results for an input sequence if that sequence is retried together with a different set of sequences in its inference batch, because of the model (“expert”) routing depends on the batch, not the single sequence: https://152334h.github.io/blog/non-determinism-in-gpt-4/
And in general, binary floating point arithmetic cannot guarantee associativity - i.e. `(a + b) + c` might not be the same as `a + (b + c)`. That in turn can lead to the model picking another token in rare cases (and it’s auto-regressive consequences, that the entire remainder of the generated sequence might differ): https://www.ingonyama.com/blog/solving-reproducibility-challenges-in-deep-learning-and-llms-our-journey
Edit: Of course, my answer assumes you are asking about the case when the model lets you set its token generation temperature (stochasticity) to exactly zero. With default parameter settings, all LLMs I know of randomly pick among the best tokens.
The samples your input is batched with on the provider's backend vary between calls and sparse mixture of experts routing when implemented for efficient utilization induces competition among tokens with either encouraged or enforced balance of expert usage among tokens in the same fixed-size group. I think it's unknown or at least undisclosed exactly why sequence non-determinism at zero temperature occurs in these proprietary implementations, but I think this is a good theory.
[1] https://arxiv.org/abs/2308.00951 pg. 4 [2] https://152334h.github.io/blog/non-determinism-in-gpt-4/
Your incorrect assumption here is that determinism comes with no tradeoffs. There are a few outsider analyses on the topic, for example, this one on sparse MoEs [1]. If OpenAI uses sparse MoE as described, then determinism would be possible but inefficient.
Even if it's not sparse MoE, chances are high that the non-determinism is introduced somewhere purely as a performance optimization. The article speculates that OpenAI knows this well and hides it to protect the model internals.
[1] https://152334h.github.io/blog/non-determinism-in-gpt-4/
One theory is it is caused by its Sparse MoE (Mixture of Experts) architecture [1]:
> The GPT-4 API is hosted with a backend that does batched inference. Although some of the randomness may be explained by other factors, the vast majority of non-determinism in the API is explainable by its Sparse MoE architecture failing to enforce per-sequence determinism.
[1] https://152334h.github.io/blog/non-determinism-in-gpt-4/