Thinking Machines Blog

Yes - THIS! I can't even believe how exhausting it is to read. I'm not sure why or what changed in Fable. Did they do this writing-style output to give it more token compression during/for training or to prefer output for less money?

I love it for a few things, but it's gotten really hard to spend any extended amount of time with it because of the lack of mental model I seem to be able to hold while working with complicated problems.

I'm guessing it's just not enough time doing RL on human feedback.

Check out the anouncement of Inkling (https://thinkingmachines.ai/news/introducing-inkling/)... the section in the middle

"Early in RL verbose, grammatical" (if you search) :

We need to understand the operator.
The 5D line element is ds² = e^{2A(x)} (ds²_4d + dx²), where A(x) = sin(x) + 4 cos(x), x in [0, 2π].
The internal coordinate is periodic.
The background is a warped product: metric g_{MN} where M,N = 0..4.
The internal direction has metric e^{2A(x)} dx²?
Wait, the ds² is e^{2A} (ds²_4d + dx²).
So the internal metric is e^{2A(x)} dx².
Actually if the total metric is ds² = e^{2A(x)} (ds²_4d + dx²), then yes, internal metric is e^{2A} dx².

vs. Post RL

We need determine eigenvalue problem for spin-2 fluctuations h_{μν}(x,y) with TT in 4d and depend on x.
For metric of form ds² = e^{2A(x)} (g_{μν}(y) + h_{μν}(y,x)) dy^μ dy^ν + e^{2A(x)}?
Wait internal metric is e^{2A} dx²?
Actually ds² = e^{2A} [ds_4² + dx²].
So internal metric is e^{2A} dx²; warp factor same for 4d and…

Determinism in agents is a complex topic because there are several different layers of abstraction, each of which may introduce its own non-determinism. But yeah, it is going to be difficult to induce determinism in a commercial coding agent, for reasons discussed below.

However, we can start by claiming that non-determinism is not necessarily a bad thing - non-greedy token sampling helps prevent certain degenerate/repetitive states and tends to produce overall higher quality responses [0]. I would also observe that part of the yin-yang of working with the agents is letting go of the idea that one is working with a "compiler" and thinking of it more as a promising but fallible collaborator.

With that out of the way, what leads to non-determinism? The classic explanation is the sampling strategy used to select the next token from the LLM. As mentioned above, there are incentives to use a non-zero temperature for this, which means that most LLM APIs are intentionally non-deterministic by default. And, even at temperature zero LLMs are not 100% deterministic [1]. But it's usually pretty close; I am running a local LLM as we speak with greedy sampling and the result is predictably the same each time.

Proprietary reasoning models are another layer of abstraction that may not even offer temperature as knob anymore[2]. I think Claude still offers it, but it doesn't guarantee 100% determinism at temperature 0 either. [3]

Finally, an agentic tool loop may encounter different…

They reference https://thinkingmachines.ai/blog/on-policy-distillation/

If I understand correctly, it's distillation via having a teacher model score each of the student's tokens for a problem based on their own probabilities of generating that token at each step in the sequence. The reward/loss is then applied as RL.

The multi-teacher bit seems to imply they're distilling from multiple models. It's light on the details, but it seems like it could be part of distilling from frontier/closed models. Provided they calculate the logprobs, which OpenAI seems to allow via API but not Anthropic. Maybe they have a way of estimating the logprobs externally?

This method can be used to learn any domain from the teacher. Biology included.

> This isn't true

From what I understand, in practice it often is true[1]:

Matrix multiplication should be “independent” along every element in the batch — neither the other elements in the batch nor how large the batch is should affect the computation results of a specific element in the batch. However, as we can observe empirically, this isn’t true.

In other words, the primary reason nearly all LLM inference endpoints are nondeterministic is that the load (and thus batch-size) nondeterministically varies! This nondeterminism is not unique to GPUs — LLM inference endpoints served from CPUs or TPUs will also have this source of nondeterminism.

[1]: https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/

After the Thinking Machines announcement (https://thinkingmachines.ai/blog/interaction-models/) I've become interested in the ideas behind speech models, and the next "step-function" In Agent<>Human interaction.

Was able to spin this up over the weekend. Here's how it works: GPT-Realtime-2 is the frontend. It has a tool to start a (Claude Code/Codex) task, and a tool to check the progress of said task.

The agents run in an async worker while the Realtime loop can continue chatting. It becomes fairly straightforward to manage several agent runs at once, as if you are having a conversation.

castform founder here. while it's "rare for a specialized model to beat a strong general model" today, i think the tech/knowhow on how to do so is getting there. we see some early signs of this in industry e.g.

https://shopify.engineering/sidekicks-continual-learning-loop
https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment-in-financial-tasks/
https://castform.com/blog/elsa/

at the end of the day, models are only as good as the data they're trained on. and if one has access to proprietary data, it should yield specialized models that do better than larger general models