Episode
Next-Token Predictor Is An AI's Job, Not Its Species
Astral Codex Ten Podcast · Apr 2, 2026 · 16 min
I. In The Argument, Kelsey Piper gives a good description of the ways that AIs are more than just "next-token predictors" or "stochastic parrots" - for example, they also use fine-tuning and RLHF. But commenters, while appreciating the subtleties she introduces, object that they're still just extra layers on top of a machine that basically runs on next-token prediction. I want to approach this…