Andrej Karpathy
10 HN recommenders · recommended in 14 months, 2025–2026
3:31:24 · 10M views · Feb 5, 2025
Andrej Karpathy's three-hour-plus general-audience deep dive into how ChatGPT-style LLMs are built, covering the full training stack from pretraining to post-training. It also gives mental models for their quirks, like hallucination, and how to use them well.
HN's most frequent recommendation for a clear, approachable foundation in how LLMs work without heavy math.
18 comments from Hacker News
Depending on how much time you have, the first thing I would recommend is to watch Karpathy's 'deep dive into LLMs' video ( this video ). It is 3.5 hours but it will give you so much instinctive understanding of how these things work and will likely save you a lot of time. Purely from a technical standpoint, things aren't that difficult really. I'd recommend get yourself an API account with all major providers (because each frontier LLM evidently has their own strengths and weaknesses) and start experimenting with your data. The feeling you'll get when you first get an API response will keep you going! :) Having said that, a caveat is, LLMs do hallucinate from time to time. If your data needs accuracy, definitely don't do the "throw everything at LLMs and let them do the work" but I'm sure you'll figure it out from trial and error. Good luck!
Since you're posting here, you're looking for the shortcut. The shortcut is Karpathy's "Let's Build GPT: from scratch, in code, spelled out" video: [LINK] Then there is a good video that dives into LLMs and how they work that is quite approachable: this video From there, flesh out knowledge with his other videos, where he goes both extremely light and extremely deep: [LINK] Anyway, I really like's Karpathy's video because he's very good at explaining LLMs at every level.
This is like the LLM era version of the search bubble that prevented people from having the same search results for ostensibly identical searches. Also keep in mind that LLMs are stochastic by design. If you haven't seen it, Karpathy's excellent "deep dive into LLMs like chatgpt" video[0] explains and demonstrates this aspect pretty well: [0] this video
I highly recommend Andrej Karpathy's Deep Dive into LLMs like ChatGPT: this video . It's a 3.5 hour general intro to the topic that I think does a great job of making sense of the tech and how it works under the hood, without needing to build up a foundation of math and programming.
I think you'll get a lot of use from this video from Andrej Karpathy: this video It is long, but don't get scared off. He goes over a ton of different stuff related to model training, but makes it very easy to understand.