Andrej Karpathy
9 HN recommenders · recommended in 10 months, 2023–2026
59:48 · 4.3M views · Nov 23, 2023
Andrej Karpathy gives a one-hour general-audience introduction to large language models, covering what they are, how they are trained, comparisons to operating systems, and security challenges such as jailbreaks.
HN recommends it as the dense, approachable starting point for understanding LLMs and often cites it for specific claims about scaling and training.
20 comments from Hacker News
Short answer is No. I highly recommended watching Andrej Karpathy's Intro to LLMs talk, particularly the section on System 1 vs System 2 thinking. Long story short, what you are describing, using more processing to prepare a better response, is something that is an area of interest, but is not currently part of ChatGPT (or any other LLM that I am aware of). See: this video
Andrej Karpathy's "Zero to Hero" series on YouTube is the ultimate guide to building LLMs. Extremely information-dense but as complete as it gets: [LINK] Also, an amazing high-level overview of LLMs, including extensive discussion about attack vectors, that he published a couple days ago: this video
All of Andrej Karpathy's stuff on YouTube is absolutely top tier. He's a rare example of a brilliant contributor in his field who is an equally brilliant educator. Start with his short intro to LLMs, only 1 hour long but is dense with valuable stuff: this video
My view is to focus on doing stuff. That's what I did. Pick up some task you want the model to do, try finetuning llama, playing with APIs from OpenAI, etc. Googling and asking GPT along the way. Foundational model training got so expensive that unless you can get hired by "owns nuclear power plant of GPUs" you are not going to get any "research" done. And as the area got white-hot those companies have more available talent than hardware nowadays. So just getting into the practitioner area is the best way to get productive with those models. And you improve as a practitioner by practicing, not by reading papers. If you're at the computer, your time is best spent writing code and interacting with those models in my opinion. If you cannot (e.g. commute) I listen to some stuff (e.g. this video - Anything from Karpathy on youtube, or [LINK] channel).
Hey, imho best overall technical intro to LLMs (I guess that´s your main interest as you mentioned qlora + llama) is by Simon Willis [1]. Additionally or if you prefer videos, the recent 1h "busy persons intro" by Andrei Karpathy is great + dense as well [2]. [1] [LINK] [2] this video EDIT: Maybe I misunderstood as you asked about papers, not general intros. I don´t think that reading papers is the best way to "catch up" as the pace is rapid and knowledge very decentralized. I can confirm what Andrej recently wrote on X [3]: "Unknown to many people, a growing amount of alpha is now outside of Arxiv, sources include but are not limited to: - [LINK] - HN - that niche Discord server - anime profile picture anons on X - reddit" [3] [LINK]