Speech and language processing
"This book offers a unified vision of speech and language processing, presenting state-of-the-art algorithms and techniques for both speech and text-based processing of natural language. This comprehensive work covers both statistical and symbolic approaches to language processing; it shows how they can be applied to important tasks such as speech recognition, spelling and grammar correction, information extraction, search engines, machine translation, and the creation of spoken-language dialog agents."--BOOK JACKET.
Stats for nerds
41recommend
56mentions
45people
2008–2026years
89% recommend11% neutral
Mentions per year
What Hacker News says
- I found the 2024 Spring CS224N course sufficient for this pre-requisite, coupled with the textbook (chapters 1-13). Like CS336, this one also has videos and assignments available, and it being from 2024 is not a problem since the basics are mostly the same as recent years. Notably this is not true for 336, which spends much more time discussing cutting edge techniques, so the 2026 version there is essential.
Course: https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1246...
Lecture videos: https://www.youtube.com/playlist?list=PLoROMvodv4rOaMFbaqxPD...
Textbook: https://web.stanford.edu/~jurafsky/slp3/
- I'll add in here https://web.stanford.edu/~jurafsky/slp3/, "Speech and Language Processing", with chapters that deal specifically with LLMs and transformers.
- Embedding „meaning“ in vector spaces goes back to 1950s structuralist linguistics and early information retrieval research, there is a nice overview in the draft for the 3rd edition of speech and language processing https://web.stanford.edu/~jurafsky/slp3/5.pdf
- bigram-trigram language models (with some smoothing tricks to allow for out-of-training-set generalization) were state of the art for many years. Ch. 3 of Jurafsky's textbook (which is modern and goes all the way to LLMs, embeddings etc.) is good on this topic.
https://web.stanford.edu/~jurafsky/slp3/ed3book_aug25.pdf
I don't know the history but I would guess there have been times (like the 90s) when the best neural language models were worse than the best trigram language models.
- I recommend having a look at 16.3 onward here if you're curious about this: https://web.stanford.edu/~jurafsky/slp3/16.pdf
I'm not familiar with Whisper in particular, but typically what happens in an ASR model is that the decoder, speaking loosely, sees "the future" (i.e. the audio after the chunk it's trying to decode) in a sentence like this, and also has the benefit of a language model guiding its decoding so that grammatical productions like "I like ice cream" are favored over "I like I scream".
- Nice that the topic gets you enthusiastic! If you want to learn about LLMs, I suggest you try educational material that doesn’t simplify it too much (i.e. not videos), before implementing something.
Personal recommendation: https://web.stanford.edu/~jurafsky/slp3/ Chapters 6-10 should give you a good understanding of these concepts, also from the mathematical point of view!
Related
Foundations of Statistical Natural Language Processing
15 recommend · 19
Natural Language Processing With Python
25 recommend · 30
Prolog and natural-language analysis
6 recommend · 11
Spoken language processing
2 recommend · 3
Introduction to information retrieval
41 recommend · 45
AI algorithms, data structures, and idioms in Prolog, Lisp, and Java
1 recommend · 8
