Episode
Dynamic Token Merging for Efficient Byte-level Language Models with Julie Kallini - #724
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) · Mar 24, 2025 · 51 min
Today, we're joined by Julie Kallini, PhD student at Stanford University to discuss her recent papers, “MrT5: Dynamic Token Merging for Efficient Byte-level Language Models” and “Mission: Impossible Language Models.” For the MrT5 paper, we explore the importance and failings of tokenization in large language models—including inefficient compression rates for under-resourced languages—and dig…