Episode
Mamba, Mamba-2 and Post-Transformer Architectures for Generative AI with Albert Gu - #693
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) · Jul 17, 2024 · 58 min
Today, we're joined by Albert Gu, assistant professor at Carnegie Mellon University, to discuss his research on post-transformer architectures for multi-modal foundation models, with a focus on state-space models in general and Albert’s recent Mamba and Mamba-2 papers in particular. We dig into the efficiency of the attention mechanism and its limitations in handling high-resolution perceptual…