Multi-head Latent Attention (MLA)

What if you could cut your transformer’s KV cache by over 90% without touching your GPU? In this video, we break down how DeepSeek’s Multi-Head Latent Attention (MLA) completely changes the game for long-context LLMs by aggressively compressing keys and values into a tiny latent space—while keeping model quality essentially unchanged. multi head latent attention multi-head latent attention deepseek mla deepseek multi head latent attention deepseek kv cache kv cache optimization llm kv cache explained long context llm efficient attention mechanisms mha vs mqa vs gqa multi query attention group…