LLM’s Billion Dollar Problem

Check out Inngest and let your AI agents wear a harness now https://innge.st/yt-bycl-1 This video originally was going to be about the Linear Attention Saga that happened between June and November last year, but it turned out I needed quite some build up to explain what the significance of Linear Attention, 1 million context window, and compute scaling are. So it accidentally became a 17 mins video... my latest project: Intuitive AI Academy https://intuitiveai.academy/ limited time code "NYNM" for 50% off forever (only a 25 spots left!) My Newsletter https://mail.bycloud.ai/ my project: find, discover & explain AI research semantically https://findmypapers.ai/ My Patreon https://www.patreon.com/c/bycloud Sauces (will try to be in order of apperance) GPT-OSS [Code] https://github.com/openai/gpt-oss [Paper] https://arxiv.org/abs/2508.10925 (they did not mention it is sliding window attention explicitly in its paper) DeepSeek V3.2 [Paper] https://arxiv.org/pdf/2512.02556 DeepSeek V2 (Multi-head Latent Attention) [Paper] https://arxiv.org/abs/2405.04434 Kimi K2.5 [Paper] https://arxiv.org/abs/2602.02276 MiniMax [Text-01] https://arxiv.org/abs/2501.08313 [M1] https://arxiv.org/abs/2506.13585 [M2] https://huggingface.co/MiniMaxAI/MiniMax-M2 [Zhihu Blog] https://www.zhihu.com/question/1965302088260104295/answer/1966810157473335067 Qwen-3 Next [Project Page] https://qwen.ai/blog?id=4074cca80393150c248e508aa62983f9cb7d27cd Hunyuan T1 [Project Page] https://tencent.github.io/llm.hunyuan.T1/README_EN.html Kimi Linear [Paper] https://arxiv.org/abs/2510.26692 Context Arena (used for comparing long context performance) [Project Page] https://contextarena.ai/ Gemini 3 Flash [Blog] https://blog.google/products-and-platforms/products/gemini/gemini-3-flash/ Claude 4.6 [Blog] https://www.anthropic.com/news/claude-opus-4-6 Try out my new fav place to learn how to code https://scrimba.com/?via=bycloudAI This video is supported by the kind Patrons & YouTube Members: