STOP Using AdamW, EVERYONE Uses Muon Now

Vuk Rosić · Advanced ·📄 Research Papers Explained ·2mo ago
github - https://github.com/vukrosic/muon-optimizer-guide/blob/main/course/muon_optimizer_guide.md Become AI Researcher - https://www.skool.com/become-ai-researcher-2669/about --- X - https://x.com/VukRosic99 Discord (do AI research with us) - https://discord.gg/6AbXGpKTwN Full AI Research Course - https://youtu.be/44l-xL7OKLo 0:00 Why Use Muon 0:24 Muon Training Experiments 1:03 Basic Weight Updates 1:33 The AdamW Problem 3:11 How Muon Works 4:55 Muon Usage Rules 5:19 Code And Implementation 6:23 Robotics AI Testing
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Related AI Lessons

⚡
The ABCs of reading medical research and review papers these days
Learn to critically evaluate medical research papers by accepting nothing at face value, believing no one blindly, and checking everything
Medium · LLM
 ⚡
#1 DevLog Meta-research: I Got Tired of Tab Chaos While Reading Research Papers.
Learn to manage research paper tabs efficiently and apply meta-research techniques to improve productivity
Dev.to AI
 ⚡
How to Set Up a Karpathy-Style Wiki for Your Research Field
Learn to set up a Karpathy-style wiki for your research field to organize and share knowledge effectively
Medium · AI
 ⚡
The Non-Optimality of Scientific Knowledge: Path Dependence, Lock-In, and The Local Minimum Trap
Scientific knowledge may be stuck in a local minimum, hindering optimal progress, and understanding this concept is crucial for advancing research
ArXiv cs.AI

Chapters (8)

Why Use Muon
0:24 Muon Training Experiments
1:03 Basic Weight Updates
1:33 The AdamW Problem
3:11 How Muon Works
4:55 Muon Usage Rules
5:19 Code And Implementation
6:23 Robotics AI Testing
Up next
Microsoft Research Forum | Season 2, Episode 4
Microsoft Research
Watch →