My NEW AI Research Results Seem CRAZY
Become AI Researcher - https://www.skool.com/become-ai-researcher-2669/about
---
github - https://github.com/vukrosic/qk_norm_collapse
Novita is giving 50% OFF on GPUs (4090, 5090, H100, B200…), juse select *spot billing*. If you use our affiliate link, Novita will gift compute to our open-source AI project ❤️ - https://novita.ai/?ref=mjqyndm&utm_source=affiliate
X - https://x.com/VukRosic99
Discord (do AI research with us) - https://discord.gg/6AbXGpKTwN
Full AI Research Course - https://youtu.be/44l-xL7OKLo
0:00 QK Norm Research
0:23 Dimension Collapse Explained
0:56 Key Vector Normalization
1:26 Gamma Scaling Impact
2:22 Better Loss Results
3:22 Orthogonal Heads Strategy
3:58 Sharing Research Publicly
4:57 Muon Conflict Hypothesis
5:59 Final Thoughts
Watch on YouTube ↗
(saves to browser)
Sign in to unlock AI tutor explanation · ⚡30
Related AI Lessons
⚡
⚡
⚡
⚡
The ABCs of reading medical research and review papers these days
Medium · LLM
#1 DevLog Meta-research: I Got Tired of Tab Chaos While Reading Research Papers.
Dev.to AI
How to Set Up a Karpathy-Style Wiki for Your Research Field
Medium · AI
The Non-Optimality of Scientific Knowledge: Path Dependence, Lock-In, and The Local Minimum Trap
ArXiv cs.AI
Chapters (9)
QK Norm Research
0:23
Dimension Collapse Explained
0:56
Key Vector Normalization
1:26
Gamma Scaling Impact
2:22
Better Loss Results
3:22
Orthogonal Heads Strategy
3:58
Sharing Research Publicly
4:57
Muon Conflict Hypothesis
5:59
Final Thoughts
🎓
Tutor Explanation
DeepCamp AI