My NEW AI Research Results Seem CRAZY
Become AI Researcher - https://www.skool.com/become-ai-researcher-2669/about
---
github - https://github.com/vukrosic/qk_norm_collapse
Novita is giving 50% OFF on GPUs (4090, 5090, H100, B200…), juse select *spot billing*. If you use our affiliate link, Novita will gift compute to our open-source AI project ❤️ - https://novita.ai/?ref=mjqyndm&utm_source=affiliate
X - https://x.com/VukRosic99
Discord (do AI research with us) - https://discord.gg/6AbXGpKTwN
Full AI Research Course - https://youtu.be/44l-xL7OKLo
0:00 QK Norm Research
0:23 Dimension Collapse Explained
0:56 Key Vector Normal…
Watch on YouTube ↗
(saves to browser)
Chapters (9)
QK Norm Research
0:23
Dimension Collapse Explained
0:56
Key Vector Normalization
1:26
Gamma Scaling Impact
2:22
Better Loss Results
3:22
Orthogonal Heads Strategy
3:58
Sharing Research Publicly
4:57
Muon Conflict Hypothesis
5:59
Final Thoughts
DeepCamp AI