The Geometry of Multi-Task Grokking: Transverse Instability, Superposition, and Weight Decay Phase Structure

1. 1Train shared-trunk Transformers on multi-task objectives
2. 2Conduct systematic weight decay sweeps to analyze phase structure
3. 3Examine the emergence of staggered grokking orders and transverse instability
4. 4Investigate the role of superposition in multi-task grokking
5. 5Analyze the weight decay phase structure to understand generalization
6. 6Apply findings to improve model performance in multi-task settings

AI engineers and ML researchers can benefit from understanding the geometric analysis of multi-task grokking to improve model performance and generalization

💡 Geometric analysis of multi-task grokking can help improve model performance and generalization

🚀 New research on multi-task grokking in Transformers reveals consistent phenomena across task settings!