Hamiltonian Mechanics of Feature Learning: Bottleneck Structure in Leaky ResNets

1. 1Understand the concept of Leaky ResNets and their interpolation between ResNets and Fully-Connected nets
2. 2Study the 'representation geodesics' $A_{p}$ in representation space
3. 3Apply Lagrangian and Hamiltonian reformulation to analyze the bottleneck structure in Leaky ResNets
4. 4Analyze the results to gain insights into feature learning and neural network optimization

Machine learning researchers and engineers working on neural network architectures can benefit from this study to improve their understanding of feature learning and representation geometry

💡 Hamiltonian mechanics can be used to understand the bottleneck structure in Leaky ResNets and optimize feature learning

💡 Hamiltonian mechanics sheds light on feature learning in Leaky ResNets