A Minimal Model of Representation Collapse: Frustration, Stop-Gradient, and Dynamics

arXiv:2604.09979v1 Announce Type: cross Abstract: Self-supervised representation learning is central to modern machine learning because it extracts structured latent features from unlabeled data and enables robust transfer across tasks and domains. However, it can suffer from representation collapse, a widely observed failure mode in which embeddings lose discriminative structure and distinct inputs become indistinguishable. To understand the mechanisms that drive collapse and the ingredients th