Konstantin Schuerholt - Weight Space Learning Treating Neural Network Weights as Data

Neural network weights encode the learned knowledge of trained models, making them a fascinating object of study. The emerging field of “Weight Space Learning” treats these weights as data, opening new possibilities for analyzing models without access to data, fusing knowledge, and generating individualized models. As a consequence of growing momentum, this year’s ICLR will hold its first workshop dedicated to Weight Space Learning, aiming to unify approaches and define new research goals. Despite the promise, working directly with weights poses unique challenges, from the sheer dimensionality of parameter spaces over space mismatches and invariances. This talk will survey the opportunities and challenges in Weight Space Learning, give an overview of existing methods, and present SANE, an approach that learns representations of network weights for more insightful model analysis and generation. Konstantin Schürholt is an AI researcher at Ndea, an intelligence science lab dedicated to deep learning–guided program synthesis. Previously, he did a postdoc at the University of St. Gallen, where he also earned his PhD in Computer Science in 2024. In his PhD, Konstantin focused on representation learning on neural network weights, exploring latent structures in populations of neural networks for both model analysis and generation. Further, he worked on phase transitions in neural networks as well as using graph neural networks to analyze power grids. Konstantin was co-advised by Damian Borth (University of St. Gallen), Michael Mahoney (UC Berkeley), and Xavier Giro-i-Nieto (UPC Barcelona). As part of his PhD, he visited ICSI Berkeley as a visiting scholar and interned at Google DeepMind. This session is brought to you by the Cohere Labs Open Science Community - a space where ML researchers, engineers, linguists, social scientists, and lifelong learners connect and collaborate with each other. We'd like to extend a special thank you to Anier Velasco Sotomayor, Thang Chu,