How Pruning Reshapes Features: Sparse Autoencoder Analysis of Weight-Pruned Language Models

1. 1Apply weight pruning to language models using magnitude and Wanda methods
2. 2Use Sparse Autoencoders as interpretability probes to analyze the reshaped feature geometry
3. 3Evaluate the effects of pruning on three model families: Gemma 3 1B, Gemma 2 2B, and Llama 3.2 1B
4. 4Analyze the results to understand how pruning reshapes the internal representations of language models

AI engineers and ML researchers benefit from this study as it provides insights into the effects of pruning on language models, which can inform model compression and interpretability strategies

💡 Weight pruning significantly alters the feature geometry of language models, which can impact their performance and interpretability

🤖 Pruning reshapes language models' features! Researchers use Sparse Autoencoders to analyze the effects of weight pruning on internal representations