Sparse Visual Thought Circuits in Vision-Language Models
Researchers test the modularity hypothesis of sparse autoencoders in vision-language models and find it often fails, leading to output drift when intervening on multiple task-selective feature sets
- Implement sparse autoencoders (SAEs) in vision-language models to improve interpretability
- Test the modularity hypothesis by intervening on task-selective feature sets
- Evaluate the effect of intervening on multiple feature sets on reasoning accuracy and output drift
- Refine the model architecture and intervention strategies based on the findings
AI researchers and engineers working on multimodal models can benefit from this study to improve the interpretability and reasoning capabilities of their models, while data scientists and ML engineers can apply these findings to develop more effective intervention-based steering methods
💡 Intervening on multiple task-selective feature sets can induce output drift, challenging the modularity hypothesis in sparse autoencoders
🤖 Modularity hypothesis in sparse autoencoders often fails, leading to output drift #AI #ML
Key Takeaways
Researchers test the modularity hypothesis of sparse autoencoders in vision-language models and find it often fails, leading to output drift when intervening on multiple task-selective feature sets
Full Article
Abstract:
arXiv:2603.25075v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) improve interpretability in multimodal models, but it remains unclear whether SAE features form modular, composable units for reasoning-an assumption underlying many intervention-based steering methods. We test this modularity hypothesis and find it often fails: intervening on a task-selective feature set can modestly improve reasoning accuracy, while intervening on the union of two such sets reliably induces output drift
DeepCamp AI