Language Models Can Explain Visual Features via Steering

1. 1Leverage the structure of Vision-Language Models to identify individual features
2. 2Apply steering to SAE features to generate explanations
3. 3Use causal interventions to analyze the relationship between language and visual features
4. 4Evaluate the effectiveness of the steering method in explaining visual features

AI researchers and engineers working on computer vision and natural language processing tasks can benefit from this approach to better understand and interpret visual features, and it can be applied by ml-researchers and ai-engineers in the development of more transparent and explainable AI models

💡 Language models can be used to explain visual features without requiring human intervention via steering

💡 Language models can explain visual features via steering! #AI #ComputerVision #NLP