Steering Sparse Autoencoder Latents to Control Dynamic Head Pruning in Vision Transformers (Student Abstract)

1. 1Train a Sparse Autoencoder on the final-layer embeddings of a Vision Transformer
2. 2Use the sparse latents to steer dynamic head pruning
3. 3Evaluate the performance of the pruned model
4. 4Refine the pruning policy based on the results

ML researchers and engineers working on Vision Transformers can benefit from this framework to improve model efficiency and interpretability, and software engineers can apply this technique to optimize their models

💡 Integrating Sparse Autoencoders with dynamic pruning can improve the interpretability and controllability of Vision Transformers

🤖 Improve Vision Transformer efficiency with Sparse Autoencoders and dynamic head pruning!