Interpreting Video Representations with Spatio-Temporal Sparse Autoencoders

1. 1Apply standard Sparse Autoencoders (SAEs) to video representations to decompose them into interpretable features
2. 2Use spatio-temporal contrastive objectives to improve temporal coherence
3. 3Implement Matryoshka hierarchical grouping to further enhance feature assignments and autocorrelation

Machine learning researchers and engineers working on video analysis tasks can benefit from this study to improve their models' performance and interpretability

💡 Spatio-temporal contrastive objectives and hierarchical grouping can recover and exceed raw temporal coherence in video representations

📹 Improve video representation interpretation with Spatio-Temporal Sparse Autoencoders!