S-SONDO: Self-Supervised Knowledge Distillation for General Audio Foundation Models

arXiv:2604.24933v1 Announce Type: new Abstract: General audio foundation models have recently achieved remarkable progress, enabling strong performance across diverse tasks. However, state-of-the-art models remain extremely large, often with hundreds of millions of parameters, leading to high inference costs and limited deployability on edge devices. Knowledge distillation is a proven strategy for model compression, but prior work in audio has mostly focused on supervised settings, relying on cl