Search papers, labs, and topics across Lattice.
The paper addresses the challenge of effectively evaluating self-supervised learning (SSL) embeddings for audio models, where fine-tuning is currently favored over probing due to the limitations of simple probing methods. They introduce Convex Gated Probing (CGP), a prototype-based probing method that utilizes a gating mechanism to leverage all frozen layers and identify task-relevant information. Guided by CGP, the authors optimize the SSL pipeline, including data preprocessing, model architecture, and pre-training, to develop Better Audio Transformer (BAT), achieving state-of-the-art results on audio benchmarks.
Audio SSL models can now be evaluated with probing, not just fine-tuning, thanks to a new method that closes the performance gap and reveals where task-relevant information hides in the embeddings.
Probing is widely adopted in computer vision to faithfully evaluate self-supervised learning (SSL) embeddings, as fine-tuning may misrepresent their inherent quality. In contrast, audio SSL models still rely on fine-tuning because simple probing fails to unlock their full potential and alters their rankings when competing for SOTA on AudioSet. Hence, a robust and efficient probing mechanism is required to guide the trajectory of audio SSL towards reliable and reproducible methods. We introduce Convex Gated Probing (CGP), a prototype-based method that drastically closes the gap between fine-tuning and probing in audio. CGP efficiently utilizes all frozen layers via a gating mechanism and exposes the location of latent task-relevant information. Guided by CGP, we rework the entire SSL pipeline of current SOTA audio models that use legacy implementations of prior SSL methods. By refining data preprocessing, model architecture, and pre-training recipe, we introduce Better Audio Transformer (BAT), and establish new SOTA on audio benchmarks.
