Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a post-training refinement method for speech foundation models that utilizes interventional contrastive learning to disentangle the general-purpose representations into distinct content and speaker subspaces. By applying an interventional dataset and a multi-part contrastive loss, the authors effectively enhance the model's ability to focus on task-specific features relevant for downstream applications. The results demonstrate significant improvements in out-of-domain speaker verification performance and confirm the successful separation of speaker and content information in the learned representations.
Interventional post-training transforms speech models to distinctly separate content and speaker representations, leading to enhanced performance in speaker verification tasks.
Speech foundation models, pre-trained on large corpora of unlabelled speech data, produce general-purpose representations which are useful across tasks. However, these representations encode information about salient speech variables in a distributed manner, while downstream speech tasks rely on only some of this variability. In this work, we propose a post-training refinement approach using interventional contrastive learning. By leveraging an interventional dataset and multi-part contrastive loss, we learn a transformation from the entangled representation space of speech foundation models into separate content and speaker subspaces. We evaluate the learnt representations on speaker verification and keyword spotting tasks, showing improved out-of-domain speaker verification performance and evidence that speaker and content information are separated across the learned subspaces.