Search papers, labs, and topics across Lattice.
This paper introduces a Cross-Feature Knowledge Distillation (CFKD) framework that enhances automatic speaker verification (ASV) performance by leveraging discrete audio tokens from neural audio codecs (NACs). The authors demonstrate that while these tokens inherently preserve speaker cues, traditional ASV training methods fail to exploit this potential effectively. Their experiments on the VoxCeleb benchmarks reveal that CFKD significantly narrows the performance gap between codec-based systems and conventional Fbank-based models, underscoring the viability of discrete audio representations in speaker verification tasks.
Discrete audio tokens can rival traditional spectral features in speaker verification when guided by a robust knowledge distillation framework.
Neural audio codecs (NACs) enable efficient audio compression and have achieved success in downstream tasks such as speech synthesis. However, their discrete representations consistently underperform traditional spectral features in automatic speaker verification (ASV). We empirically demonstrate that speaker cues are implicitly preserved in discrete tokens but remain underutilized by conventional ASV training paradigms. To address this, we propose a Cross-Feature Knowledge Distillation (CFKD) framework. By guiding the codec-based student to mimic the embedding space of a strong Fbank-based teacher, CFKD provides structured supervision for effective utilization of speaker information in tokens. Experiments on the VoxCeleb benchmarks show that CFKD substantially improves the ASV performance of codec-based systems, allowing them to approach the accuracy of Fbank-based teacher models and highlighting the potential of discrete audio tokens for diverse speech tasks.
