Search papers, labs, and topics across Lattice.
This paper introduces PS4, a novel proxy-supervised training framework designed to enhance target speaker extraction (TSE) in real conversational mixtures, addressing the scarcity of large-scale training corpora and clean target speech. The authors constructed a comprehensive dataset of 71,771 samples from multiple public datasets, encompassing both Chinese and English, and employed a joint training strategy that fine-tunes a BSRNN-based TSE model through four differentiable objectives. The results demonstrate that PS4 ranks 2nd on the REAL-T challenge leaderboard, achieving superior performance in speaker similarity and timing F1 metrics compared to other systems.
Achieving top-tier performance in speaker extraction from real conversations with a novel training framework that leverages proxy supervision and a large-scale dataset.
Training target speaker extraction (TSE) models for real conversational mixtures remains challenging because large-scale training corpora and clean target speech for supervision are unavailable. We present PS4, a proxy-supervised training framework for TSE in real conversational mixtures, with two main contributions. First, we construct a large-scale corpus of 71,771 training samples derived from four public datasets, covering both Chinese and English scenarios. Each sample contains an overlapping speech mixture, per-speaker enrollment audio, a ground-truth transcript, and frame-level voice activity labels. Second, we propose a proxy-supervised joint training strategy that fine-tunes a BSRNN-based TSE model using four complementary differentiable objectives: ASR cross-entropy, speaker similarity, frame-level voice activity detection, and perceptual audio quality. Starting from a publicly available pre-trained checkpoint, only the BSRNN separator is updated during fine-tuning. On the REAL-T challenge leaderboard, PS4 ranks 2nd overall, achieving the best speaker similarity and timing F1 among all submitted systems.
