Search papers, labs, and topics across Lattice.
This paper introduces a two-stage text-guided target speech extraction (TSE) framework that leverages inter-speaker relative cues to improve target speaker selection. The framework first uses a speech separation model to generate candidate sources and then employs a text-guided classifier to select the target speaker based on embedding similarity, using either relative or independent cues. Experiments demonstrate that relative cues lead to higher classification accuracy and improved TSE performance compared to independent cues, and the two-stage approach outperforms single-stage methods.
Relative cues like "who spoke first" or "who is louder" dramatically boost text-guided speech extraction, even outperforming systems that rely solely on audio.
This paper investigates the use of relative cues for text-based target speech extraction (TSE). We first provide a theoretical justification for relative cues from the perspectives of human perception and label quantization, showing that relative cues preserve fine-grained distinctions often lost in absolute categorical representations. Building on this analysis, we propose a two-stage TSE framework, in which a speech separation model generates candidate sources, followed by a text-guided classifier that selects the target speaker based on embedding similarity. Using this framework, we train two separate classification models to evaluate the advantages of relative cues over independent cues in terms of both classification accuracy and TSE performance. Experimental results demonstrate that (i) relative cues achieve higher overall classification accuracy and improved TSE performance compared with independent cues, (ii) the two-stage framework substantially outperforms single-stage text-conditioned extraction methods on both signal-level and objective perceptual metrics, and (iii) certain relative cues (language, gender, loudness, distance, temporal order, speaking duration, random cue and all cue) can surpass the performance of an audio-based TSE system. Further analysis reveals notable differences in discriminative power across cue types, providing insights into the effectiveness of different relative cues for TSE.
