Search papers, labs, and topics across Lattice.
This paper introduces an automatic framework for detecting speaker drift in synthesized speech by framing it as a binary classification task of utterance-level speaker consistency. The method leverages cosine similarity between overlapping segments of speech and prompts LLMs with structured representations of these similarities to assess drift. The authors provide theoretical guarantees for cosine-based drift detection and validate their approach on a new, human-annotated synthetic benchmark, demonstrating the effectiveness of their embedding-to-reasoning pipeline.
Speaker drift, a subtle but pervasive flaw in modern TTS, can now be automatically detected by prompting LLMs with geometric representations of speaker embeddings.
Recent diffusion-based text-to-speech (TTS) models achieve high naturalness and expressiveness, yet often suffer from speaker drift, a subtle, gradual shift in perceived speaker identity within a single utterance. This underexplored phenomenon undermines the coherence of synthetic speech, especially in long-form or interactive settings. We introduce the first automatic framework for detecting speaker drift by formulating it as a binary classification task over utterance-level speaker consistency. Our method computes cosine similarity across overlapping segments of synthesized speech and prompts large language models (LLMs) with structured representations to assess drift. We provide theoretical guarantees for cosine-based drift detection and demonstrate that speaker embeddings exhibit meaningful geometric clustering on the unit sphere. To support evaluation, we construct a high-quality synthetic benchmark with human-validated speaker drift annotations. Experiments with multiple state-of-the-art LLMs confirm the viability of this embedding-to-reasoning pipeline. Our work establishes speaker drift as a standalone research problem and bridges geometric signal analysis with LLM-based perceptual reasoning in modern TTS.
