Search papers, labs, and topics across Lattice.
DEFINE is an end-to-end zero-shot TTS framework that disentangles speaker identity and accent by conditioning on separate audio exemplars, enabling continuous accent control via a single inference-time guidance weight. This approach, built on F5-TTS with LoRA adaptation and an exemplar encoder, achieves significant improvements in accent accuracy on seen accents and matches the accent transfer performance of cascaded TTS-voice-conversion systems for out-of-domain accents while preserving speaker similarity. The framework demonstrates independent control of speaker identity and accent from audio exemplars, even for accents not encountered during training.
Control accent strength in zero-shot TTS with a single model and inference-time weight, outperforming cascaded systems on out-of-domain accents without sacrificing speaker identity.
Zero-shot text-to-speech (TTS) can reproduce an unseen speaker from a short reference recording, but typically entangles speaker identity and accent within the same reference. We introduce DEFINE, an end-to-end framework that decouples these factors by conditioning speaker identity and target accent on separate audio exemplars. A single inference-time guidance weight continuously controls accent strength without retraining. Built on F5-TTS with parameter-efficient LoRA adaptation, DEFINE maps short accent exemplars into a conditioning space using an exemplar encoder supervised through learned accent prototypes, requiring neither accent labels at inference time nor post-synthesis waveform conversion. On seen accents, increasing accent guidance improves accent-probe accuracy from 6.5% to 19.6%. More importantly, a single DEFINE model generalizes accent control beyond its training accent set: on seen and out-of-domain accents, though not on held-out accents, it matches the accent transfer performance of a two-model TTS-voice-conversion cascade while achieving higher speaker similarity and comparable predicted speech quality. These results demonstrate that speaker identity and accent can be independently controlled from audio exemplars within a single zero-shot TTS model, including for accents unseen during training.
