Search papers, labs, and topics across Lattice.
This paper introduces Joycent, a novel diffusion-based text-to-speech (TTS) model that synthesizes accented speech directly from standard phone sequences without the need for accented phone prediction. By leveraging conditional layer normalization (CLN) to integrate accent and speaker representations, Joycent addresses the limitations of existing two-stage systems that suffer from error accumulation and require extensive paired data. Experimental results demonstrate that Joycent significantly enhances the accentedness of synthesized speech while maintaining speaker identity, outperforming traditional baseline methods.
Joycent synthesizes accented speech directly from standard phone sequences, eliminating the need for error-prone accented phone predictions.
Accent text-to-speech (TTS) aims to synthesize speech with target accents. Existing accent TTS systems typically rely on a two-stage pipeline that first converts standard phone sequences into accented phone sequences and then synthesizes accented speech. However, such approaches suffer from error accumulation and require paired standard-accented phone sequence data, which is often limited in practice. Moreover, text-based accented phone representations are insufficient to model acoustic accent characteristics such as prosody and rhythm. In this work, we propose Joycent, a diffusion-based accent TTS model that synthesizes accented speech directly from standard phone sequences and speech references without accented phone prediction. Joycent integrates accent and speaker representations through conditional layer normalization (CLN) in the text encoder. We introduce WhisAID, a Mandarin accent identification model trained on accented Mandarin speech to extract accent representations. Experimental results show that Joycent improves accentedness while preserving speaker identity compared with baseline systems. We release our code and demos at: https://github.com/oshindow/Joycent-code.
NUS