Search papers, labs, and topics across Lattice.
This paper introduces Emo-LiPO, a novel listwise preference optimization framework that addresses the challenge of fine-grained emotion intensity control in LLM-based text-to-speech systems. By framing emotion intensity control as a learning-to-rank problem, Emo-LiPO effectively aligns the generated speech with the relative emotion intensity indicated in the text, thus bridging the semantic-acoustic gap. Experiments using the newly constructed ESD-plus dataset reveal that Emo-LiPO significantly enhances both emotion accuracy and intensity controllability, particularly excelling at higher intensity levels compared to existing baselines.
Fine-grained emotional expression in TTS systems can be dramatically improved with a learning-to-rank approach that captures global intensity ordering.
Large language model (LLM)-based text-to-speech (TTS) systems enable prompt-conditioned emotional control but struggle with fine-grained emotion intensity due to the semantic -- acoustic gap between text and speech. To address this challenge, we formulate emotion intensity control in LLM-based TTS as a learning-to-rank problem and propose Emo-LiPO, a listwise preference optimization framework that aligns prompt-conditioned speech generation with relative emotion intensity expressed in text. Emo-LiPO explicitly models global intensity ordering within each emotion under fixed transcripts, enabling more faithful and continuous emotional expression. We further construct ESD-plus, a multi-speaker dataset with explicit emotion intensity variations, to support fine-grained emotion modeling and evaluation. Experiments on ESD-plus demonstrate that Emo-LiPO significantly improves emotion accuracy and intensity controllability over both supervised- and DPO-based LLM TTS baselines, with particularly pronounced gains at high intensity levels.
