Search papers, labs, and topics across Lattice.
This paper introduces ParaIntent, a novel benchmark for spoken emotional dialogue that categorizes user intent into 14 distinct categories, addressing the dual nature of intent expression through explicit lexical content and implicit paralinguistic cues. To optimize policy learning, the authors propose Acoustic-Lexical Decoupled Policy Optimization (ALPO), which separates the learning signals for text and speech, allowing for more effective reinforcement learning. The results show that ALPO outperforms standard methods in both automatic metrics and subjective evaluations, particularly enhancing emotional expressiveness in generated responses.
ALPO achieves significant improvements in emotional expressiveness by decoupling text and speech learning signals, revealing the potential for more nuanced spoken dialogue systems.
Spoken emotional dialogue requires a model to understand a user's spoken input and generate a response that is both semantically appropriate and emotionally expressive. This is challenging because communicative intent may be stated explicitly in lexical content or conveyed more implicitly through paralinguistic cues, which can complement or diverge from the words themselves. However, two limitations constrain progress in this area: the scarcity of benchmarks that distinguish these intent expressions, and the lack of reinforcement learning objectives that jointly account for response quality and emotional expression. To address the lack of suitable benchmarks, we introduce ParaIntent, a Chinese benchmark comprising 14 intent categories with balanced explicit and implicit samples, together with a multidimensional evaluation protocol covering intent fulfillment, response quality, and emotional expression. For policy optimization, existing approaches either use a shared objective for text and speech or apply reinforcement learning to only one modality, leaving modality-specific learning signals entangled within policy optimization. Motivated by this, we propose Acoustic-Lexical Decoupled Policy Optimization (ALPO), which computes independent textual and acoustic advantages and routes them to the corresponding text and speech tokens within a unified rollout. Under identical reward functions and training budgets, ALPO improves over standard GRPO on most automatic metrics and achieves the best subjective results among the fine-tuned variants, with particularly clear gains in emotional expressiveness on both the synthetic and human-recorded test sets.
