Search papers, labs, and topics across Lattice.
This paper introduces DeSRPA, a novel framework for creating Speech Role-Playing Agents (SRPAs) that utilizes inference-time intervention on frozen model backbones to enhance character immersion. By employing a dual-level control vector mechanism鈥擨nternal Cognitive Steering and External Expressive Rendering鈥擠eSRPA effectively synchronizes cognitive reasoning with paralinguistic nuances, addressing the limitations of end-to-end fine-tuning approaches. Experimental results on the SpeechRole and OmniCharacter benchmarks reveal that DeSRPA outperforms traditional methods in personality and emotional consistency while achieving high speech naturalness comparable to advanced proprietary models.
DeSRPA achieves superior emotional consistency and speech naturalness in SRPAs without the need for extensive retraining, challenging the dominance of end-to-end fine-tuning approaches.
While Large Language Models (LLMs) have revolutionized text-based role-playing, creating immersive Speech Role-Playing Agents (SRPAs) requires a seamless bridge between cognitive reasoning and paralinguistic nuances. Current SRPAs primarily rely on end-to-end (E2E) fine-tuning. However, this paradigm suffers from poor generalization to unseen characters due to its reliance on role-specific data, while imposing a "modality alignment tax" that degrades intrinsic LLM reasoning capabilities. We propose DeSRPA, an agentic framework for character role play via inference-time intervention on frozen backbones. DeSRPA employs a dual-level control vector mechanism, Internal Cognitive Steering and External Expressive Rendering, to synchronize "mind" and "voice". Experiments on SpeechRole and OmniCharacter benchmarks demonstrate that DeSRPA significantly outperforms E2E baselines in personality and emotional consistency. It achieves high speech naturalness, narrowing the gap with proprietary models like GPT-4o Audio, while remaining a scalable and training-free paradigm.
