Search papers, labs, and topics across Lattice.
This paper introduces TRACE, a novel framework for detecting emotional entrainment in dyadic speech by modeling interactions as ordered sequences of acoustic embeddings derived from emotion fine-tuned Whisper representations. By leveraging a new dataset, DyadEE, which includes both naturally entrained and disrupted conversations, the authors demonstrate that incorporating conversational context and social relationship information significantly enhances detection accuracy. TRACE achieves a remarkable accuracy of 97.01%, underscoring the importance of temporal dynamics in understanding emotional interactions in conversational AI.
Emotional entrainment detection can reach 97.01% accuracy by considering the temporal dynamics of dyadic speech interactions.
With the proliferation of speech AI agents, understanding emotional entrainment in conversational interaction has become increasingly important. Emotional entrainment is shaped by social relationships and conversational context, influencing affective coordination over time. We introduce DyadEE, a dataset for emotional entrainment detection in dyadic speech interactions, containing both emotionally entrained conversations and synthetic interactions where entrainment is disrupted through partner swapping and emotion resynthesis. We further propose TRACE, a window-level framework that models dyadic interaction as ordered sequences of acoustic embeddings derived from emotion fine-tuned Whisper representations, treating each sample as an interaction trace rather than pooled utterances. Experimental results on DyadEE show that incorporating conversational context and relationship information improves emotional entrainment detection, with TRACE achieving the best accuracy of 97.01%.
