Search papers, labs, and topics across Lattice.
ZHENGXIMIN135@pingan.com.cn Corresponding author. Abstract Estimating Emotional Mimicry Intensity (EMI) in naturalistic environments is a critical yet challenging task in affective computing. The primary difficulty lies in effectively modeling the complex, nonlinear temporal dynamics across highly heterogeneous modalities, especially when physical signals are corrupted or missing. To tackle this, we propose TAEMI (Text-Anchored Emotional Mimicry Intensity estimation), a novel multimodal framework designed for the 10th ABAW Competition. Motivated by the observation that continuous visual and acoustic signals are highly susceptible to transient environmental noise, we break the traditional symmetric fusion paradigm. Instead, we leverage textual transcripts—which inherently encode a stable, time-independent semantic prior—as central anchors. Specifically, we introduce a Text-Anchored Dual Cross-Attention mechanism that utilizes these robust textual queries to actively filter out frame-level redundancies and align the noisy physical streams. Furthermore, to prevent catastrophic performance degradation caused by inevitably missing data in unconstrained real-world scenarios, we integrate Learnable Missing-Modality Tokens and a Modality Dropout strategy during training. Extensive experiments on the Hume-Vidmimic2 dataset demonstrate that TAEMI effectively captures fine-grained emotional variations and maintains robust predictive resilience under imperfect conditions. Our framework achieves a state-of-the-art mean Pearson correlation coefficient across six continuous emotional dimensions, significantly outperforming existing baseline methods. 1 Introduction Emotional mimicry is a fundamental mechanism in human social interaction, wherein individuals unconsciously replicate the facial expressions, vocal prosody, and body movements of others to establish empathy and strengthen social bonds. Endowing machines with the ability to accurately perceive and quantify these dynamic emotional shifts has become a critical challenge in affective computing. Consequently, the automatic estimation of Emotional Mimicry Intensity (EMI)—a task dedicated to capturing the continuous and nuanced variations in emotional transmission—has emerged as a cutting-edge research direction with profound implications for virtual assistants, social robotics, and mental health assessment. However, existing methodologies encounter significant bottlenecks when addressing EMI estimation tasks in the wild. First, emotional mimicry is an inherently complex, multimodal temporal process. Subtle facial expressions and dynamic vocal fluctuations intertwine non-linearly over time. Current research predominantly relies on unimodal analysis or naive late-fusion strategies (e.g., simple feature concatenation), leaving the inter-modal synergy severely underutilized. For instance, baseline models utilizing ViT [2] for visual features achieve a mere 0.09 mean Pearson correlation coefficient, demonstrating that unimodal visual networks struggle to capture the temporal evolution of emotional intensity. Second, fine-grained temporal alignment requires robust cross-modal semantic understanding. While acoustic features like Wav
1
0
3
0
By using text as an anchor, this model achieves state-of-the-art emotional mimicry intensity estimation, even when visual and acoustic data are noisy or missing.