Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the challenge of affective behavior analysis by proposing a novel attention pooling framework that enhances the extraction of true affective cues from facial expressions while mitigating the influence of spurious factors. By incorporating causal supervision to focus attention on invariant facial regions, applying cross-covariance independence regularization, and utilizing a gated nonlinear SwiGLU transformation, the authors significantly improve the model's performance across multiple tasks. The proposed method achieves competitive results on the s-Aff-Wild2 database, with a combined performance score of 1.2214, demonstrating its effectiveness in real-world emotion recognition scenarios.
Causal supervision in attention mechanisms can drastically improve emotion recognition by focusing on invariant facial features, leading to a significant boost in predictive performance.
Affective Behaviour Analysis aims to enable machines to infer human affective states from behavioural signals, particularly facial expressions, in real-world environments. The \textit{11th Affective Behaviour Analysis in-the-wild Competition} includes the Multi-Task Learning Challenge based on the s-Aff-Wild2 database, where participants develop a unified framework for Valence-Arousal Estimation, Expression Recognition, and Action Unit Detection. This is challenging because emotion-related cues must be distinguished from spurious factors such as identity, illumination, pose, and demographic variation. Attention mechanisms are well suited as they aggregate information from the most informative facial regions, but may still exploit dataset-specific correlations instead of true affective cues. To improve generalization, we propose an attention pooling framework that promotes subject-invariant attention while increasing feature expressiveness. Our method consists of three components. First, we introduce causal supervision to enforce attention on facial regions with invariant predictive value across subjects. Second, we apply a cross-covariance independence regularization between Key (K) and Value (V) projections to encourage complementary, non-redundant representations. Finally, we replace the linear Value projection with a gated nonlinear SwiGLU transformation to increase feature expressiveness and capture finer-grained affective cues. Our method achieves $CCC_{VA}=0.5123$ for VA estimation on the official validation set, together with $F1_{EX}=0.3116$ and $F1_{AU}=0.3974$ for expression recognition and action unit detection, respectively, resulting in an overall $P$ score (the sum of the individual task metrics) of $1.2214$.