Search papers, labs, and topics across Lattice.
This paper investigates a task-specific feature fusion approach for multi-task affective behavior analysis in the ABAW11 challenge, where diverse tasks such as valence-arousal prediction and facial expression recognition benefit from tailored visual features and processing strategies. By adapting and freezing two pretrained visual backbones, the authors extract complementary features and systematically evaluate various fusion and prediction methods, ultimately selecting the most effective strategies for each task. The resulting system achieves competitive performance metrics, indicating that task-adaptive fusion of frozen features can significantly enhance multi-task analysis in affective computing.
Task-adaptive feature fusion can dramatically improve multi-task affective behavior analysis, achieving an overall validation score of 1.6341 on the ABAW11 benchmark.
The 11th Affective Behavior Analysis in-the-wild (ABAW11) Multi-Task Learning Challenge requires a unified system to predict valence-arousal, categorical expressions, and facial action units from the official s-Aff-Wild2 images. Although these tasks are naturally related through facial behavior, our validation experiments show that they benefit from different visual features, temporal processing strategies, fusion mechanisms, and calibration procedures. In this paper, we study task-adaptive feature fusion for ABAW11 multi-task affective behavior analysis. We first adapt two pretrained visual backbones, DINOv2 ViT-L and DINOv3 ConvNeXt-base, on an external expression-oriented facial image set and then freeze them to extract complementary frame-level features from the official ABAW11 data. On top of these frozen features, we systematically compare frame-level prediction heads, temporal convolutional heads, post-hoc temporal smoothing, LightGBM models, feature concatenation, gated fusion, residual fusion, late logit fusion, threshold calibration, and shared MTL structures. The final system selects task-specific fusion and prediction strategies rather than forcing all tasks to share a single architecture. On the ABAW11 validation set, the selected system achieves an EXPR macro-F1 of 0.4222, an AU macro-F1 of 0.5402, and a mean VA CCC of 0.6717, resulting in an overall validation score of 1.6341. The results suggest that task-adaptive fusion of frozen visual features is a simple and effective strategy for ABAW-style multi-task affective behavior analysis.
