Search papers, labs, and topics across Lattice.
This paper introduces MiRA, a novel attention redistribution framework for Vision Transformers that enhances the model's ability to capture subtle facial dynamics in video data. By deriving frame-level confidence and intra-frame concentration statistics from self-attention maps, MiRA effectively reallocates attention to localized facial cues, addressing the limitations of existing models that focus on dominant global motions. Experimental results on Facial Expression Recognition benchmarks show that MiRA significantly outperforms traditional ViT baselines without adding extra trainable parameters.
Attention redistribution can dramatically enhance facial expression understanding in videos, leading to improved performance on challenging recognition tasks.
Understanding facial expressions in videos requires modeling subtle and localized facial dynamics under unconstrained conditions. Although recent Vision Transformer~(ViT)-based video models have shown strong performance through large-scale self-supervised pretraining, their attention mechanisms often emphasize dominant global motions and coarse temporal dynamics, limiting sensitivity to fine-grained facial variations. To address this limitation, we propose MiRA (Marginal-induced Attention Redistribution), a plug-in frame-marginal attention redistribution framework for ViT backbones that enhances spatio-temporal selectivity toward subtle facial dynamics without introducing additional trainable parameters. MiRA derives frame-level confidence and intra-frame concentration statistics from self-attention maps to estimate frame-wise marginal importance and redistribute attention toward spatiotemporally localized facial cues. We first introduce a principled \textit{exact mode} based on post-softmax attention redistribution. To further improve efficiency, we propose \textit{flashLite mode}, a lightweight pre-softmax approximation that integrates frame-marginal redistribution into FlashAttention kernels while preserving the effectiveness of the exact formulation. Experimental results on challenging Facial Expression Recognition~(FER) benchmarks demonstrate consistent improvements over strong ViT baselines.
