Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the faithfulness of visual attention in Vision-Language Models (VLMs) through causal perturbation analysis, revealing three distinct processing modes: Faithful-Sufficient, Faithful-Distributed, and Non-Focal. The findings indicate that human-annotated regions only align with model attention rankings about 60% of the time, highlighting a significant divergence between model behavior and human intuition. This research underscores the heterogeneous nature of visual attention faithfulness, which varies with processing demands and model architectures, challenging the assumption of uniform attention reliability in VLMs.
Visual attention in VLMs is not uniformly reliable; it varies across three distinct modes, revealing a surprising gap between model predictions and human intuition.
Whether attention weights faithfully reflect model reasoning has been actively debated in NLP, yet this question remains largely unexplored for the visual modality in Vision-Language Models (VLMs). We address this gap through causal perturbation analysis on current VLMs, evaluating both the comprehensiveness and sufficiency gap of attention-ranked visual tokens. Our analysis reveals that visual attention faithfulness is heterogeneous, manifesting in three distinct processing modes: Faithful-Sufficient, where top-$k$ attention tokens are both necessary and sufficient for prediction; Faithful-Distributed, where they are necessary but broader visual context remains required; and Non-Focal, where no localized attention region is individually necessary while visual information remains an essential trigger for prediction. Furthermore, human-annotated ground-truth regions satisfy comprehensiveness in only $\sim 60$% of cases compared with model attention rankings, revealing systematic divergence between model visual reliance and human intuition. We demonstrate these patterns across both general VQA on VQAv2 and document tasks on VRDU and ChartQA, showing that visual attention faithfulness varies systematically with processing demands and model architectures rather than being uniformly faithful or unfaithful.