Search papers, labs, and topics across Lattice.
The paper introduces BiCAM, a bidirectional class activation mapping method for Vision Transformers (ViTs) that captures both positive and negative contributions to model predictions, addressing the limitation of prior CAM methods that discard negative signals. By preserving signed attributions, BiCAM generates more complete and contrastive explanations, improving localization and faithfulness on ImageNet, VOC, and COCO. The paper also introduces a Positive-to-Negative Ratio (PNR) derived from BiCAM, demonstrating its utility in detecting adversarial examples without retraining.
Discarding negative attribution signals in Vision Transformers hurts interpretability: BiCAM leverages both positive and negative signals to improve localization, faithfulness, and adversarial detection.
Vision Transformers (ViTs) achieve strong performance in visual recognition, yet their decision-making remains difficult to interpret. We propose BiCAM, a bidirectional class activation mapping method that captures both supportive (positive) and suppressive (negative) contributions to model predictions. Unlike prior CAM-based approaches that discard negative signals, BiCAM preserves signed attributions to produce more complete and contrastive explanations. BiCAM further introduces a Positive-to-Negative Ratio (PNR) that summarizes attribution balance and enables lightweight detection of adversarial examples without retraining. Across ImageNet, VOC, and COCO, BiCAM improves localization and faithfulness while remaining computationally efficient. It generalizes to multiple ViT variants, including DeiT and Swin. These results suggest the importance of modeling both supportive and suppressive evidence for interpreting transformer-based vision models.
