Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To address spurious feature fixation in weakly supervised Multiple Instance Learning (MIL), CAR-MIL couples standard attention pooling with a counterfactual regularization branch that discovers the minimal attention distribution shift required to alter the bag-level prediction. This constraint forces the model to decouple genuinely causal instances from confounders, yielding complementary factual and counterfactual attention maps that respectively support and challenge the final output. Across synthetic benchmarks with instance-level ground truth and five digital pathology datasets, the framework substantially improves attention localization fidelity while driving classification gains on the most challenging tasks.
Weakly supervised attention maps routinely latch onto spurious confounders—until you regularize them with the minimal attention shift required to flip the model's decision.
Multiple Instance Learning (MIL) is widely used for weakly supervised learning, particularly in digital pathology, where fine-grained annotations are costly. Most MIL methods aggregate instance features via attention mechanisms. However, attention weights do not always faithfully reflect instance importance and may focus on spuriously correlated regions. In this work, we propose CAR-MIL, a framework that explicitly guides attention learning through a counterfactual attention regularization objective inspired by counterfactual explanations. Built on a standard attention-based MIL architecture, our approach introduces a lightweight counterfactual attention branch trained to produce an alternative prediction while remaining close to the factual attention distribution. This encourages prediction changes to arise from minimal, structured redistributions of attention, leading to more informative evidence allocation. The resulting factual and counterfactual attention maps capture complementary evidence: the former highlights regions supporting the prediction, while the latter reveals regions whose reweighting would challenge it. We evaluate our method on synthetic MIL benchmarks with instance-level ground truth enabling controlled analysis of attention behavior and on five digital pathology datasets across four tasks. CAR-MIL maintains competitive classification performance, with the largest gains observed on more challenging tasks, while improving attention reliability, demonstrating the benefits of integrating counterfactual explainability reasoning into attention learning. Code is available at: https://github.com/ImaneCR/CAR-MIL/.