Search papers, labs, and topics across Lattice.
This paper introduces QG-MIL, a gated transformer aggregator designed to enhance attention-based Multiple Instance Learning (MIL) in medical imaging by mitigating issues of attention concentration that lead to overconfident predictions. By integrating architectural innovations such as RMSNorm-based pre-normalization and fine-grained attention output gating, QG-MIL achieves more stable training and improved attention distribution across instances. Evaluated on six diverse benchmarks, QG-MIL consistently outperforms leading baselines, achieving an average improvement of +6.1 mean macro F1 points, demonstrating its robustness across various MIL scales.
QG-MIL achieves a remarkable +6.1 mean macro F1 point improvement across diverse medical imaging tasks, redefining stability and performance in attention-based learning.
Attention-based Multiple Instance Learning aggregators in medical imaging are prone to attention concentration, producing overconfident and unstable predictions. We introduce QG-MIL, a gated transformer aggregator that addresses this through four synergistic architectural components: RMSNorm-based pre-normalization, per-head QK normalization, fine-grained attention output gating, and SwiGLU-style feed-forward modules. Together, these design choices stabilize training and distribute attention more uniformly across instances without auxiliary losses, masking, or multi-stage regularization. We evaluate QG-MIL across six benchmarks spanning whole-slide pathology and cell-level hematology, covering two fundamentally different MIL scales. The best-performing QG-MIL variants outperform leading baselines on all six benchmarks, with an average improvement of +6.1 mean macro F1 points. Attention overlays and attention mass analysis confirm more distributed instance weighting. Ablation studies show that while individual components can match the full model on specific datasets, the QG-MIL design provides the most consistent cross-domain performance and tightest variance when compared to selected baselines. We release a configurable implementation to support reproducibility at: https://github.com/unica-visual-intelligence-lab/QG-MIL
