Search papers, labs, and topics across Lattice.
This study addresses the challenge of integrating multi-scale information in pathological image analysis by introducing a novel cross-scale training and evaluation paradigm for vision-language models (VLMs). The authors develop a leakage-aware curation pipeline to mitigate the issue of text-only shortcuts in multi-image visual question answering (VQA), resulting in the creation of the Scale-VQA benchmark with 4,685 questions based on 2,537 pathology images. The proposed ScaleReasoner-R1 model, trained via reinforcement learning, achieves state-of-the-art performance on the cross-scale VQA task and demonstrates significant improvements in pathological understanding, even with limited supervision.
A new benchmark and model reveal that even minimal cross-scale supervision can dramatically enhance pathological image interpretation.
Pathological images are inherently multi-scale, requiring pathologists to integrate evidence from global tissue architecture at low magnification to cellular morphology at higher magnification for accurate diagnosis. While existing pathological datasets for vision-language model (VLM) include various scales, they often lack an explicit cross-scale reasoning objective. This limitation prevents VLMs from capturing essential cross-scale representations and learning evidence-based reasoning. To bridge this gap, we introduce the first cross-scale training and evaluation paradigm that formulates pathology interpretation as multi-magnification reasoning. However, creating such a task reveals a critical challenge: multi-image visual question answering (VQA) is prone to text-only shortcuts, which allow models to guess answers using magnification-dependent artifacts rather than visual evidence. To address this, we propose a leakage-aware curation pipeline that combines adversarial text-only screening with constraint-guided question design. Using this pipeline, we construct Scale-VQA, a high-quality benchmark with 4,685 multiple-choice questions grounded in 2,537 pathology images across multiple magnification levels. Finally, we present ScaleReasoner-R1, a model trained via reinforcement learning to optimize performance on the cross-scale VQA task. ScaleReasoner-R1 achieves state-of-the-art performance on our cross-scale reasoning benchmark and generalizes to SOTA performance on established single-scale benchmarks. Findings suggest that even the limited cross-scale supervision can significantly improve pathological understanding. The code and demos will be open-sourced.
NUS