Search papers, labs, and topics across Lattice.
This paper introduces a vision-language framework for comparative reasoning in radiology, addressing the gap between isolated image interpretation and the comparative analysis essential for diagnosis. By constructing MedReCo-DB, a large-scale dataset with over 690,000 images and detailed annotations, the authors enable the development of MedReCo, an entity-aware visual encoder, and MedReCo-VLM, a vision-language model for generative interpretation. The results show that MedReCo outperformed existing methods in retrieval tasks, while MedReCo-VLM significantly enhanced longitudinal follow-up accuracy, highlighting the potential of entity-aware reasoning in medical imaging AI.
Entity-aware comparative reasoning can be learned from routine clinical data, leading to significant improvements in diagnostic accuracy and retrieval performance in radiology.
Medical imaging artificial intelligence has achieved strong performance in isolated image interpretation, but remains poorly aligned with radiological practice, where diagnosis and follow-up rely on comparison across prior studies and analogous reference cases. Here we formulate radiological comparison as an entity-aware cross-image reasoning problem and introduce a framework that supports both reference-case retrieval and temporal comparative interpretation. We construct MedReCo-DB, a large-scale comparative imaging resource derived from routine image-report pairs, comprising more than 690,000 images from over 160,000 patients across eight institutions, four countries and seven imaging modalities. Reports are decomposed into anatomical structures, abnormal findings and pathological conditions to provide supervision for entity-conditioned retrieval and comparative visual question answering. Using this resource, we develop MedReCo, an entity-aware visual encoder for controllable retrieval of clinically analogous cases, and MedReCo-VLM, a vision--language extension for generative interpretation of interval change. Across internal, external and cross-center evaluations, MedReCo achieved the highest Recall@1 in all 12 internal retrieval settings and improved external retrieval by a mean of 6.0 percentage points. In clinically confusable differential groups, it consistently outperformed the strongest baselines. MedReCo-VLM achieved the best performance across all comparative generation evaluations and improved longitudinal follow-up accuracy by 14.5-46.5 percentage points on chest radiographs and 13.0-27.9 percentage points on CT. These findings suggest that entity-aware comparative reasoning can be learned from routine clinical data at scale and may provide a more clinically aligned foundation for medical imaging AI.
