Search papers, labs, and topics across Lattice.
This paper introduces LoMeVQA, a benchmark comprising 206K longitudinal visual question answering pairs designed to enhance temporal medical image analysis. It highlights the inadequacies of existing multimodal large language models (MLLMs) in handling longitudinal data, revealing significant performance gaps in temporal reasoning tasks. The authors also present MedLong-8B, a model that achieves state-of-the-art results across all tasks, while providing insights into failure modes and potential improvements for future research.
Longitudinal medical visual reasoning is critically underexplored, with existing models failing to grasp temporal nuances, as evidenced by their poor performance on the new LoMeVQA benchmark.
In clinical practice, patients often undergo multiple imaging examinations over successive visits, yielding longitudinal data. Modeling such temporal information is crucial for reliable assessment of disease progression and treatment response. However, despite the rapid advancement of multimodal large language models (MLLMs), longitudinal medical visual reasoning remains largely underexplored. To fill this gap, we propose LoMeVQA, a comprehensive benchmark consisting of 206K longitudinal visual question answering (VQA) pairs for temporal medical image analysis. LoMeVQA covers five tasks: progress classification, progress description, progress report generation, differential region grounding, and differential region description. To construct the dataset, we develop an automated pipeline that (1) organizes patient records chronologically, (2) extracts clinically meaningful entities via a medical knowledge graph, and (3) models their temporal evolution to guide large language models in generating high-quality longitudinal VQA pairs. Extensive evaluations demonstrate that both general-purpose and medical-domain MLLMs perform poorly on LoMeVQA, revealing substantial limitations in temporal reasoning. To address these limitations, we introduce MedLong-8B, which achieves state-of-the-art performance across all tasks. Beyond benchmarking, we conduct detailed analyses that uncover key failure modes and shed light on how to improve longitudinal medical visual reasoning. Our data is available at: https://github.com/pepperbubble/LoMeVQA
