Search papers, labs, and topics across Lattice.
This study introduces OCT-Bench, a novel benchmark designed to evaluate multimodal large language models (MLLMs) on their understanding of optical coherence tomography (OCT) images through a comprehensive set of 10,076 multiple-choice questions derived from 4,137 OCT images. By establishing a hierarchical capability taxonomy with 20 fine-grained tasks across perception, cognition, and reasoning, the research highlights the inadequacies of current MLLMs in reliably interpreting OCT data, revealing that neither model scaling nor medical-domain adaptation significantly enhances performance. The findings underscore the need for targeted improvements in MLLMs to achieve clinically relevant OCT understanding, paving the way for more effective applications in retinal disease diagnosis and treatment.
Current multimodal large language models struggle with OCT image understanding, falling short even with specialized adaptations.
Optical coherence tomography (OCT) imaging is essential for the diagnosis and treatment of retinal diseases. Although multimodal large language models (MLLMs) have demonstrated considerable potential in medical image analysis, existing benchmarks largely reduce OCT understanding to coarse-grained disease classification or isolated visual question answering, leaving the complete cognitive process from visual perception to clinical reasoning insufficiently evaluated. To address this limitation, we introduce OCT-Bench, a comprehensive benchmark dedicated to OCT image understanding. OCT-Bench comprises 10,076 high-quality multiple-choice questions constructed from 4,137 OCT images across seven public datasets. Following the real-world clinical interpretation workflow, we establish a hierarchical capability taxonomy consisting of 20 fine-grained tasks across three dimensions: Perception, Cognition, and Reasoning. These tasks cover a broad range of capabilities, including imaging attributes, retinal anatomy, lesion characteristics, spatial relationships, disease assessment, therapeutic decision-making, and prognostic management. We systematically evaluate 20 representative MLLMs, including proprietary models, open-source general-purpose models, and medical-domain models. Experimental results demonstrate that current models remain substantially short of reliable OCT understanding. Moreover, neither medical-domain adaptation nor increased model scale consistently improves performance across capability levels. OCT-Bench enables comprehensive and fine-grained evaluation of MLLMs, providing a foundation for identifying capability bottlenecks and advancing clinically grounded OCT understanding.
