Search papers, labs, and topics across Lattice.
This paper introduces MedREAL, a novel framework that integrates linguistic reasoning with spatial localization in medical image analysis, addressing the limitations of existing Multimodal Large Language Models (MLLMs) that rely on global image features. By employing Seg Anchored Reasoning Pooling (SARP) to extract task-relevant semantic evidence from segmentation tokens, and a Reasoning-to-Visual (R2V) fusion mechanism, MedREAL enhances the accuracy of visual question answering and segmentation tasks. Experimental results show that MedREAL outperforms state-of-the-art methods, achieving significant improvements in both global and contextual intersection over union metrics, thereby enhancing clinical trustworthiness in medical AI applications.
MedREAL achieves a remarkable 68.49% gIoU and 70.47% cIoU, setting a new benchmark for interpretable medical image analysis that aligns reasoning with pixel-level accuracy.
Although Multimodal Large Language Models (MLLMs) have demonstrated impressive performance in Medical Visual Question Answering (Med-VQA), their reliance on global image features often lacks precise pixel-level grounding, thereby limiting clinical trustworthiness. To bridge the semantic gap between high-level clinical reasoning and spatial localization, we propose \textsc{\textsc{MedREAL}} (\textbf{Med}ical \textbf{RE}asoning-driven \textbf{A}nswering and \textbf{L}ocalization), a unified framework that seamlessly aligns linguistic reasoning with spatial grounding. Specifically, \textsc{MedREAL} introduces \textbf{S}eg \textbf{A}nchored \textbf{R}easoning \textbf{P}ooling (SARP) to distill task-relevant semantic evidence directly from \texttt{[SEG]} tokens within the MLLM's hidden states. Furthermore, a \textbf{R}easoning-to-\textbf{V}isual (R2V) fusion mechanism is proposed to effectively inject these reasoning-aware features into a segmentation pipeline for accurate mask decoding. To facilitate this paradigm, we construct MedRAVS-13K, a comprehensive dataset comprising 13,824 expertly validated samples across four diverse imaging modalities. Extensive experiments demonstrate that \textsc{MedREAL} significantly outperforms state-of-the-arts, achieving 68.49\% gIoU and 70.47\% cIoU on benchmark evaluations. By generating evidence masks that are strictly consistent with textual diagnoses, \textsc{MedREAL} provides a robust, interpretable framework for reasoning-driven medical image analysis.
