Search papers, labs, and topics across Lattice.
This paper addresses the limitations of Multimodal Large Language Models (MLLMs) in 3D volumetric imaging by introducing a novel slice-wise data synthesis paradigm that models the diagnostic workflow of radiologists. By creating a large-scale structured reasoning dataset, the authors enhance the MLLM's ability to perform spatial reasoning through a Chain-of-Thought (CoT) approach that incorporates sequential spatial tracking and multi-slice awareness. Evaluations show that this method significantly improves performance on 3D medical benchmarks, enabling the model to rival traditional 3D architectures while avoiding the need for costly 3D-specific pre-training.
A novel data synthesis approach enables MLLMs to achieve robust 3D spatial reasoning, rivaling traditional 3D models without expensive pre-training.
While Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in 2D medical image understanding, their extension to 3D volumetric imaging remains hindered by prohibitive annotation costs and dataset opacity. Current data formats, predominantly consisting of rigid Visual Question Answering (VQA) pairs or unstructured final clinical reports, typically fail to capture explicit clinical reasoning. To address this limitation, we introduce a large-scale structured reasoning dataset constructed via a novel slice-wise data synthesis paradigm. Inspired by the genuine diagnostic workflow of radiologists, this paradigm models visual cognition by decomposing the complex 3D reading process, translating global clinical priors into fine-grained, per-slice observations that are subsequently synthesized into an interpretable Chain-of-Thought (CoT). Crucially, this synthesized reasoning framework enforces essential clinical principles: sequential spatial tracking, multi-slice spatial awareness for artifact mitigation, and differential exclusion. To validate this approach, we instruction-tune a standard 2D-pretrained MLLM baseline using the synthesized data to enhance its volumetric comprehension. Comprehensive evaluations across multiple 3D medical benchmarks demonstrate that our method yields significant performance improvements over the 2D baseline. Furthermore, the resulting model exhibits robust spatial reasoning capabilities and rivals resource-intensive native 3D architectures, effectively bridging the performance gap. Ultimately, this data-centric strategy unlocks deep volumetric understanding and highly interpretable clinical logic without requiring computationally expensive 3D-specific pre-training. The complete repository, including datasets and training workflows, is publicly available at https://github.com/2020420145009/hounsfield.
