Search papers, labs, and topics across Lattice.
This paper introduces MIND, a Multimodal Intent-Driven Network utilizing Diffusion Transformers for enhanced medical image fusion, addressing the limitations of existing methods that apply uniform fusion rules without considering diagnostic intents. By employing BioMedGPT to generate intent-driven fusion texts and incorporating a Multi-scale Latent Adapter to maintain spatial continuity, MIND effectively aligns image features with diagnostic needs. Experimental results demonstrate that MIND not only achieves superior fusion quality but also significantly enhances brain tumor segmentation accuracy, indicating its potential for intelligent clinical decision support systems.
MIND achieves a breakthrough in medical image fusion by integrating intent-driven diagnostics, resulting in significantly improved segmentation accuracy for brain tumors.
Medical image fusion aims to integrate complementary information from diverse imaging modalities to support clinical diagnosis. Existing methods typically apply uniform fusion rules globally, lacking a deep understanding of diagnostic intents and pathological structures. To address these limitations, we propose MIND, a Multimodal Intent-Driven Network via Diffusion Transformers (DiTs) for medical image fusion. Specifically, we utilize BioMedGPT to generate intent-driven fusion texts from source images, guiding the fusion process with pathology-aware diagnostic intents. To combat the loss of 2D spatial continuity caused by 1D sequence flattening in DiTs, we design a Multi-scale Latent Adapter. This module explicitly extracts source image features before serialization, injecting them into the network via strict dimensional alignment to effectively supplement image features. To resolve the semantic shift caused by decoupling image outputs from diagnostic intents, we design a medical semantic consistency loss. This loss ensures deep semantic locking between fused images and fusion texts while maintaining the stability of the underlying physical manifold reconstruction. Comprehensive experiments on the Harvard, BraTS, and GFP datasets reveal that MIND delivers superior fusion quality, significantly improves downstream brain tumor segmentation accuracy, and enables flexible interactive fusion, holding significant promise for intent-driven intelligent clinical decision support systems.
