Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces MCSeg, a volumetric transformer-based network specifically designed for multi-modal cardiac image segmentation, addressing the architectural mismatches in existing hybrid networks through a novel Scaling Feature Pyramid (SFP). The SFP effectively integrates the outputs of a single-scale 3D Vision Transformer encoder with a multi-scale CNN decoder, enhancing the utilization of global contextual information. Experimental results demonstrate that MCSeg outperforms eleven state-of-the-art methods across multiple datasets, including CT and MRI, and shows strong performance in few-shot learning scenarios, indicating its robustness in data-limited environments.
MCSeg outperforms existing methods in cardiac image segmentation by effectively bridging transformer and CNN architectures, showcasing its prowess in few-shot learning scenarios.
Automatic cardiac image segmentation is pivotal for diagnosing and treating cardiac diseases. In this work, we introduce MCSeg, a volumetric transformer-based network tailored for multi-modal cardiac segmentation. To overcome the architectural mismatch inherent in existing hybrid networks, we propose a novel Scaling Feature Pyramid (SFP). Unlike conventional skip connections, the SFP effectively bridges the single-scale 3D Vision Transformer (ViT) encoder and the multi-scale CNN decoder by transforming the ViT's output into a hierarchical feature pyramid, ensuring that global contextual information is effectively leveraged. For the training paradigm, the ViT encoder first undergoes self-supervised pre-training via masked image modeling. Subsequently, the network is fine-tuned on downstream tasks, during which a regional mutual information (RMI) loss is integrated to improve boundary segmentation accuracy. In experiments, MCSeg consistently outperforms eleven SOTA methods on CT dataset ImageCHD, multi-modal dataset MM-WHS, MRI dataset HVSMR-2.0 and MSD Heart, highlighting the effectiveness of our MCSeg for multi-modal cardiac segmentation tasks. Furthermore, MCSeg's superior performance in few-shot experiment showcases its significant potential in adapting to limited data scenarios. Codes and pre-trained ViT-B weights are open-sourced at https://openi.pcl.ac.cn/OpenMedIA/MCSeg