Search papers, labs, and topics across Lattice.
This paper introduces UniMedSeg, a unified Transformer-based framework that integrates various medical image segmentation paradigms, including visual in-context learning, interactive segmentation, and language-guided segmentation, into a single model capable of processing both 2D and 3D images. By employing Decoupled Split Attention, the model significantly reduces attention complexity while maintaining efficient computation, allowing for the joint learning of heterogeneous medical supervision. Extensive evaluations across 27 public datasets demonstrate that UniMedSeg achieves state-of-the-art performance without the need for task-specific fine-tuning, showcasing its strong generalization capabilities across diverse medical imaging tasks.
Achieving state-of-the-art segmentation performance without task-specific fine-tuning, UniMedSeg unifies disparate paradigms into a single scalable model.
Medical image segmentation foundation models are expected to generalize across diverse clinical scenarios, yet existing universal methods remain fragmented by prompt paradigms and spatial dimensions. Visual in-context learning, interactive segmentation, and language-guided segmentation are typically handled by paradigm-specific models, while 2D and 3D images are also modeled separately. Such isolation prevents heterogeneous annotations and data from being jointly absorbed by a single scalable model and limits cross-paradigm knowledge transfer. To address this bottleneck, we propose UniMedSeg, a Transformer-centric universal segmentation framework that maps visual examples, geometric interactions, language instructions, and 2D/3D images into a shared sequence space, enabling heterogeneous medical supervision to be jointly learned through a unified in-context interface without prompt- or dimension-specific branches. To overcome the long-sequence memory bottleneck caused by visual contexts, we introduce Decoupled Split Attention, which reduces attention complexity to linear while preserving hardware-friendly computation and focused context-target interaction. Extensively trained and evaluated on a large corpus curated from 27 public datasets, UniMedSeg achieves state-of-the-art performance across visual in-context, interactive, and language-guided segmentation without task-specific fine-tuning, demonstrating strong generalization on diverse held-out tasks. The code and model weights are publicly available at https://github.com/Lii1228/UniMedSeg
