Search papers, labs, and topics across Lattice.
This paper introduces Mask to Concept (M2C), a novel framework that enhances the SAM3 segmentation model for few-shot medical annotation by enabling it to autonomously search for transferable visual concepts without requiring external modules or manual text engineering. By initializing a learnable concept embedding and employing a Hybrid Uncertainty Estimation (HUE) module, M2C effectively minimizes segmentation errors while actively managing uncertainty in predictions. Experimental results demonstrate that M2C achieves state-of-the-art performance in few-shot segmentation tasks, significantly improving annotation efficiency in medical imaging applications.
M2C transforms SAM3 into an auto-promptable annotator that achieves state-of-the-art few-shot segmentation with minimal expert intervention.
Transforming foundation segmentation models from human-prompted tools into auto-promptable annotators is critical for scalable medical data annotation. Current methods commonly depend on external feature matchers or auxiliary networks to automate geometric prompting, but introducing architectural overhead and limiting performance scalability. Although SAM3 natively supports concept segmentation via reusable text prompts, its direct use in medical imaging is hindered by a lack of fine-grained clinical knowledge and the ambiguity of human-written descriptions. In this work, we propose Mask to Concept (M2C), an efficient framework that adapts SAM3 for medical few-shot annotation without external modules, parameter retraining, or manual text engineering. Using only a few labeled images, M2C enables SAM3 to automatically search for transferable visual concepts entirely within its frozen architecture: it initializes a learnable concept embedding, uses it to prompt segmentation, and updates the embedding by gradients of minimizing the concept segmentation error. We further introduce a Hybrid Uncertainty Estimation (HUE) module that calculates the prediction entropy and maps concept predictions back to the box prompts, measuring concept-geometry prompting inconsistency. Highly uncertain samples are flagged actively for human correction, and the corrected masks are then fed back to M2C to continuously search for more precise concept embeddings, forming a self-enhancing annotation loop with minimal expert effort. Experiments on medical segmentation benchmarks show that our method achieves SOTA few-shot segmentation performance and outstanding annotation efficiency, offering a practical and efficient pathway toward scalable medical image labeling. Codes are at https://github.com/Huster-Hq/M2C.
