Search papers, labs, and topics across Lattice.
This paper introduces the Concept Alignment Contrast and Long-Short Prompt Memory (CM-TTA) framework to enhance the Test-Time Adaptation (TTA) of the Segment Anything Model 3 (SAM3) for medical image segmentation. By employing a novel Concept Alignment Contrast metric to evaluate prediction quality and a Long-Short Prompt Memory module to balance rapid and stable adaptation, the authors address the challenges of domain gaps and the lack of reliable supervision in medical imaging. Experimental results show that CM-TTA significantly improves segmentation performance on prostate and skin lesions compared to existing TTA methods.
Concept Alignment Contrast enables robust evaluation of prediction quality, leading to superior segmentation performance in medical imaging despite the domain gap.
Concept segmentation models like Segment Anything Model 3 (SAM3) show strong generalization on natural images, yet their performance degrades in medical imaging due to the domain gap caused by different imaging principles and styles. Test-Time Adaptation (TTA) is essential for improving the testing performance by updating the model on the fly without annotations. However, existing vision-language TTA methods are mainly driven by image-level uncertainty minimization, which does not necessarily reflect region-level semantic correctness in medical segmentation. Moreover, they often lack mechanisms to maintain stability in continual one-pass adaptation, leading to limited performance when reliable dense supervision is missing for segmentation. To address these issues, we propose Concept Alignment Contrast and LongShort Prompt Memory for Test-Time Adaptation (CM-TTA) of SAM3 for medical images. First, for a test sample with multiple augmentations, we introduce a novel Concept Alignment Contrast (CAC) metric, which leverages textual-visual semantic consistency to robustly evaluate prediction quality to select the best augmented view as the supervision. Second, to balance rapid and stable adaptation, we design a Long-Short Prompt Memory (LSPM) module. The short memory dynamically fuses recent prompts based on CAC scores for agile local adaptation, while the long memory maintains a stable global prompt to generate enhanced pseudo-labels. Finally, a Densely Supervised Prompt Update (DSPU) strategy is proposed to optimize the prompt embeddings with enhanced pseudo labels as dense supervision. Extensive experiments on prostate and skin lesion segmentation demonstrate that our CM-TTA framework significantly outperforms existing methods for TTA of SAM3.
