Search papers, labs, and topics across Lattice.
This paper introduces PGE-SAM, a novel framework that enhances the Segment Anything Model (SAM) for interactive segmentation by leveraging user prompts and prior mask predictions to focus feature restoration on relevant regions. By incorporating Multi-Scale Features Interaction and a Foreground Reconstruction Loss, PGE-SAM effectively recovers fine-grained details lost due to real-world imaging artifacts, outperforming existing methods. Extensive experiments validate that PGE-SAM achieves state-of-the-art robustness across various degradation levels in both medical and natural image domains while maintaining generalization to clean images with minimal additional parameters.
PGE-SAM not only recovers fine details lost to degradation but also adapts to user input, redefining interactive segmentation in challenging imaging conditions.
Segment Anything Model (SAM) has revolutionized promptable image segmentation with strong zero-shot generalization. However, its performance degrades substantially under real-world imaging artifacts such as noise, blur, and compression. Existing methods restore features globally without focusing on segmentation-relevant regions and neglect SAM's iterative refinement mechanism, leading to suboptimal performance in interactive settings. We propose Prompt-Guided Feature Enhancement SAM (PGE-SAM), a framework that explicitly leverages user prompts and prior mask predictions to spatially guide the feature restoration process toward regions of interest through a Prompt Guidance Generator. To recover fine-grained details lost under degradation, we introduce Multi-Scale Features Interaction to incorporate low-level encoder features, along with a Foreground Reconstruction Loss that restricts feature-level supervision to the segmentation target. Furthermore, we present DM-Seg, a benchmark for interactive segmentation on degraded medical images, spanning multiple imaging modalities with both general and modality-specific degradations at varying severity levels. Extensive experiments demonstrate that PGE-SAM achieves SOTA robustness on both medical and natural image domains across multiple degradation levels, while maintaining generalization to clean images and adding less than one-fifth of the parameters of prior methods.
