Search papers, labs, and topics across Lattice.
The paper introduces SAPNet++, a novel architecture for point-prompted instance segmentation (PPIS) that addresses granularity ambiguity and boundary uncertainty inherent in single-point annotations. SAPNet++ incorporates Point Distance Guidance and Box Mining Strategy to resolve granularity ambiguity, and Multi-level Affinity Refinement to improve boundary delineation. Experiments on four datasets demonstrate that SAPNet++ significantly improves segmentation performance compared to existing PPIS methods by integrating semantic and spatial awareness.
Achieve more precise instance segmentation from single-point prompts by explicitly addressing granularity ambiguity and boundary uncertainty, outperforming prior methods on challenging datasets.
Single-point annotation is increasingly prominent in visual tasks for labeling cost reduction. However, it challenges tasks requiring high precision, such as the point-prompted instance segmentation (PPIS) task, which aims to estimate precise masks using single-point prompts to train a segmentation network. Due to the constraints of point annotations, granularity ambiguity and boundary uncertainty arise the difficulty distinguishing between different levels of detail (eg. whole object vs. parts) and the challenge of precisely delineating object boundaries. Previous works have usually inherited the paradigm of mask generation along with proposal selection to achieve PPIS. However, proposal selection relies solely on category information, failing to resolve the ambiguity of different granularity. Furthermore, mask generators offer only finite discrete solutions that often deviate from actual masks, particularly at boundaries. To address these issues, we propose the Semantic-Aware Point-Prompted Instance Segmentation Network (SAPNet). It integrates Point Distance Guidance and Box Mining Strategy to tackle group and local issues caused by the point's granularity ambiguity. Additionally, we incorporate completeness scores within proposals to add spatial granularity awareness, enhancing multiple instance learning (MIL) in proposal selection termed S-MIL. The Multi-level Affinity Refinement conveys pixel and semantic clues, narrowing boundary uncertainty during mask refinement. These modules culminate in SAPNet++, mitigating point prompt's granularity ambiguity and boundary uncertainty and significantly improving segmentation performance. Extensive experiments on four challenging datasets validate the effectiveness of our methods, highlighting the potential to advance PPIS.
