Search papers, labs, and topics across Lattice.
This paper introduces ReferEndoscopy, a large-scale benchmark for Referring Image Segmentation (RIS) tailored for endoscopic imagery, addressing the challenges of limited annotations and complex image-text relationships. The authors propose the Attribute Retrieval-based Endoscopic-RIS (AR-ERIS) framework, which utilizes attribute retrieval to enhance open-vocabulary segmentation capabilities. Their approach achieves state-of-the-art performance and demonstrates robust generalization across both simulated and real-world endoscopic data, significantly advancing the field of RIS in medical imaging.
State-of-the-art performance in endoscopic referring segmentation is now achievable through a novel attribute retrieval approach, transforming how we interpret complex medical imagery.
Referring Image Segmentation (RIS) aims to segment image regions specified by natural language, enabling fine-grained and controllable visual understanding. Extending RIS to endoscopic imagery, however, presents unique challenges, including scarce high-quality annotations and complex, domain-specific image-text relationships. Although recent vision-language models demonstrate strong cross-domain alignment, they often fail to capture fine-grained textual cues in endoscopic settings, resulting in suboptimal performance and limited generalization. To address these challenges, we introduce ReferEndoscopy, a large-scale benchmark for RIS in the endoscopy field. Building on this dataset, we propose the Attribute Retrieval-based Endoscopic-RIS (AR-ERIS) framework for open-vocabulary endoscopic compositional referring segmentation. AR-ERIS leverages attribute retrieval for open-vocabulary endoscopic compositional referring segmentation and is pretrained on the curated ReferEndoscopy dataset, achieving state-of-the-art performance with strong generalization across both simulated and real-world endoscopic data. The dataset and code will be publicly released upon completion of the review process.
Tsinghua AI