Search papers, labs, and topics across Lattice.
This paper introduces ARTEMIS, a novel framework for video polyp segmentation (VPS) that effectively leverages imperfect supervision through agent-guided reliability-aware temporal mask evolution. By integrating weak annotations and semi-supervised learning, ARTEMIS addresses challenges such as boundary leakage and motion blur, utilizing a vision-language agent to select reliable temporal anchors and refine segmentation masks across frames. Experimental results on datasets like SUN-SEG and CVC-ClinicDB-612 show that ARTEMIS outperforms existing methods, achieving state-of-the-art segmentation accuracy under various supervision conditions.
ARTEMIS transforms weakly supervised video polyp segmentation by evolving reliable masks over time, achieving unprecedented accuracy in challenging clinical settings.
Imperfectly supervised video polyp segmentation (VPS) aims to learn dense, temporally consistent masks from inexpensive supervision, including weak annotations (points, scribbles) and semi-supervision with few densely labeled frames. This setting is clinically valuable but challenging due to weak contrast, ambiguous boundaries, motion blur, and specular highlights, compounded by sparse pixel-level guidance. While SAM2 can generate dense masks from sparse inputs, direct pseudo-labeling often yields geometry-degraded masks with boundary leakage, underutilizes temporal consistency, and ignores reliability. To address these issues, we propose ARTEMIS, a unified framework for imperfectly supervised VPS driven by agent-guided reliability-aware temporal mask evolution. ARTEMIS initializes coarse masks from available supervision: SAM2 converts points/scribbles, while dense labels serve as reliable anchors. A debate-and-judge vision-language agent selects reliable temporal anchors under weak supervision, which are propagated bidirectionally with SAM2 to refine unreliable or unlabeled frames. Finally, ARTEMIS trains the segmenter using temporal reliability-aware robust learning, incorporating reliability-guided reference selection, a Reference Prototype Transport Module, and reliability-aware robust loss. These components assess mask reliability, evolve anchors over time, transport target identity across frames, and down-weight noisy supervision instead of discarding difficult samples. Experiments on SUN-SEG and CVC-ClinicDB-612 under scribble, point, and limited-label settings demonstrate that ARTEMIS achieves state-of-the-art performance. Code will be released at https://github.com/wangtong627/ARTEMIS.
