Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
GazeRefine introduces a novel, training-free framework for medical image segmentation that leverages expert gaze as a zero-shot inference-time prompt. By converting sparse, duration-weighted gaze fixations into foreground and background priors, the method initializes semantic prototypes in a frozen DINOv3 feature space, which are then refined through a series of discriminative and propagative techniques. The approach demonstrates strong performance in polyp segmentation and competitive results in prostate MRI segmentation, highlighting its potential for efficient, human-in-the-loop applications without the need for extensive training data or fine-tuning.
GazeRefine achieves effective medical image segmentation without any training, relying solely on expert gaze to guide the process.
Medical image segmentation remains difficult to scale because high-performing methods typically rely on dense expert annotations and task-specific training. We introduce GazeRefine, a training-free framework that uses gaze as an inference-time prompt for zero-shot medical image segmentation. Sparse, duration-weighted fixations are converted into foreground and background priors that initialize semantic prototypes in frozen DINOv3 feature space. These prototypes are iteratively refined through foreground-background discrimination, feature-space affinity propagation, and anchoring to the initial gaze guidance, allowing segmentation to extend beyond directly fixated regions while limiting semantic drift. GazeRefine requires no segmentation masks, fine-tuning, adapters, prompt encoders, or gradient updates. We evaluate the method on gaze-annotated polyp segmentation and prostate MRI segmentation. The results show strong performance on colonoscopy images and competitive performance on prostate MRI, supporting gaze-guided prototype refinement as a promising approach for segmentation-label-efficient, human-in-the-loop medical image segmentation. Our tools and code can be found in the following repository: https://github.com/MohammedOussamaBEN/GazeRefine.git