Search papers, labs, and topics across Lattice.
This paper introduces SurgNarrator, a generative retrieval framework specifically designed for surgical video understanding, addressing the limitations of existing video-language models in clinical settings. By constructing a specialized surgical vocabulary and adapting the Qwen3-VL-Embedding-8B model with a temporally-aware contrastive objective, the framework enhances the accuracy and speed of event retrieval in surgical videos. Evaluation across twelve benchmarks demonstrates that SurgNarrator significantly outperforms state-of-the-art methods while achieving over 100 times reduction in output-stage latency, making it suitable for real-time clinical applications.
Surgeons can now access real-time insights from surgical videos with over 100x faster response times, thanks to a novel generative retrieval framework.
Surgical procedures unfold as structured and recurring clinical events, whose real-time understanding via intraoperative surgical videos is critical for intraoperative decision-making and support. However, existing video understanding methods force a trade-off: autoregressive video-language models support comprehensive reasoning but are not practical for time-sensitive clinical applications, whereas contrastive models offer low latency but struggle with complex scene understanding. Recently, generative retrieval has been explored for general-domain video understanding, but transferring it to surgery is not trivial because near-identical visual appearances may indicate semantically distinct events, and the terminology involved is highly surgery-specific. To this end, we propose SurgNarrator, a new generative retrieval framework tailored for surgical video understanding. We construct a well-curated surgery-centric vocabulary from surgical captions to define a clinically meaningful retrieval space. We then adapt the pre-trained Qwen3-VL-Embedding-8B to learn discriminative clinical representations with a temporally-aware contrastive objective. During inference, a hierarchical, procedure-aware retrieval strategy narrows the search space to the relevant procedure type, delivering fast and effective responses. Our method is comprehensively evaluated on twelve benchmarks in a zero-shot setting and achieves consistent performance gains over state-of-the-art baselines, while reducing output-stage latency by more than two orders of magnitude compared with the generative baseline.
