Search papers, labs, and topics across Lattice.
SurgAtlas is introduced as the largest surgical video-language dataset, encompassing 15,291 videos totaling 2,391 hours across 18 surgical specialties, with a significant focus on both open and minimally invasive surgeries. This dataset establishes standardized benchmarks for open-surgery video understanding and includes a validated subset with visual question-answer pairs for enhanced surgical reasoning. By leveraging a multi-tier annotation pipeline and fine-tuning the Qwen3-VL-8B model, the authors achieve competitive results on various surgical benchmarks, paving the way for advanced multimodal surgical AI systems.
The largest surgical video-language dataset to date not only enhances surgical video understanding but also sets new benchmarks for AI-driven surgical reasoning.
We introduce SurgAtlas, the largest surgical video-language dataset to date, comprising 15,291 videos (2,391 hours) spanning 18 surgical specialties and over 5,000 procedure types, sourced entirely from publicly available YouTube content. SurgAtlas is also the first surgical video-language dataset to include open surgery at scale, with 6,182 open procedure videos alongside over 9,000 minimally invasive recordings, and the first to establish standardized benchmarks for open-surgery video understanding. We additionally provide an expert-validated subset with verified visual question-answer pairs across diverse open and minimally invasive procedures, serving as a clinically grounded benchmark for surgical reasoning. Compared with existing surgical video-language datasets, SurgAtlas provides one of the most diverse annotation schemas, combining segment-level captions, step- and phase-level descriptions, video-level surgical descriptions, and reasoning-oriented question-answer pairs organized within a hierarchical taxonomy. These annotations are constructed through an automated multi-tier pipeline with LLM-based enrichment and a staged VQA generation framework with explicit groundedness verification. The scale and diversity of SurgAtlas enable training surgical foundation models with broad procedural coverage: we finetune Qwen3-VL-8B through a two-stage captioning-then-instruction pipeline and achieve competitive or state-of-the-art results on multiple established surgical benchmarks, including phase recognition, triplet detection, and reasoning question answering. More broadly, SurgAtlas provides a large native public video corpus that can support future large-scale pretraining of multimodal surgical AI systems and contribute to the development of next-generation foundation models for surgery.
