Search papers, labs, and topics across Lattice.
The authors introduce SurgRec, a scalable and reproducible pretraining recipe for surgical video understanding, addressing limitations in existing surgical foundation models related to data scale, diversity, and evaluation. They curated a large, multi-source corpus of 10,535 surgical videos and developed a unified pretraining pipeline with balanced sampling. Experiments across 16 downstream datasets show SurgRec consistently outperforms SSL baselines and vision-language models, particularly in fine-grained temporal recognition where VLMs struggle.
Vision-language models falter at the fine-grained temporal recognition crucial for surgical video understanding, while SurgRec excels.
Surgical video understanding is essential for computer-assisted interventions, yet existing surgical foundation models remain constrained by limited data scale, procedural diversity, and inconsistent evaluation, often lacking a reproducible training pipeline. We propose SurgRec, a scalable and reproducible pretraining recipe for surgical video understanding, instantiated with two variants: SurgRec-MAE and SurgRec-JEPA. We curate a large multi-source corpus of 10,535 videos and 214.5M frames spanning endoscopy, laparoscopy, cataract, and robotic surgery. Building on this corpus, we develop a unified pretraining pipeline with balanced sampling and standardize a reproducible benchmark across 16 downstream datasets and four clinical domains with consistent data splits. Across extensive comparisons against SSL baselines and vision-language models, SurgRec consistently achieves superior performance across downstream datasets. In contrast, VLMs prove unreliable for fine-grained temporal recognition, exhibiting both performance gaps and sensitivity to prompt phrasing. Our work provides a reproducible, scalable foundation for the community to build more general surgical video models. All code, models, and data will be publicly released.
