Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces SurgVIL, a novel framework that enhances surgical robot imitation learning by integrating kinematically labeled phantom demonstrations with open-source surgical videos. The approach addresses the scarcity of synchronized video and action data in clinical settings by using weak supervision to estimate robot kinematics from the videos. Evaluation on da Vinci robot tasks shows that incorporating surgical videos significantly improves the model's generalization to real-tissue scenarios and out-of-distribution tasks, indicating a promising direction for scalable surgical robot training.
Surgical robots can now learn from open-source videos, bridging the gap between phantom training and real-world application.
Learning-based surgical robot autonomy requires large-scale demonstrations with synchronized videos and robot actions, but such data are exceedingly rare in clinical or realistic tissue settings because robot kinematics are typically inaccessible outside controlled research systems. In contrast, phantom data collected on research platforms provide accurate action labels but lack the visual diversity of real tissue. We propose SurgVIL, a framework for scaling surgical robot imitation learning using open-source surgical videos. SurgVIL combines kinematically labeled phantom robot demonstrations with surgical videos from open-source datasets and online sources for policy learning. Since these videos lack robot motion labels, we estimate approximate kinematics as weak supervision. We evaluate SurgVIL on two da Vinci robot tasks: needle pick-up and cholecystectomy cutting. Across ACT, $\pi_0$, and GR00T-H backbones, adding surgical videos substantially improves generalization to real-tissue and out-of-distribution settings, suggesting a scalable path from phantom training toward generalizable surgical robot policies.