Search papers, labs, and topics across Lattice.
This paper introduces Track2Map, an online 3D Gaussian Splatting pipeline designed for real-time deformable scene reconstruction in robotic surgery, overcoming the limitations of offline methods that rely on accurate camera trajectory priors. By jointly optimizing camera trajectory and 3D scene representation from surgical video, Track2Map enhances robustness against noisy or absent priors, effectively functioning as a Simultaneous Localization and Mapping (SLAM) method. Experimental results demonstrate that Track2Map significantly improves both reconstruction quality and camera trajectory accuracy compared to existing SLAM and non-SLAM approaches.
Track2Map achieves real-time 3D reconstruction in robotic surgery, even when camera trajectory data is unreliable or missing.
Gaussian splatting is the current state-of-the-art for dense, deformable 3D anatomy reconstruction in robot-assisted minimally invasive surgery (RAMIS); however, most pipelines are offline and depend on accurate camera trajectory priors (often from robotic kinematics), limiting applicability when priors are missing or noisy. To address these limitations, we propose Track2Map, an online 3D Gaussian Splatting pipeline that jointly optimizes camera trajectory and 3D deformable scene representation directly from surgical video. Track2Map is therefore capable of robust 3D reconstructions when camera trajectory priors are either absent or noisy, and due to its online nature it effectively works as a Simultaneous Localisation and Mapping (SLAM) method. To stabilize optimization in the presence of tissue motion and ambiguous visual cues, we introduce a track-anchored deformation initialization using dense 2D point tracks. Track statistics are further utilized to disentangle camera motion from scene deformation by detecting static camera periods and reducing drift during incremental mapping. Experiments on StereoMIS show improved reconstruction quality and camera trajectory against competing SLAM methods, as well as compared to non-SLAM methods that utilize camera trajectory priors. The code is available at https://track2map.github.io/.
