Search papers, labs, and topics across Lattice.
This paper introduces HyVGGT-VO, a tightly coupled visual odometry framework that combines traditional sparse VO with the dense reconstruction capabilities of the VGGT feed-forward model. The system uses an adaptive hybrid tracking frontend, switching between optical flow and the VGGT tracking head for robustness, and incorporates a hierarchical optimization framework to refine VO poses and VGGT scale for global consistency. Results on EuRoC and KITTI datasets demonstrate a 5x speedup and significant error reduction compared to existing VGGT-based methods.
Achieve real-time dense visual odometry by cleverly fusing classic sparse methods with modern feed-forward networks, outperforming end-to-end approaches by a wide margin.
Dense visual odometry (VO), which provides pose estimation and dense 3D reconstruction, serves as the cornerstone for applications ranging from robotics to augmented reality. Recently, feed-forward models have demonstrated remarkable capabilities in dense mapping. However, when these models are used in dense visual SLAM systems, their heavy computational burden restricts them to yielding sparse pose outputs at keyframes while still failing to achieve real-time pose estimation. In contrast, traditional sparse methods provide high computational efficiency and high-frequency pose outputs, but lack the capability for dense reconstruction. To address these limitations, we propose HyVGGT-VO, a novel framework that combines the computational efficiency of sparse VO with the dense reconstruction capabilities of feed-forward models. To the best of our knowledge, this is the first work to tightly couple a traditional VO framework with VGGT, a state-of-the-art feed-forward model. Specifically, we design an adaptive hybrid tracking frontend that dynamically switches between traditional optical flow and the VGGT tracking head to ensure robustness. Furthermore, we introduce a hierarchical optimization framework that jointly refines VO poses and the scale of VGGT predictions to ensure global scale consistency. Our approach achieves an approximately 5x processing speedup compared to existing VGGT-based methods, while reducing the average trajectory error by 85% on the indoor EuRoC dataset and 12% on the outdoor KITTI benchmark. Our code will be publicly available upon acceptance. Project page: https://geneta2580.github.io/HyVGGT-VO.io.
