Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Stipple, a method that integrates real-time visual-inertial tracking with an efficient incremental approach to 3D Gaussian Splatting (3DGS) for immediate scene reconstruction. By leveraging visual-inertial odometry and a novel Rust-based implementation, the authors significantly reduce the preprocessing and training overhead traditionally associated with 3DGS, allowing for real-time performance in robotics and XR applications. The results demonstrate that this combined approach not only enhances the efficiency of the training pipeline but also enables seamless tracking and reconstruction in dynamic environments.
Real-time 3D reconstruction is now feasible by merging visual-inertial tracking with an innovative incremental Gaussian splatting technique.
3D Gaussian Splatting (3DGS) provides efficient rendering of photo-realistic scenes, but its heavy preprocessing and training steps make it a poor fit for applications that require real-time reconstruction in robotics or XR. This capability is important since it allows immediate feedback and interaction with new environments. Visual-inertial odometry (VIO) and simultaneous localization and mapping (VI-SLAM) systems, on the other hand, specifically target these real-time applications, which makes them a good choice for integration with 3DGS. We propose a new method that tracks and reconstructs simultaneously in real-time by leveraging an efficient visual-inertial tracking system based on Basalt together with a novel incremental method built on top of Brush, an efficient Rust-based GPU-vendor-agnostic implementation of 3D Gaussian Splatting. We show that many of the heavy preprocessing and training steps of 3DGS can be replaced with a more efficient incremental training strategy that has direct access to the information generated by the visual-inertial tracking system. Furthermore, we propose and combine multiple practical improvements to increase the efficiency of the training pipeline and adapt it to run in real-time, parallel to the tracking thread. This work highlights the value of exploiting the complementary nature of SLAM and 3DGS, and how that can lead to promising results for real-time 3D reconstruction.
TU Munich