Search papers, labs, and topics across Lattice.
This paper introduces MoonSplat, a novel online voxelized 3D Gaussian Splatting (3DGS) framework that integrates global Sim(3) optimization to enhance camera pose estimation and rendering quality in real-time 3D reconstruction from monocular image sequences. By addressing challenges such as fragile camera tracking and low optimization efficiency, the method employs a color residual learning strategy to accelerate convergence and improve rendering fidelity. Extensive experiments validate that MoonSplat achieves state-of-the-art performance in both accuracy and efficiency, making it suitable for practical applications in robotics and AR/VR.
Real-time 3D reconstruction can now achieve state-of-the-art accuracy and rendering quality thanks to a novel integration of global optimization and color residual learning.
Online 3D reconstruction from monocular image sequences is a challenging and ongoing research topic. 3D Gaussian Splatting (3DGS), leveraging its high-quality real-time rendering capability, empowers online 3D reconstruction to represent dense scenes with enhanced expressiveness, and thus holds great promise for a wide range of applications such as robotics and AR/VR. However, existing online 3DGS methods still suffer from some key challenges: fragile camera pose estimation due to the lack of global optimization, and low optimization efficiency in large-scale or long-sequence scenarios. To address these issues, we propose a robust and efficient online voxelized 3DGS reconstruction framework integrated with global $\text{Sim}(3)$ optimization, which enables reliable camera tracking and efficient global loop closure for both camera poses and voxelized 3DGS. To accelerate the convergence of the voxelized 3DGS, we further introduce a color residual learning strategy, which not only boosts optimization speed but also enhances rendering quality. Extensive experiments on diverse indoor and outdoor datasets demonstrate that our method achieves state-of-the-art performance in both camera pose estimation accuracy and rendering quality, while retaining real-time efficiency. Additionally, we develop and deploy a real-world UAV-based active reconstruction system grounded on our proposed method, validating its robustness and generalizability for practical online 3D reconstruction tasks. Our code and data are available at https://github.com/TrickyGo/MoonSplat.
