Search papers, labs, and topics across Lattice.
This paper analyzes the limitations of 3D Gaussian Splatting (3DGS) for pose refinement, identifying pose prior uncertainty and geometric uncertainty as key error sources. To address these, the authors propose a relocalization framework that combines Monte Carlo pose sampling to handle pose prior uncertainty with Fisher Information-based PnP optimization to account for geometric uncertainty. Experiments on indoor and outdoor datasets demonstrate improved localization accuracy and robustness to noise compared to existing 3DGS-based pose refinement methods, without requiring retraining or extra supervision.
Even with high-quality differentiable rendering, 3D Gaussian Splatting's pose refinement falters due to pose prior and geometric uncertainties, but this work shows how to overcome these limitations with a novel relocalization framework.
3D Gaussian Splatting (3DGS) has recently emerged as a powerful scene representation and is increasingly used for visual localization and pose refinement. However, despite its high-quality differentiable rendering, the robustness of 3DGS-based pose refinement remains highly sensitive to both the initial camera pose and the reconstructed geometry. In this work, we take a closer look at these limitations and identify two major sources of uncertainty: (i) pose prior uncertainty, which often arises from regression or retrieval models that output a single deterministic estimate, and (ii) geometric uncertainty, caused by imperfections in the 3DGS reconstruction that propagate errors into PnP solvers. Such uncertainties can distort reprojection geometry and destabilize optimization, even when the rendered appearance still looks plausible. To address these uncertainties, we introduce a relocalization framework that combines Monte Carlo pose sampling with Fisher Information-based PnP optimization. Our method explicitly accounts for both pose and geometric uncertainty and requires no retraining or additional supervision. Across diverse indoor and outdoor benchmarks, our approach consistently improves localization accuracy and significantly increases stability under pose and depth noise.
