Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper addresses significant tracking drift in feedforward foundation models for long-context streaming 3D reconstruction by introducing CLoSeR, a novel loop closure mechanism. CLoSeR employs global descriptor retrieval to identify loop candidates and constructs loop-conditioned windows for relative pose estimation, subsequently optimizing frame poses on the SE(3) manifold. Experimental results demonstrate that CLoSeR effectively reduces drift and yields consistent kilometer-scale reconstructions, surpassing existing state-of-the-art methods.
Kilometer-scale 3D reconstructions are now drift-free by closing the loop with SE(3) pose optimization, bypassing Sim(3) limitations.
Feedforward foundation models have recently shown remarkable 3D reconstruction capabilities. However, existing models exhibit large tracking drift in long-context streaming reconstruction due to error accumulation. In this paper, we revisit loop closure with streaming reconstruction foundation models to enable accurate, drift-free, kilometer-scale reconstruction. Specifically, our method detects loop candidates through global descriptor retrieval, and constructs loop-conditioned windows to estimate the relative poses between looped frames. Given the observation that our adopted streaming reconstruction backbone produces a globally consistent scale, we optimize all frame poses on the SE(3) manifold with sequential and loop closure constraints, avoiding the pose graph optimization on the Sim(3) or higher-dimensional SL(4) manifolds employed in prior works. Extensive experiments show that our method reduces drift and produces consistent geometry on kilometer-scale sequences, significantly outperforming the state of the art. Code is available at https://github.com/MoyangLi00/CLoSeR.git.
ETH
Mila