Search papers, labs, and topics across Lattice.
Addressing catastrophic drift in long-sequence online 3D reconstruction, Scal3R reformulates camera tracking as a multi-reference relative pose query problem using lightweight tokens injected into a frozen backbone via asymmetric attention. This approach leverages the insight that vision backbones retain accurate local geometry even when absolute first-frame pose regression collapses, pairing the multi-frame queries with an online pose-graph optimization backend for robust loop closure. The resulting system trains to convergence in just 8 hours on a single GPU, slashes average absolute trajectory error by over 60% on KITTI, and achieves state-of-the-art results across six standard benchmarks.
Online 3D reconstruction failure on long videos is not a representation collapse but an artifact of single-anchor pose extrapolation鈥攁nd querying relative poses across multiple keyframes fixes it with just 1% parameter overhead.
Online 3D reconstruction models perform poorly on long videos. This happens because regressing poses relative to a fixed first-frame anchor forces extrapolation far beyond the training distribution. Small drifts accumulate and amplify into significant geometric collapse. However, we observe that per-frame depth remains stable throughout this failure. The backbone's local geometry remains intact; only the global pose head breaks down. Motivated by this decoupling, we introduce Scal3R. This approach reformulates online reconstruction as multi-reference relative pose querying. We use lightweight learnable tokens, which make up about ~1% of the parameters, and inject them into a completely frozen backbone via asymmetric attention. This setup queries poses relative to multiple past keyframes. An online pose-graph optimization system with loop closure suppresses long-range drift. Scal3R reaches convergence in 8 hours on a single GPU. It reduces the average ATE by over 60% on KITTI compared to the online baseline. It also achieves state-of-the-art performance across Virtual KITTI, Sintel, TUM-Dynamic, ScanNet, and 7-Scenes. Project page: https://linjohnss.github.io/scal3r/
