Search papers, labs, and topics across Lattice.
The paper introduces $R^3$, a novel approach to 3D reconstruction that uses relative regression to overcome the limitations of global coordinate frame assumptions in feed-forward geometry foundation models. By predicting confidence-weighted relative constraints with a lightweight MLP, $R^3$ avoids unbounded translation magnitudes and enables long-context and streaming reconstruction. Experiments in both offline and streaming settings demonstrate the effectiveness of the relative mechanism for improved 3D reconstruction.
Forget global coordinates: this new method unlocks long-context and streaming 3D reconstruction by predicting relative constraints between frames.
Recent feed-forward geometry foundation models have demonstrated impressive generalization by recovering depth and poses in a single forward pass. However, these models are typically constrained by a global coordinate frame assumption. This dependency becomes a significant bottleneck for long-context and streaming reconstruction, as it forces the network to maintain an arbitrary temporal origin and handle translation magnitudes that grow unbounded over time. Our solution, which we call $R^3$, employs relative regression. We employ a lightweight MLP to predict confidence-weighted relative constraints. These confidences serve as a unified anchor: weighting losses during training and guiding pose aggregation during inference. $R^3$ supports both full-context offline reconstruction and causal, bounded-memory streaming. Our evaluation in both offline and streaming settings validates the effectiveness of our relative mechanism. Project page: https://kevinxu02.github.io/r3-site
