Search papers, labs, and topics across Lattice.
This paper introduces Self-Geometry, a novel test-time adaptation pipeline that enforces explicit multi-view geometric consistency in Vision Foundation Models (VFMs) using 2D pixel correspondences as pseudo ground-truth. By integrating Geometric Disentanglement Optimization and a view sampling strategy based on SO(3) geodesic distances, the method effectively mitigates gradient conflicts while enhancing pose and geometry estimation. The results demonstrate significant performance improvements across six VFMs and four benchmark datasets, addressing the limitations of prior implicit consistency approaches.
Self-Geometry achieves substantial gains in 3D vision accuracy by directly enforcing geometric constraints at test time, outperforming existing implicit methods.
Recent Vision Foundation Models (VFMs) predict depth, camera pose, and pointmap in a single forward pass without per-scene optimization, achieving strong generalization. However, enforcing explicit multi-view geometric consistency, e.g., through bundle adjustment, is computationally costly and is thus not imposed during VFM pretraining, so such inconsistency can arise. To address this, implicit self-consistency derived from model outputs (e.g., pointmaps, features), though enforced at test-time in prior work, delivers inherently limited performance gain, especially on scenes where the pretrained VFM is highly inaccurate. In contrast to this implicit signal, we propose Self-Geometry, a plug-and-play test-time adaptation pipeline that directly imposes explicit multi-view geometric constraints using 2D pixel correspondences as pseudo ground-truth. Our proposed Self-Geometry consists of Geometric Disentanglement Optimization, which combines Multi-View Consistency and Epipolar Consistency losses with Gradient Disentanglement to prevent gradient conflict; Frame Angular-Neighbor, a view sampler based on SO(3) geodesic distances for lightly imposing these constraints; and Lightweight TTA, which adapts VFMs via LoRA. Our method achieves consistent improvements in both pose and geometry estimation across six VFMs (VGGT, $\pi^3$, DA3-Giant/Large/Base/Small) and four benchmarks (7Scenes, ETH3D, ScanNet++, HiRoom).
