Search papers, labs, and topics across Lattice.
This paper investigates factors influencing feed-forward visual geometry estimation, finding that data diversity/quality is key, confidence-aware losses can be detrimental, and joint per-frame/sequence supervision is beneficial. They introduce a consistency loss aligning depth maps, camera parameters, and point clouds, along with a high-resolution architecture. The resulting CARVE model demonstrates strong performance on point cloud reconstruction, video depth estimation, and camera pose/intrinsic estimation.
Seemingly innocuous choices in loss functions and training regimes can significantly hinder visual geometry estimation, even for state-of-the-art methods.
Feed-forward visual geometry estimation has recently made rapid progress. However, an important gap remains: multi-frame models usually produce better cross-frame consistency, yet they often underperform strong per-frame methods on single-frame accuracy. This observation motivates our systematic investigation into the critical factors driving model performance through rigorous ablation studies, which reveals several key insights: 1) Scaling up data diversity and quality unlocks further performance gains even in state-of-the-art visual geometry estimation methods; 2) Commonly adopted confidence-aware loss and gradient-based loss mechanisms may unintentionally hinder performance; 3) Joint supervision through both per-sequence and per-frame alignment improves results, while local region alignment surprisingly degrades performance. Furthermore, we introduce two enhancements to integrate the advantages of optimization-based methods and high-resolution inputs: a consistency loss function that enforces alignment between depth maps, camera parameters, and point maps, and an efficient architectural design that leverages high-resolution information. We integrate these designs into CARVE, a resolution-enhanced model for feed-forward visual geometry estimation. Experiments on point cloud reconstruction, video depth estimation, and camera pose/intrinsic estimation show that CARVE achieves strong and robust performance across diverse benchmarks.
