Search papers, labs, and topics across Lattice.
This paper introduces VI3, a model-agnostic framework that enhances pretrained 3D foundation models (3DFMs) by integrating inertial measurement unit (IMU) data to achieve accurate metric scale predictions. By leveraging IMU readings, VI3 effectively anchors the outputs of 3DFMs, allowing for improved scale recovery without the need for ground-truth supervision. Experimental results on both synthetic and real aerial datasets show that VI3 maintains geometric consistency and adapts to varying conditions of motion, significantly refining scale estimation in 3D modeling tasks.
VI3 achieves accurate metric scale recovery for 3D models using only inertial data, eliminating the need for ground-truth supervision.
3D foundation models (3DFMs) excel at predicting camera poses and dense depth from multiple views of a scene, showcasing strong zero-shot generalization. However, as metric scale is not observable from monocular images, their absolute scale predictions are typically inaccurate. Inertial measurement units (IMUs), present in most devices, naturally complement monocular cameras by observing scaled motion. We introduce VI3, a model-agnostic framework that metrically anchors a pretrained 3DFM using only IMU readings. VI3 initializes and preintegrates the IMU to obtain a metric motion reference, which is then used to recover the scale of the 3DFM outputs. Our method includes adaptable anchoring strategies tailored to diverse 3DFM architectures. Experiments on synthetic and real aerial datasets demonstrate that VI3 recovers metric scale without ground-truth supervision while preserving geometric consistency, acting as a fine refinement under well-conditioned motion and as a strong prior when motion is less informative.
