Search papers, labs, and topics across Lattice.
The paper introduces Flow4R, a unified framework for dynamic 3D scene reconstruction and tracking that uses camera-space scene flow as the central representation. A Vision Transformer predicts per-pixel 3D position, scene flow, pose weight, and confidence from two-view inputs, enabling joint inference of geometry and motion. By avoiding explicit pose regressors and bundle adjustment, Flow4R achieves state-of-the-art performance on 4D reconstruction and tracking tasks after being trained on both static and dynamic datasets.
By predicting camera-space scene flow directly, Flow4R elegantly unifies 3D reconstruction, object motion, and camera motion without explicit pose regressors or bundle adjustment.
Reconstructing and tracking dynamic 3D scenes remains a fundamental challenge in computer vision. Existing approaches often decouple geometry from motion: multi-view reconstruction methods assume static scenes, while dynamic tracking frameworks rely on explicit camera pose estimation or separate motion models. We propose Flow4R, a unified framework that treats camera-space scene flow as the central representation linking 3D structure, object motion, and camera motion. Flow4R predicts a minimal per-pixel property set-3D point position, scene flow, pose weight, and confidence-from two-view inputs using a Vision Transformer. This flow-centric formulation allows local geometry and bidirectional motion to be inferred symmetrically with a shared decoder in a single forward pass, without requiring explicit pose regressors or bundle adjustment. Trained jointly on static and dynamic datasets, Flow4R achieves state-of-the-art performance on 4D reconstruction and tracking tasks, demonstrating the effectiveness of the flow-central representation for spatiotemporal scene understanding.
