Search papers, labs, and topics across Lattice.
The paper introduces UFO, a recurrent neural network architecture for 4D driving scene reconstruction that unifies feed-forward and optimization-based approaches to address the limitations of existing feed-forward methods in handling long-range sequences. UFO iteratively refines a 4D scene representation using a visibility-based filtering mechanism for efficient processing and incorporates an object pose-guided modeling approach for accurate dynamic object tracking. Experiments on the Waymo Open Dataset demonstrate that UFO outperforms both per-scene optimization and existing feed-forward methods, achieving faster reconstruction times (0.5s for 16s logs) with improved visual quality and geometric accuracy.
Forget quadratic complexity: UFO reconstructs 16-second driving scenes in half a second by unifying feed-forward and optimization-based methods.
Dynamic driving scene reconstruction is critical for autonomous driving simulation and closed-loop learning. While recent feed-forward methods have shown promise for 3D reconstruction, they struggle with long-range driving sequences due to quadratic complexity in sequence length and challenges in modeling dynamic objects over extended durations. We propose UFO, a novel recurrent paradigm that combines the benefits of optimization-based and feed-forward methods for efficient long-range 4D reconstruction. Our approach maintains a 4D scene representation that is iteratively refined as new observations arrive, using a visibility-based filtering mechanism to select informative scene tokens and enable efficient processing of long sequences. For dynamic objects, we introduce an object pose-guided modeling approach that supports accurate long-range motion capture. Experiments on the Waymo Open Dataset demonstrate that our method significantly outperforms both per-scene optimization and existing feed-forward methods across various sequence lengths. Notably, our approach can reconstruct 16-second driving logs within 0.5 second while maintaining superior visual quality and geometric accuracy.
