Search papers, labs, and topics across Lattice.
This paper introduces Envision4D, a self-supervised feed-forward framework designed to enhance future scene extrapolation in autonomous driving by addressing the limitations of existing interpolation-focused methods. The framework incorporates a Future Pose Prediction module and employs In-layer Temporal Attention alongside Conditioned Motion Lifting to effectively manage non-linear dynamics and mitigate ghosting artifacts. Extensive experiments reveal that Envision4D significantly surpasses current state-of-the-art techniques in future view synthesis, showcasing its potential for real-world applications in autonomous navigation.
Envision4D transforms future scene extrapolation in autonomous driving, achieving unprecedented accuracy and stability in dynamic environments.
Forecasting the future evolution of dynamic scenes is crucial in autonomous driving. However, existing feed-forward paradigms are primarily designed for interpolation. When extended to future extrapolation, they suffer from ghosting artifacts under large displacements and are constrained by simplified motion assumptions or strict future priors. To overcome these challenges, we propose Envision4D, a fully self-supervised feed-forward framework for pose-free future extrapolation. Specifically, we introduce a Future Pose Prediction module that infers future camera parameters via an iterative denoising process. Furthermore, to capture non-linear dynamics, we propose In-layer Temporal Attention and employ Conditioned Motion Lifting, which transforms the highly uncertain extrapolation process into robust relational mappings. Finally, a Progressive Training Strategy is utilized to stabilize unsupervised motion learning against error accumulation. Extensive experiments demonstrate that Envision4D achieves state-of-the-art performance, significantly outperforming existing methods in future view synthesis.
Tsinghua AI