Search papers, labs, and topics across Lattice.
GA-Drive is introduced, a novel driving scene generation framework that decouples geometry and appearance to enable free-viewpoint rendering and editing. The method synthesizes novel views using scene geometry and then refines them into photorealistic images using a video diffusion model. This decoupling allows for appearance editing while preserving geometric consistency across different viewpoints and trajectories.
Edit the appearance of driving scenes and generate consistent free-viewpoint images along novel trajectories, opening new possibilities for autonomous driving simulation.
A free-viewpoint, editable, and high-fidelity driving simulator is crucial for training and evaluating end-to-end autonomous driving systems. In this paper, we present GA-Drive, a novel simulation framework capable of generating camera views along user-specified novel trajectories through Geometry-Appearance Decoupling and Diffusion-Based Generation. Given a set of images captured along a recorded trajectory and the corresponding scene geometry, GA-Drive synthesizes novel pseudo-views using geometry information. These pseudo-views are then transformed into photorealistic views using a trained video diffusion model. In this way, we decouple the geometry and appearance of scenes. An advantage of such decoupling is its support for appearance editing via state-of-the-art video-to-video editing techniques, while preserving the underlying geometry, enabling consistent edits across both original and novel trajectories. Extensive experiments demonstrate that GA-Drive substantially outperforms existing methods in terms of NTA-IoU, NTL-IoU, and FID scores.
