Search papers, labs, and topics across Lattice.
This paper introduces DriveWeaver, a framework that utilizes point-conditioned video inpainting to insert vehicles with predefined trajectories into autonomous driving simulations. By addressing the limitations of existing methods that rely on pre-reconstructed 3D assets, DriveWeaver achieves seamless integration of vehicles into diverse scenes while maintaining visual realism and geometric consistency. Extensive experiments show that this approach significantly enhances scene diversity and supports scalable testing of autonomous driving models.
DriveWeaver achieves seamless vehicle insertion in simulations, outperforming traditional methods by ensuring visual realism and geometric consistency without the need for pre-reconstructed 3D assets.
A pivotal step in autonomous driving simulation involves inserting foreground vehicles with predefined trajectories into simulated scenes. This process enhances scene diversity and facilitates the creation of various corner cases for testing and improving autonomous driving models. However, existing methods often rely on pre-reconstructed 3D assets, which frequently lead to lighting inconsistencies between the inserted foreground and the background. Moreover, the reliance on limited, manually-curated 3D assets hinders large-scale deployment. To address these challenges, we propose DriveWeaver, a novel framework for controllable vehicle insertion in autonomous driving simulation. Specifically, for a masked target insertion area, DriveWeaver performs video inpainting conditioned on vehicle point clouds to generate high-quality, temporally consistent vehicles. This video-inpainting-based approach ensures seamless blending between the foreground and background, while the readily available point cloud conditions enable superior generalization. To support long-term generation, we further design a global-to-local hierarchical inpainting strategy, ensuring the consistent identity and appearance of the inserted vehicles. Meanwhile, we extract explicit 3D Gaussian representations of the inserted vehicles through an urban reconstruction pipeline to enable real-time rendering for autonomous driving simulation. Extensive experiments across diverse datasets demonstrate that our method outperforms existing baselines in visual realism and geometric consistency, providing a robust tool for scalable autonomous driving scene augmentation.
