Search papers, labs, and topics across Lattice.
This paper introduces Asset Harvester, a comprehensive pipeline that transforms sparse observations from autonomous driving logs into complete, simulation-ready 3D assets for enhanced AV testing and training. The approach integrates large-scale data curation, geometry-aware preprocessing, and a robust training framework that combines multiview generation with 3D Gaussian lifting, specifically addressing the challenges posed by limited-angle views. The results demonstrate that Asset Harvester can effectively generate reusable 3D assets, significantly improving the fidelity and utility of simulations in AV development.
Transforming sparse driving log observations into complete 3D assets could revolutionize simulation fidelity in autonomous vehicle development.
Closed-loop simulation is a core component of autonomous vehicle (AV) development, enabling scalable testing, training, and safety validation before real-world deployment. Neural scene reconstruction converts driving logs into interactive 3D environments for simulation, but it does not produce complete 3D object assets required for agent manipulation and large-viewpoint novel-view synthesis. To address this challenge, we present Asset Harvester, an image-to-3D model and end-to-end pipeline that converts sparse, in-the-wild object observations from real driving logs into complete, simulation-ready assets. Rather than relying on a single model component, we developed a system-level design for real-world AV data that combines large-scale curation of object-centric training tuples, geometry-aware preprocessing across heterogeneous sensors, and a robust training recipe that couples sparse-view-conditioned multiview generation with 3D Gaussian lifting. Within this system, SparseViewDiT is explicitly designed to address limited-angle views and other real-world data challenges. Together with hybrid data curation, augmentation, and self-distillation, this system enables scalable conversion of sparse AV object observations into reusable 3D assets.
