Search papers, labs, and topics across Lattice.
This paper introduces OpenLongTail, an open-source generative data engine designed to enhance the training of autonomous driving policies by synthesizing long-tail driving data from heterogeneous sources. By employing a pose-informed extrapolative view synthesis pipeline, the authors generate missing multi-view assets that improve the robustness of driving policies in edge cases, addressing the critical gap in existing datasets. The results show a marked increase in closed-loop driving performance when trained with the synthesized data, demonstrating the effectiveness of the approach in improving long-tail generalization.
Generating missing multi-view data from diverse driving videos boosts closed-loop driving robustness in edge cases by over 30%.
Scaling robust driving policies is fundamentally bottlenecked by the scarcity of edge cases in curated datasets. While the real world continuously captures these critical events, such long-tail events remain underutilized when collected from heterogeneous sources. Specifically, diverse but valuable in-the-wild long-tail videos lack the full view coverage required for training policy models, often missing multi-view poses or originating solely from monocular dash cameras. This modality gap prevents these ubiquitous observations from being converted into scalable training data for long-tail generalization. We introduce OpenLongTail, an open-source generative data engine for scaling autonomous driving policies under long-tail events. To transform heterogeneous data sources into view-aligned and temporally coherent multi-view assets that are useful for policy learning, we develop a pose-informed extrapolative view synthesis pipeline that generates the missing views. We further enhance cross-view consistency and the temporal alignment for the newly generated views by injecting Pl\"ucker ray geometry into the scalable generation engine. By synthesizing heterogeneous long-tail data, we observe a significant improvement in closed-loop driving robustness in handling long-tail events. By measuring the extrapolative view synthesis and pose metrics, we validate the effectiveness of OpenLongTail in visual fidelity, cross-view consistency, and ego-trajectory recovery.
