Search papers, labs, and topics across Lattice.
This paper introduces Shift&Drift, a dual-track benchmark that rigorously evaluates autonomous driving motion planners under distribution shifts, focusing on semantic and state-distribution challenges. By transforming the DeepScenario Open 3D dataset for use in the nuPlan simulation framework, the authors enable zero-shot testing of planners across diverse urban environments, revealing significant vulnerabilities in imitation learning methods, especially in pedestrian-rich scenarios. The study finds that while imitation learning excels in in-distribution performance, reinforcement learning approaches demonstrate superior robustness and safety in the face of execution perturbations, highlighting a critical trade-off in motion planning strategies.
Imitation learning methods may shine in controlled environments, but they falter dramatically in real-world urban settings, revealing a stark trade-off in motion planning resilience.
While closed-loop motion planners trained on large-scale, object-level datasets, e.g., nuPlan, demonstrate strong in-distribution (ID) performance, their generalization to novel urban topologies and recovery mechanisms following execution perturbations remain under-explored. To address this, we present Shift&Drift, a novel dual-track benchmark designed to rigorously stress-test motion planners across two critical axes of distribution shift: (1) The Semantic Shift Track leverages a novel conversion pipeline that transforms the aerial, DeepScenario Open 3D dataset into the nuPlan simulation framework. This enables zero-shot evaluation of planners trained on North American and Singaporean data against 1,182 scenarios spanning four German cities and the US city of San Francisco featuring dense pedestrian-cyclist interactions. (2) The State-Distribution Drift Track injects stochastic perturbations into the ego vehicle's dynamics to quantify robustness against compounding execution errors. Based on this, we systematically evaluate the failure modes of diverse planning paradigms under semantic and state-distribution shifts. While imitation learning methods achieve high scores in ID benchmarks, they exhibit significant failures under semantic shift, particularly in pedestrian-dense environments, and suffer from persistent drift when subjected to temporally correlated actuation noise. In contrast, the evaluated reinforcement-learning-based planner demonstrates more graceful degradation, maintaining higher safety and progress metrics across both tracks. Our findings reveal an empirical trade-off between imitation fidelity and closed-loop resilience, providing the community with a rigorous benchmark to evaluate progress toward reliable deployment.
