Search papers, labs, and topics across Lattice.
This paper introduces an agent-driven simulation framework for evaluating autonomous driving systems, utilizing instruction-following large language models to control surrounding road participants. By enabling intentional and reactive behaviors while maintaining physical plausibility, the framework addresses the limitations of existing simulators that rely on log replay or rule-based agents. Evaluation through the SemanticPlan benchmark reveals that current state-of-the-art planners struggle with safe and effective task completion in diverse long-tail scenarios, highlighting the ongoing challenges in autonomous driving evaluation.
State-of-the-art planners falter in long-tail scenarios, revealing critical gaps in autonomous driving safety and effectiveness.
Evaluating autonomous driving systems in closed-loop settings requires realistic and interactive simulation, yet existing simulators largely rely on log replay or rule-based agents, limiting behavioral diversity and long-tail coverage. We propose an agent-driven simulation framework in which surrounding road participants are controlled by instruction-following large language models through a structured action interface, enabling intentional and reactive behaviors while preserving physical plausibility. Furthermore, we introduce SemanticPlan, a benchmark of closed-loop planning in long-tail and semantically rich scenarios that augment real nuPlan scenes with multiple interactive agents following diverse language instructions. Evaluation results show that state-of-the-art planners still struggle to consistently achieve safe and effective task completion, suggesting that these long-tail scenarios remain challenging.
Tsinghua AI