Search papers, labs, and topics across Lattice.
This paper introduces DriveJudge, a novel evaluation agent for autonomous driving that integrates rule-based metrics with Vision-Language Model (VLM) reasoning to enhance context-aware driving assessments. By curating a dataset of 33,577 annotated driving scenarios, the authors tackle the challenge of evaluating driving quality in a way that is both interpretable and grounded in physical reality. DriveJudge significantly outperforms existing methods, achieving a 21.23 AUC improvement over traditional metrics and a 6.5% enhancement over the latest VLM-based evaluations, thereby setting a new benchmark in the field.
DriveJudge redefines autonomous driving evaluation by achieving unprecedented accuracy in context-aware assessments, outperforming traditional metrics by over 21 AUC points.
Autonomous driving has shifted towards end-to-end policy learning, where reliable, interpretable policy evaluation is a fundamental challenge as driving quality is highly context-dependent. Commonly used rule-based driving metrics like EPDMS are interpretable but lack context-awareness, while recent VLMbased evaluations are context-aware but limited by ambiguous VLM outputs and weak physical grounding. To evaluate driving in a manner that is both interpretable and context-aware, we introduce DriveJudge. DriveJudge is a driving evaluation agent that combines rule-grounded evaluation with Vision-Language Model (VLM) reasoning and selectively invokes physically-grounded deterministic rule functions after interpreting the environmental context. To train and evaluate DriveJudge, we curate a large-scale dataset of 33,577 challenging driving samples with human annotations on whether the driving behavior is reasonable in the given scenario. With this dataset, we address the underexplored problem of driving metric evaluation, and introduce two human-aligned benchmark tasks: Driving Quality Classification and Trajectory Preference Selection. DriveJudge outperforms EPDMS for driving quality classification by 21.23 AUC, and the recent VLM-based DriveCritic for trajectory preference selection by 6.5%, setting a new standard for interpretable and precise driving evaluation.
