Search papers, labs, and topics across Lattice.
This paper introduces CAViAR, a comprehensive dataset designed to enhance causal reasoning in autonomous driving by providing 2,249 annotated dashcam videos of real-world accidents. The annotations include detailed labels for environmental conditions, accident types, and responsibility assessments, addressing a significant gap in current benchmarks that focus primarily on perception. Benchmarking state-of-the-art vision-language models reveals a pronounced Perception鈥揜easoning Gap, where models excel in recognizing context but struggle with accurately determining fault and rule violations in driving scenarios.
Current vision-language models can identify context in driving videos but fail to accurately assign fault in accidents, revealing a critical gap in autonomous driving AI capabilities.
While modern autonomous driving systems excel at perception tasks such as object detection and trajectory prediction, they lack the high-level causal reasoning required to interpret traffic accidents. In particular, determining responsibility, such as identifying who is at fault and which traffic rule was violated, remains largely unexplored in current benchmarks. To this end, we introduce CAViAR (Causal Accident Video and Incident Analysis Repository), a human-annotated dashcam benchmark comprising 2,249 real-world accident videos collected from CarCrashDataset (CCD) and Nexar. Each video is annotated with structured labels spanning environmental conditions, accident type, causal explanation, apparent At-Fault Agent, affected agent, and apparent rule-violation category. We benchmark state-of-the-art vision-language models (VLMs), including Cosmos-Reason2, Qwen3-VL, and InternVL3. Once class imbalance is accounted for with majority/random baselines and balanced metrics, perceptual competence is uneven--lighting is nearly solved, whereas weather and road-condition accuracy fall at or below the majority-class baseline---and all models degrade sharply on accident type and responsibility reasoning. Overall, CAViAR exposes a practical Perception--Reasoning Gap: current VLMs may recognize salient context, but do not reliably map visible agent actions to annotated rule-relevant responsibility categories in safety-critical driving scenarios. Code, annotation schema, prompts, and evaluation scripts are available at: https://github.com/nec-labs-ma/CAViAR
