Search papers, labs, and topics across Lattice.
CrashSight, a new vision-language benchmark, is introduced to evaluate traffic crash understanding using real-world roadside camera data, comprising 250 crash videos and 13K QA pairs. The benchmark is structured in a two-tier taxonomy, assessing both visual grounding and higher-level reasoning skills like causal attribution and temporal progression. Benchmarking 8 state-of-the-art VLMs reveals that while these models exhibit strong scene description, they struggle with temporal and causal reasoning in safety-critical crash scenarios.
Despite advances in vision-language models, they still struggle to reason about the complex dynamics of traffic crashes, highlighting a critical gap in safety-critical applications.
Cooperative autonomous driving requires traffic scene understanding from both vehicle and infrastructure perspectives. While vision-language models (VLMs) show strong general reasoning capabilities, their performance in safety-critical traffic scenarios remains insufficiently evaluated due to the ego-vehicle focus of existing benchmarks. To bridge this gap, we present \textbf{CrashSight}, a large-scale vision-language benchmark for roadway crash understanding using real-world roadside camera data. The dataset comprises 250 crash videos, annotated with 13K multiple-choice question-answer pairs organized under a two-tier taxonomy. Tier 1 evaluates the visual grounding of scene context and involved parties, while Tier 2 probes higher-level reasoning, including crash mechanics, causal attribution, temporal progression, and post-crash outcomes. We benchmark 8 state-of-the-art VLMs and show that, despite strong scene description capabilities, current models struggle with temporal and causal reasoning in safety-critical scenarios. We provide a detailed analysis of failure scenarios and discuss directions for improving VLM crash understanding. The benchmark provides a standardized evaluation framework for infrastructure-assisted perception in cooperative autonomous driving. The CrashSight benchmark, including the full dataset and code, is accessible at https://mcgrche.github.io/crashsight.
