Search papers, labs, and topics across Lattice.
The authors formulate vision-language driver alerting as an adaptive tri-action policy over SILENT, OBSERVE, and ALERT, replacing brittle static risk score thresholding with an explicit evidence-gathering mechanism that dynamically shifts the observation window. Moving beyond binary risk anticipation allows safety models to defer uncertain decisions in ambiguous visual scenes under partial observability. Evaluated on the unified VLAlert-Bench and held-out naturalistic ADAS takeover clips, the framework improves AUROC from 0.610 to 0.689 and increases 5-second takeover recall from 74.2% to 88.7%.
Giving vision-language models an explicit action to defer judgment and gather more evidence boosts safety-critical takeover recall from 74.2% to 88.7% over conventional threshold-based alerting.
Driver alerting from dashcam video requires sequential decision-making under partial observability: a system must decide not only whether a scene is risky, but also when the evidence is sufficient to warn. Most existing accident anticipation models output a binary risk score, leaving ambiguous scenes to be handled by thresholding. We propose VLAlert, a vision-language alerting framework that casts warning generation as a tri-action policy over SILENT, OBSERVE, and ALERT. The OBSERVE action acts as an internal evidence-gathering decision that delays uncertain warnings and changes the next observation window, creating a lightweight perception-action loop for adaptive alerting. VLAlert uses Qwen3-VL-4B as a safety-evidence generator and pools hidden states from structured belief spans to form compact representations for danger estimation and policy prediction. We evaluate VLAlert on VLAlert-Bench, a unified per-tick benchmark from four real-world dashcam alert datasets, and further test transfer to held-out naturalistic ADAS takeover clips. On VLAlert-Bench validation, VLAlert achieves the highest deployment-oriented utility among tested baselines, with DAUS 0.4878 compared with 0.4752 for Open-BADAS, and improves AUROC, AP_tick, F1_t, and balanced accuracy from 0.610, 0.176, 0.276, and 0.581 to 0.689, 0.195, 0.297, and 0.648, respectively. On 221 held-out ADAS-TO-Critic clips, VLAlert improves R@5s from 74.2% to 88.7% and F1 from 0.585 to 0.686. These results indicate that adaptive observation and safety-focused VLM representations provide measurable gains for driver-facing alert decisions.
