Search papers, labs, and topics across Lattice.
This paper introduces SliceScorer, a deterministic scoring rule designed to identify missing slices in driving vision-language models (VLMs) by utilizing an exposure-based coverage prior and a neighbor-failure prior. The method enhances the interpretability and auditability of the verification process, which is crucial for safety-critical applications, and is integrated into the SliceNav pipeline for orchestrating verification workflows. Experiments reveal that SliceNav outperforms existing slice-discovery methods in identifying high-risk coverage gaps while ensuring diverse recommendations across various conditions.
SliceScorer reveals critical coverage gaps in driving VLMs that traditional methods overlook, enhancing safety validation in real-world applications.
Driving vision-language models (VLMs) must accurately understand scenes across diverse conditions defined by Operational Design Domains (ODDs), yet verification remains sparse: many slices are missing, making empirical failure rates unreliable. We propose SliceScorer, a deterministic scoring rule for missing-slice recommendation that combines (i) an exposure-based coverage prior to prioritize rare, under-tested regions, and (ii) a neighbor-failure prior that propagates risk from similar tested conditions. SliceScorer is deliberately simple - interpretable, auditable, and conservative - properties essential for safety-critical validation. For stress testing beyond the declared ODD, we embed SliceScorer within SliceNav, an LLM-orchestrated verification pipeline where the model interprets developer queries to select relevant operators (triage, scoring, acquisition, evaluation) and vocabulary extensions, composing verification workflows while keeping all scoring deterministic and auditable. Experiments on three driving VLMs (WiseAD, DriveMM, Cosmos-Reason2-2B) show that SliceNav surfaces high-risk coverage gaps more effectively than prior slice-discovery methods while maintaining diverse recommendations across the condition space. Ablations confirm both scoring components contribute, and qualitative analysis demonstrates end-to-end workflows from developer query to targeted evaluation.
