Search papers, labs, and topics across Lattice.
STADA is introduced as a novel framework for generating autonomous driving scenarios directly from formal safety specifications written in LTLf. It systematically constructs initial scenes, generates diverse continuations, and runs simulations to validate the agent's behavior against the specification. Experiments using SCENEFLOW and three coverage criteria show STADA achieves significantly higher coverage (2x on the finest criteria, 75% on the coarsest) compared to existing test generation techniques, while also requiring fewer simulations.
Achieve 2x better coverage of autonomous driving safety requirements with 6x fewer simulations by automatically generating test scenarios from formal LTLf specifications.
Simulation-based testing has become a standard approach to validating autonomous driving agents prior to real-world deployment. A high-quality validation campaign will exercise an agent in diverse contexts comprised of varying static environments, e.g., lanes, intersections, signage, and dynamic elements, e.g., vehicles and pedestrians. To achieve this, existing test generation techniques rely on template-based, manually constructed, or random scenario generation. When applied to validate formally specified safety requirements, such methods either require significant human effort or run the risk of missing important behavior related to the requirement. To address this gap, we present STADA, a Specification-based Test generation framework for Autonomous Driving Agents that systematically generates the space of scenarios defined by a formal specification expressed in temporal logic (LTLf). Given a specification, STADA constructs all distinct initial scenes, a diverse space of continuations of those scenes, and simulations that reflect the behaviors of the specification. Evaluation of STADA on a variety of LTLf specifications formalized in SCENEFLOW using three complementary coverage criteria demonstrates that STADA yields more than 2x higher coverage than the best baseline on the finest criteria and a 75% increase for the coarsest criteria. Moreover, it matches the coverage of the best baseline with 6 times fewer simulations. While set in the context of autonomous driving, the approach is applicable to other domains with rich simulation environments.
