Search papers, labs, and topics across Lattice.
This paper introduces Safety-Aware Mutation Testing (SAMT) as a novel approach to enhance the safety evaluation of Autonomous Driving Systems (ADS) by systematically injecting temporally bounded faults into inter-module communications. Unlike traditional mutation testing that focuses on individual components, SAMT leverages top-down safety engineering frameworks to ensure that generated mutants accurately represent realistic hazards, thereby improving the relevance of test scenarios. The authors argue that this method not only provides a more rigorous measure of test adequacy but also facilitates automated scenario generation and guides effective ADS repairs.
SAMT transforms how we test ADS by focusing on realistic interaction failures rather than isolated component faults, ensuring that safety evaluations are both comprehensive and relevant.
Simulation-based testing is essential for ensuring the safety of Autonomous Driving Systems (ADS), yet the community lacks a systematic criterion for determining when we can safely stop additional test scenario generation. Existing coverage metrics typically focus on individual component reliability or treat the ADS as a black box, failing to capture certain component interactions that cause most ADS accidents. While traditional mutation testing provides a falsifiable measure of test adequacy, directly porting code- and deep learning model-level mutations to the corresponding modules of ADS is insufficient. In this vision paper, we propose a paradigm shift toward Safety-Aware Mutation Testing (SAMT). Unlike traditional mutation testing, which creates mutants (i.e., faulty versions of the software under test) by injecting artificial faults into individual components, SAMT systematically injects temporally bounded faults into the messages exchanged between ADS modules to simulate realistic interaction failures. To ensure these mutants represent genuine hazards, we propose deriving mutant generation rules directly from top-down safety engineering frameworks, such as System-Theoretic Process Analysis (STPA). By embedding systems thinking into the mutation testing pipeline, SAMT provides a rigorous mechanism for evaluating test adequacy, enabling automated scenario generation, and guiding ADS repair. We also outline critical open challenges.
