Search papers, labs, and topics across Lattice.
This paper introduces intent-based mutation testing, a novel approach that generates program mutations by altering the programming intents rather than the syntax of the code itself. By leveraging Large Language Models (LLMs) to create these intent mutations, the method captures a broader range of faults and corner cases that traditional mutation testing might miss, as 55% of the generated mutations were found to be unique compared to conventional methods. The results indicate that intent-based mutation testing can significantly enhance the robustness of software testing by providing semantically diverse and complex mutations that align closely with original program specifications.
Intent-based mutation testing reveals that over half of the generated mutations are unique, highlighting a new frontier in fault detection that traditional methods overlook.
This paper presents intent-based mutation testing, a testing approach that generates mutations by changing the programming intents that are implemented in the programs under test. In contrast to traditional mutation testing, which changes (mutates) the way programs are written, intent mutation changes (mutates) the behavior of the programs by producing mutations that implement (slightly) different intents than those implemented in the original program. The mutations of the programming intents represent possible corner cases and misunderstandings of the program behavior, i.e., program specifications, and thus can capture different classes of faults than traditional (syntax-based) mutation. Moreover, since programming intents can be implemented in different ways, intent-based mutation testing can generate diverse and complex mutations that are close to the original programming intents (specifications) and thus direct testing towards the intent variants of the program behavior/specifications. We implement intent-based mutation testing using Large Language Models (LLMs) that mutate programming intents and transform them into mutants. We evaluate intent-based mutation on 29 programs and show that it generates mutations that are syntactically complex, semantically diverse, and quite different (semantically) from the traditional ones. We also show that 55% of the intent-based mutations are not subsumed by traditional mutations. Overall, our analysis shows that intent-based mutation testing can be a powerful complement to traditional (syntax-based) mutation testing.
