Search papers, labs, and topics across Lattice.
This paper explores the use of Retrieval Augmented Generation (RAG) to improve LLM performance in software testing and code inspection. By providing LLMs with supplementary knowledge, the RAG pipeline mitigates hallucination and enhances the quality of generated test cases and code reviews. Experiments demonstrate that RAG improves the effectiveness and efficiency of these V&V activities, leading to reduced project costs.
RAG can significantly boost LLM performance in software testing and inspection, making them more reliable and cost-effective.
In this paper, we focus on automating two of the widely used Verification and Validation (V&V) activities in the Software Development Lifecycle (SDLC): Software testing and software inspection (also known as review). Concerning the former, we concentrate on automated test case generation using Large Language Models (LLMs). For the latter, we enable inspection of the source code by LLMs. To address the known LLM hallucination problem, in which LLMs confidently produce incorrect outputs, we implement a Retrieval Augmented Generation (RAG) pipeline to integrate supplementary knowledge sources and provide additional context to the LLM. Our experimental results indicate that incorporating external context via the RAG pipeline has a generally positive impact on both test case generation and code inspection. This novel approach reduces the total project cost by saving human testers'/inspectors'time. It also improves the effectiveness and efficiency of these V&V activities, as evidenced by our experimental study.
