Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
APITestGenie automates API integration test generation by combining LLMs, RAG, and prompt engineering, using business requirements and OpenAPI specifications as input. Evaluated on 10 real-world APIs, the tool achieved an 89% success rate in generating valid test scripts, even uncovering previously unknown API defects. API complexity and the level of detail in requirements and documentation were identified as key factors influencing test generation success.
Automating API test generation with LLMs not only slashes manual effort, but also uncovers hidden integration bugs that manual testing misses.
Modern software systems rely heavily on Web APIs, yet creating meaningful and executable test scripts remains a largely manual, time-consuming, and error-prone task. In this paper, we present APITestGenie, a novel tool that leverages Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), and prompt engineering to automatically generate API integration tests directly from business requirements and OpenAPI specifications. We evaluated APITestGenie on 10 real-world APIs, including 8 APIs comprising circa 1,000 live endpoints from an industrial partner in the automotive domain. The tool was able to generate syntactically and semantically valid test scripts for 89\% of the business requirements under test after at most three attempts. Notably, some generated tests revealed previously unknown defects in the APIs, including integration issues between endpoints. Statistical analysis identified API complexity and level of detail in business requirements as primary factors influencing success rates, with the level of detail in API documentation also affecting outcomes. Feedback from industry practitioners confirmed strong interest in adoption, substantially reducing the manual effort in writing acceptance tests, and improving the alignment between tests and business requirements.