Search papers, labs, and topics across Lattice.
iCoRe addresses the limitations of existing LLM-based bug reproduction test (BRT) generation by introducing an iterative, correlation-aware retrieval approach. It differentiates retrieval strategies for source code and test cases, incorporates function call relationships alongside semantic similarity, and establishes a feedback loop between retrieval and generation. Experiments on SWT-bench Lite and TDD-bench Verified demonstrate a 19.7%-31.7% relative improvement in Fail-to-Pass rate compared to existing retrieval methods when integrated with an LLM-based BRT generator.
Stop feeding your LLM-based bug reproduction tools irrelevant code: iCoRe's correlation-aware retrieval boosts test generation accuracy by up to 31.7%.
Automatically generating bug reproduction tests (BRT) from issue descriptions is crucial for software maintenance. LLM-based approaches have shown great potential for this task. Their effectiveness heavily relies on retrieving high-quality context from the codebase. The retrieval phase of existing approaches relies on either traditional methods like BM25 or LLM-driven strategies. LLM-based retrieval strategies typically equip an LLM with tools to autonomously explore the repository or select the most relevant files and code snippets from a provided list as context. However, these retrieval methods suffer from three key limitations: 1) They often employ a unified strategy for retrieving both source code and test cases, overlooking their distinct retrieval requirements. 2) They focus solely on semantic similarity while ignoring function call relationships, leading to irrelevant context. 3) The retrieval lacks a feedback loop from the generation phase, preventing it from refining the context based on execution results. These limitations collectively result in low-quality context, thereby hindering the accuracy of bug reproduction. To address these challenges, we propose iCoRe, an iterative, correlation-aware context retrieval approach explicitly aware of three key correlations: 1) between source code and test cases, which requires differentiated retrieval, 2) between textual semantics and function call structures for accurate relevance assessment, and 3) between the retrieval and generation phases, which enables iterative feedback and refinement. To evaluate iCoRe, we integrate it with an LLM-based BRT generator and conduct a comprehensive evaluation on the SWT-bench Lite and TDD-bench Verified benchmarks. Experimental results show that our method achieves a Fail-to-Pass rate of 42.0% and 52.8% respectively, representing 19.7%-31.7% relative improvements over existing retrieval methods.
