Search papers, labs, and topics across Lattice.
This paper introduces TDD-Bench-Java, a benchmark of 250 Java repository issues for reproduction test generation, addressing a gap in existing benchmarks that primarily focus on Python. They also present e-Otter++ for Java, an adaptation of a Python reproduction test generator, and evaluate it on both the new benchmark and a proprietary dataset. Results demonstrate the feasibility of automated reproduction test generation for Java, potentially improving diagnosis and validation in enterprise software development.
Java developers drowning in unfixed bugs, rejoice: automated reproduction test generation is now a viable option, thanks to a new benchmark and adapted generator.
Given an issue on a software repository, a reproduction test confirms its presence in the code before it gets fixed and its absence after. Reproduction tests provide crucial execution-based feedback for diagnosis and validation during software development. Unfortunately, they are usually missing. Therefore, recent work has introduced both benchmarks and a thriving literature on solutions for reproduction test generation from issues. However, that work has focused on Python and neglected other languages such as Java, which is important for enterprise software. This paper introduces both a benchmark and a solution for Java repository-level reproduction test generation. The benchmark, TDD-Bench-Java, is the first to model this problem and comprises 250 instances sourced from popular open-source repositories. The solution, e-Otter++ for Java, adapts a state-of-the-art reproduction test generator for Python to yield high performance on Java. To evaluate in an industry setting, besides empirical results with TDD-Bench-Java, this paper also presents results with a contamination-free proprietary dataset. Overall, we hope that this paper contributes to bringing better diagnosis and validation to Java software development.
