Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To evaluate whether coding agents maintain software quality or accumulate technical debt across multi-turn workflows, the authors built RepoReuse, a multi-turn benchmark tracking cross-turn structural redundancy and code reuse in real repositories. This addresses a major evaluation blind spot, as standard functional correctness metrics mask agent-generated code duplication and context abandonment. Across an audit of over 3,000 turns, agents introduced duplicated logic in 50.8% of task chains by turn 5 and progressively stopped exploring relevant repository context, even while standard pass rates remained virtually unchanged.
Standard pass rates hide severe technical debt: coding agents duplicate logic in over half of multi-turn task chains by turn 5 while progressively abandoning repository exploration.
Coding agents are increasingly deployed for iterative development on real repositories, yet existing evaluation barely answers a basic question: \emph{do coding agents reuse existing code or reinvent the wheel?} The question matters: every duplicated implementation is a fix applied twice and agents produce code far faster than humans can audit, so redundancy accumulates unsupervised. Thus, we present \textbf{RepoReuse}, a multi-turn benchmark for auditing code reuse in real repositories, where requirements are revealed turn by turn and the workspace accumulates across turns. It is built by a fully automated pipeline combining AST-based dependency graphs, guided evidence collection, and execution-verified task synthesis, and scales readily to new repositories. Beyond pass rates, we measure the reuse rate together with recall and cross-turn structural redundancy. An audit over 3{,}000 turns shows that agents progressively stop exploring relevant repository code, reuse their own history less even when it is fully in the workspace, and leave duplicated logic in 50.8\% of task chains by turn~5---all while pass rates barely move. Such deficiencies are invisible to pass rates, underscoring the need to evaluate code generation beyond functional correctness.
Tsinghua AI