Search papers, labs, and topics across Lattice.
This paper constructs a dataset of 110,000 open-source pull requests to compare the contributions of five popular coding agents (Codex, Claude Code, Copilot, Jules, Devin) across dimensions like merge frequency, file types edited, and developer interaction. The study then analyzes the survival and churn rates of agent-generated versus human-authored code over time. The key finding is that while agent activity is increasing, agent-generated code exhibits higher churn rates compared to human-authored code.
Agent-generated code is more likely to be reworked or removed entirely, suggesting current AI coding tools may increase code churn despite boosting initial contribution rates.
The rise of large language models for code has reshaped software development. Autonomous coding agents, able to create branches, open pull requests, and perform code reviews, now actively contribute to real-world projects. Their growing role offers a unique and timely opportunity to investigate AI-driven contributions and their effects on code quality, team dynamics, and software maintainability. In this work, we construct a novel dataset of approximately $110,000$ open-source pull requests, including associated commits, comments, reviews, issues, and file changes, collectively representing millions of lines of source code. We compare five popular coding agents, including OpenAI Codex, Claude Code, GitHub Copilot, Google Jules, and Devin, examining how their usage differs in various development aspects such as merge frequency, edited file types, and developer interaction signals, including comments and reviews. Furthermore, we emphasize that code authoring and review are only a small part of the larger software engineering process, as the resulting code must also be maintained and updated over time. Hence, we offer several longitudinal estimates of survival and churn rates for agent-generated versus human-authored code. Ultimately, our findings indicate an increasing agent activity in open-source projects, although their contributions are associated with more churn over time compared to human-authored code.
