Search papers, labs, and topics across Lattice.
This study systematically investigates security vulnerabilities in autonomous coding agents by analyzing 16,112 file changes from 4,022 pull requests using the AIDev dataset. The findings reveal that 38.9% of agent-generated pull requests contain security code smells, with supply chain integrity issues and hard-coded credentials being the most prevalent. Notably, human collaborators are responsible for introducing the majority of critical security leaks, highlighting significant gaps in current review processes and the need for enhanced security measures in AI-assisted development workflows.
Nearly 40% of agent-generated pull requests harbor security vulnerabilities, with human collaborators unintentionally introducing the majority of critical leaks.
The increasing adoption of autonomous coding agents accelerates software development but also introduces scoped security risks within high-impact file paths that can outpace traditional human review capacity. While prior research has primarily evaluated these systems in terms of functional correctness and productivity, this paper presents a large-scale empirical study using the AIDev dataset to systematically characterize security code smells in agent-generated pull requests (PRs). Through a combination of a validated LLM-as-a-judge framework and manual qualitative analysis, we identify and classify security misconfigurations across 16,112 file changes spanning 4,022 pull requests. Our results reveal that 38.9% of agent-generated PRs contain at least one security smell, with supply chain integrity issues accounting for 82.3% of all detected security smells. Furthermore, hard-coded credentials constitute 99.6% of all critical-severity security smells. Crucially, we find that human collaborators are responsible for introducing 67.6% of genuine leaked secrets within these agent-assisted workflows, while existing automated and human review processes fail to detect 81.1% of these credentials prior to integration. These findings highlight substantial security risks in agent-assisted software development workflows and suggest a potential reduction in developer vigilance. They also underscore the urgent need for context-aware security guardrails implemented directly at the point of human-AI collaboration.
