Search papers, labs, and topics across Lattice.
This study investigates the mutation patterns of AI coding agents by analyzing 1,254 performance-relevant changes from 216 pull requests across five agent systems. The findings reveal that less than 1% of the analyzed pull requests target performance improvements, with the most common mutation types being name modification, object creation, and type change. This classification not only highlights the distinct mutation vocabularies of different agents but also suggests that agent identity and performance strategies can significantly inform search-based software engineering (SBSE) practices.
AI coding agents exhibit a surprising focus on name modifications and object creations, reshaping our understanding of their mutation strategies in performance optimization.
AI coding agents are black boxes: we cannot inspect how they generate code, but we can inspect what they change. This distinction matters for search-based software engineering (SBSE), where techniques such as genetic improvement (in the performance-optimisation application we study) depend on mutation operators that reflect how code is actually transformed. Fewer than 1% of the 33,596 agent PRs in AIDev-pop target performance, making each case a rare window into otherwise opaque agent behaviour. We classify 1,254 performance-relevant diff hunks from 216 of these PRs, spanning five agent systems, against the 18-category syntactic mutation taxonomy of Even-Mendoza et al. (2025) using a dual-LLM intersection pipeline. Three categories dominate: name modification (37.0%), object creation (26.4%), and type change (22.7%), a profile markedly different from prior GI corpora where no change accounted for 84%. Each agent's deployed system commits to a distinctive mutation vocabulary, and each performance strategy activates a largely disjoint category subset. Agent identity and target strategy are therefore informative priors that narrow the effective SBSE operator space. Replication package: https://github.com/5uper6rain/ssbse-challenge-2026
