Search papers, labs, and topics across Lattice.
This study investigates the impact of coding agents on developer productivity and understanding by comparing two AI systems: one that edits user code and another that allows users to write code independently. The findings reveal that while these agents enhance task completion speed, they significantly impair users' code comprehension, hindering their ability to extend and manage their code effectively. Notably, interactions that require minimal effort, such as copy-pasting and accepting edits without review, correlate with poorer understanding, despite users' preference for the convenience of coding agents.
Coding agents may boost productivity, but they risk diminishing developers' understanding and long-term coding skills.
Coding agents (e.g., Cursor) improve developer productivity by optimizing task completion, but shifting users from writing code to prompting and reviewing may harm their understanding, impeding oversight, learning, and communication. To probe this, we have 54 students create a website with one of two AI systems: an agent that edits user code; or a chatbot where users write code alone or adapt generic code snippets. We test understanding via comprehension questions and a task where users extend their code without agents, showing: (1) While agents aid initial task completion, they harm users'code comprehension and thus do not prepare users to extend their code; (2) Low-effort agent interaction types, like copy+paste prompts and auto-accepted edits, are linked with lower comprehension; and (3) Despite self-reported weaker understanding, users still prefer coding agents because they are quick and easy to use. While users stay in the loop for coding workflows, understanding should not be forgotten. Towards this goal, we distill our analyses into future research directions for coding agent developers: dissuading low-effort prompting, creating readable code, and promoting active engagement.
CMU ML