Search papers, labs, and topics across Lattice.
This study analyzes a four-year dataset of undergraduate programming submissions to understand how students use natural language comments as specifications for AI-assisted code generation. By introducing a taxonomy that categorizes comments based on type, code expression level, and code construct, the research reveals a notable shift in student commentary from "What" to "How" as they engage with more procedural constructs. The key finding indicates that students prioritize verifying generated code over rewriting comments, highlighting a significant change in the coding process facilitated by AI tools.
Students increasingly focus on verifying AI-generated code rather than refining their specifications, revealing a shift in programming pedagogy.
As AI code tools become integrated into programming environments, students increasingly describe intended behavior in natural language and rely on these tools to generate code, shifting emphasis from code writing to specification. Yet little is known about the comments students write as specifications in AI-assisted programming tasks. We analyze a four-year dataset of undergraduate programming submissions and reflections from tasks in which students wrote comments to guide code generation and refined solutions using test-case feedback. We introduce a taxonomy spanning three dimensions: comment type, code expression level, and code construct. Using automated classification, we examine how these dimensions vary across attempts and how students describe the process in their reflections. Our findings show that students mostly wrote natural-language What comments, shifted toward How comments for more procedural constructs, and focused more on verifying generated code than on repeatedly rewriting comments.
