Search papers, labs, and topics across Lattice.
This study investigates how students engage with Generative AI (GenAI) code generation in computer science education by injecting realistic bugs into otherwise correct solutions. By analyzing 2,636 sessions from 917 students, the research reveals that students exhibit distinct responses to deliberately injected bugs compared to naturally occurring prompt-related failures, with the former leading to more direct code edits and higher success in subsequent attempts. The findings underscore the importance of fostering a verification mindset and practical debugging skills in students, highlighting how prompt-centered programming activities can be effectively adapted for educational purposes.
Deliberately injecting bugs into GenAI-generated code significantly enhances students' debugging skills and success rates in subsequent attempts.
As Generative AI (GenAI) becomes increasingly central to software development, CS education is integrating prompt-centered workflows where students describe intended program behavior in natural language to elicit code. However, professional practice requires careful review and verification of GenAI-generated code that may appear correct while containing subtle faults. This creates a challenge for CS1-level activities, where current models often solve tasks correctly and reduce students'incentive to closely inspect generated outputs. We investigate how prompt-centered programming activities can be adapted to better foster these practices. Specifically, we explore an approach where realistic, runnable bugs are injected into otherwise correct solutions, thus requiring students to read and repair generated outputs. We analyzed 2,636 sessions from 917 students, and examined behavior across instances of naturally occurring prompt-related failures and deliberately injected bugs within each session. Our findings show that students responded differently across bug sources. Deliberately injected bugs more often led to direct code edits and higher next-attempt success, suggesting localized repair of near-miss solutions. Prompt-related failures instead more often led students to refine prompts by clarifying constraints, updating function signatures, adding edge cases, or reframing the task. Student reflections reinforce the emphasis on review and repair, describing useful practice in code understanding, code review, and debugging, as well as a more careful verification mindset and greater awareness of GenAI limitations. Ultimately, prompt-related failures and injected bugs together support a pedagogically useful GenAI workflow, where students practice both specification refinement through prompts and debugging through code editing.
