Search papers, labs, and topics across Lattice.
This study investigates how novice programmers interact with code-generating AI models through a pedagogical approach called Prompt Problems, where they write natural-language prompts for computational tasks. Analyzing the experiences of over 900 students, the research identifies common mistakes related to omitting critical details and highlights students' preferences for prompting over traditional coding tasks. Key findings reveal that when faced with incorrect code generation, students prioritize clarifying their intent rather than debugging the generated output, indicating a potential gap in their understanding of the AI's capabilities and limitations.
Novice programmers often overlook crucial details in prompts, leading to a reliance on AI that can hinder their debugging skills and understanding of code generation.
Learning to communicate with code-generating AI models is an emerging skill for novice programmers. One recent pedagogical approach, Prompt Problems, has students solve computational tasks by writing natural-language prompts for code-generating AI models. However, little is known about the specific prompt-level mistakes novice programmers make, the kinds of computational details they fail to communicate, and what strategies they use to recover when generated code is incorrect. In a CS1 course, we studied attempts by more than 900 students to solve dialogue-based Prompt Problems. We analyzed student reflections, unsuccessful prompts, and reported debugging strategies. Compared to traditional coding tasks, students generally found prompting easier, more enjoyable, and better targeted at developing problem-solving skills. The most common mistakes are related to the omission of key details, suggesting both a failure to acknowledge their importance and over-reliance on AI to infer them. When prompts failed, students focused more on clarifying their intent and reflecting on the provided problem details than on tracing generated code or examining test cases.
