Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates the pedagogical effectiveness of evaluating AI-generated solutions versus directly solving problems in a large upper-division algorithms course with 220 students. The randomized A/B crossover design revealed no significant difference in exam performance or overall course grades between the two conditions, indicating that the method of engagement may be less critical than the inherent difficulty of the tasks. Additionally, student surveys indicated that those who adapted their study habits found the GenAI evaluation activity to be more beneficial, highlighting the potential for thoughtful integration of AI in educational contexts.
Evaluating AI-generated solutions can be as effective as direct problem-solving in enhancing student performance, challenging traditional views on learning methods in computer science education.
Generative AI (GenAI) now pervades computer science education, but its pedagogical value depends on how it is integrated. This study explores whether evaluating AI-generated solutions can be as effective as solving problems directly. In a large upper-division algorithms course, we conducted a twelve-week randomized A/B crossover (N = 220) where students alternated between grading ChatGPT's answers and solving comparable problems themselves. Across exams and overall course grades, performance was statistically indistinguishable between conditions. Homework differences favored whichever cohort encountered the easier half of the syllabus, suggesting task difficulty (as opposed to the evaluation activity) drove those deltas. Surveys showed neutral-to-positive perceptions, with students who reported changing their study habits rating the GenAI-evaluation activity as more helpful. We discuss design choices for GenAI-evaluation tasks that better elicit metacognition without harming achievement.