Search papers, labs, and topics across Lattice.
This study develops a self-explanation tutor, ESSE, that leverages a large language model (LLM) to provide immediate feedback on students' explanations of worked examples in an introductory programming course. By comparing the LLM's assessments to those of human experts and non-expert raters, the research identifies the model's reliability and systematic biases in evaluating student responses. The findings reveal that LLM-based feedback not only enhances the quality of student explanations but also encourages persistence and deeper learning, demonstrating the effectiveness of LLMs in educational contexts.
LLM feedback transforms passive learning into an active, engaging process, leading to richer student explanations and improved learning outcomes in programming.
Worked examples are a important part of introductory programming, but reading their expert explanations is passive. Self explanation, students explaining the problem and its solution to themselves with subgoal level analysis, turns that study into an active task, yet it is hard to scale because assessing free-text explanations and returning timely feedback has had no easy automated solution. We investigate whether a large language model (LLM) can fill that gap. We build a self-explanation tutor for introductory programming, ESSE, in which students explain lines of worked examples and receive immediate LLM feedback on the correctness and completeness of each explanation, and we pursue two goals. First, we ask whether the LLM judges student explanations well enough to serve as the engine of the tutor; we assess its judgments against two independent human reference standards of different kinds, a single domain expert and a crowd of non-expert raters, each with its own strengths and weaknesses, characterizing both where the LLM is reliable and the systematic tendencies in how it diverges. Second, we ask whether the LLM-based tutoring benefits students; deploying it in an introductory Java course, we find that its feedback leads students to persist and revise rather than abandon a line, that their explanations grow more complete and conceptually richer across attempts, and that students show evidence of learning. These indicate that LLM-based assessment is good enough to power a self-explanation tutor, and that the tutor positively shapes how students study worked examples.
