Search papers, labs, and topics across Lattice.
This paper introduces a hybrid dialogue system that combines rule-based scaffolding, informed by self-regulated learning theory, with LLM-generated prompts to support learner reflections in a robotics summer camp. The system aims to leverage the strengths of both approaches: structured guidance from rule-based systems and context-sensitive responsiveness from LLMs. The study found that the hybrid system facilitated richer reflections on goals and activities but also encountered issues with LLM repetitiveness and misalignment, which negatively impacted engagement.
Injecting LLMs into rule-based dialogue systems for learner reflection can boost the depth of insights, but risks disengagement due to repetitiveness and misalignment.
Dialogue systems have long supported learner reflections, with theoretically grounded, rule-based designs offering structured scaffolding but often struggling to respond to shifts in engagement. Large Language Models (LLMs), in contrast, can generate context-sensitive responses but are not informed by decades of research on how learning interactions should be structured, raising questions about their alignment with pedagogical theories. This paper presents a hybrid dialogue system that embeds LLM responsiveness within a theory-aligned, rule-based framework to support learner reflections in a culturally responsive robotics summer camp. The rule-based structure grounds dialogue in self-regulated learning theory, while the LLM decides when and how to prompt deeper reflections, responding to evolving conversation context. We analyze themes across dialogues to explore how our hybrid system shaped learner reflections. Our findings indicate that LLM-embedded dialogues supported richer learner reflections on goals and activities, but also introduced challenges due to repetitiveness and misalignment in prompts, reducing engagement.
CMU ML