Search papers, labs, and topics across Lattice.
This study investigates the impact of guided versus unrestricted access to large language models (LLMs) on student learning outcomes in an undergraduate Probability and Statistics course. By organizing students into three conditions鈥攏o access, unrestricted access, and guided access with explicit training on reasoning-focused help-seeking鈥攖he research reveals that guided LLM use fosters more effective learning interactions and enhances independent performance on assessments. Key findings indicate that students in the guided condition prioritized reasoning and stepwise support, leading to better quiz results compared to those with unrestricted access, who primarily used LLMs for task completion rather than deep learning engagement.
Guided LLM access not only improves reasoning skills but also leads to significantly better independent learning outcomes compared to unrestricted use.
Large language models (LLMs) are increasingly entering students' learning practices, but their educational value depends on whether they support reasoning or enable task completion without engagement. This study examines guided LLM use in an undergraduate Probability and Statistics course, focusing on the gap between assigned access and actual interaction quality. In a four-week quasi-experimental summer program, students were organized into three balanced conditions: no LLM access, unrestricted LLM access, and guided LLM access. The guided condition used the same LLM platform as the unrestricted condition, but students received explicit training and rules promoting reasoning-focused help-seeking, stepwise hints, verification, and ethical use. All quizzes and the delayed final exam were completed without LLM or external assistance, allowing us to distinguish AI-supported practice performance from independent learning. Results show that guided use was associated with clearer learning-oriented interaction patterns than unrestricted access, especially in prioritizing reasoning over final answers and requesting stepwise support. Guided-LLM students showed stronger no-help quiz performance during the intervention phase, whereas unrestricted access appeared more useful for assisted practice completion than for consistently improving independent performance. Available time measures did not support a simple duration-based explanation, and self-assessment calibration suggested better alignment between perceived and demonstrated understanding in the Guided-LLM condition. Overall, LLM access alone appears to be an incomplete educational intervention. For Artificial Intelligence in Education (AIED), the central design challenge is to scaffold how students use LLMs so that these systems function as partners in reasoning rather than answer-getting tools.
