Search papers, labs, and topics across Lattice.
We introduce Sherpa, a multi-turn reinforcement learning framework that trains LLM teachers to adapt their instruction by directly optimizing for the learning outcomes of distinct simulated student archetypes. This approach moves beyond static pedagogical signals by grounding teaching effectiveness in student progress, leading to significant performance gains. Sherpa-trained teachers improve instructed student performance by an average of 20.5 percentage points and achieve a 79.2% pedagogy score on MathTutorBench, outperforming the base model in human evaluations.
LLM tutors trained to adapt to individual student learning preferences achieve a 20.5pp performance boost and significantly outperform base models in human preference studies.
Large language models (LLMs) have become increasingly capable problem solvers, but being able to solve a problem is not the same as being able to teach it. Existing approaches to training LLMs as teachers rely on demonstrations, preference data, or predefined pedagogical criteria that specify what good teaching looks like. However, these signals are often not grounded in individual student learning outcomes, where effective teaching strategies can vary substantially across learners. To address this, we introduce Sherpa, a multi-turn reinforcement learning framework that instantiates multiple student archetypes with LLMs conditioned on distinct learning preferences and trains a teacher model to adapt its instruction by directly maximizing their learning outcomes. Teacher LLMs trained with Sherpa improve instructed students'performance across all archetypes by an average of 20.5 percentage points. Under MathTutorBench's evaluation, Sherpa raises the overall pedagogy score from 52.5% to 79.2%, indicating better teaching responses. Our human studies show that the trained teacher is preferred over the base model in 79.6% of pairwise comparisons. Together, Sherpa trains LLM teachers to adapt to diverse simulated students and become better aligned with human teachers, paving the road towards AI tutors teaching real students.
