Search papers, labs, and topics across Lattice.
This paper introduces a multi-agent framework to improve the quality of LLM-generated personalized math problems by iteratively generating, validating, and revising them. Four specialized validator agents assess solvability, realism, readability, and authenticity. Experiments on 600 problems personalized to 20 student interests reveal that authenticity and realism are the most common failure modes, but a single refinement iteration significantly reduces these issues, with different refinement strategies excelling on different criteria.
LLM-generated personalized math problems often fail on realism and authenticity, but can be substantially improved with a multi-agent validation and refinement loop.
Students benefit from math problems contextualized to their interests. Large language models (LLMs) offer promise for efficient personalization at scale. However, LLM-generated personalized problems may often have problems such as unrealistic quantities and contexts, poor readability, limited authenticity with respect to students'experiences, and occasional mathematical inconsistencies. To alleviate these problems, we propose a multi-agent framework that formalizes personalization as an iterative generate--validate--revise process; we use four specialized validator agents targeting the criteria of solvability, realism, readability, and authenticity, respectively. We evaluate our framework on 600 problems drawn from a popular online mathematics homework platform, ASSISTments, personalizing each problem to a fixed set of 20 student interest topics. We compare three refinement strategies that differ in how validation feedback is coordinated into revisions. Results show that authenticity and realism are the most frequent failure modes in initial LLM-personalized problems, but that a single refinement iteration substantially reduces these failures. We further find that different refinement strategies have different strengths on different criteria. We also assess validator reliability via human evaluation. Results show that reliability is highest on realism and lowest on authenticity, highlighting the need for better evaluation protocols that consider teachers'and students'personal characteristics.
