Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces FATE, an 8B-parameter language model specifically designed for the automated evaluation of AI tutors in educational contexts. By employing knowledge distillation from a leading LLM, FATE significantly enhances evaluation accuracy across key pedagogical dimensions, achieving performance improvements of up to 22.63 percentage points. The model's effectiveness is validated through benchmarking against popular commercial models, revealing that Gemini 2.5 Flash outperforms its competitors in instructional response quality.
FATE reveals that automated evaluation of AI tutors can achieve unprecedented accuracy, outperforming leading models by significant margins.
The rapid integration of Large Language Models (LLMs) into K-12 and higher education has outpaced the development of reliable methods for evaluating their pedagogical quality. As the research community starts to explore the space of automating evaluation of AI tutors, we introduce FATE (FLC AI Tutor Evaluator), a specialized 8B-parameter language model designed to evaluate AI tutors. Aligned with the four core evaluation tracks from the BEA 2025 Shared Task, our model assesses pedagogical ability across Mistake Identification, Mistake Location, Guidance, and Actionability. Because pedagogical evaluation is a specialized task with limited labeled data, we leverage knowledge distillation from a frontier LLM to generate additional supervision, yielding absolute performance gains up to 22.63 percentage points. Finally, we demonstrate FATE's utility as an automated evaluator by benchmarking instructional responses generated by popular commercial models, including ChatGPT, Claude, Gemini, and DeepSeek. On average, we have found that Gemini 2.5 Flash perfomed best (82.88%), then ChatGPT 5.5 Instant (80.75%), followed by DeepSeek V4 Flash (80.13%) and Claude Sonnet 4.6 (74.00%).