Search papers, labs, and topics across Lattice.
This paper evaluates the out-of-distribution (OOD) performance of foundation models (FMs) on tasks related to teaching and learning in schoolchildren, finding that inter-model behaviors correlate more strongly than with expert human behaviors. The study reveals a misalignment between LLM performance and downstream measures of teaching quality, often negatively impacting learning outcomes. Furthermore, ensembling techniques exacerbate this misalignment, and a significant portion of the misalignment error is shared across models, suggesting a common pretraining influence.
Ensembling LLMs for educational tasks can backfire, worsening misalignment with actual learning outcomes despite improved benchmark performance.
LLMs increasingly excel on AI benchmarks, but doing so does not guarantee validity for downstream tasks. This study evaluates the performance of leading foundation models (FMs, i.e., generative pre-trained base LLMs) with out-of-distribution (OOD) tasks of the teaching and learning of schoolchildren. Across all FMs, inter-model behaviors on disparate tasks correlate higher than they do with expert human behaviors on target tasks. These biases shared across LLMs are poorly aligned with downstream measures of teaching quality and often \textit{negatively aligned with learning outcomes}. Further, we find multi-model ensembles, both unanimous model voting and expert-weighting by benchmark performance, further exacerbate misalignment with learning. We measure that 50\% of the variation in misalignment error is shared across foundation models, suggesting that common pretraining accounts for much of the misalignment in these tasks. We demonstrate methods for robustly measuring alignment of complex tasks and provide unique insights into both educational applications of foundation models and to understanding limitations of models.
