Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study rigorously evaluates the capabilities of state-of-the-art large language models (LLMs) in providing corrective feedback and methodological explanations essential for language learners. By systematically varying model parameters and employing both automated metrics and human expert judgments, the research reveals that while LLMs can effectively correct surface-level language errors, they often fall short in delivering the depth of explanation and pedagogical insight necessary for effective language teaching. The findings highlight a significant gap between the current enthusiasm for AI in education and the actual pedagogical competence of these models, raising important questions about their role in language learning environments.
LLMs may excel at correcting grammar, but they struggle to provide the nuanced explanations that effective language teaching demands.
While various organizations now actively encourage LLM use in classrooms, we still lack rigorous, systematic evaluations of how well these models actually perform the fundamental tasks of language pedagogy. This paper examines whether state-of-the-art LLMs can deliver the kind of corrective feedback and methodological explanations that language learners need. The study tests multiple large language models on their ability to identify, correct, and explain common learner mistakes in English, by systematically varying model parameters to investigate how these technical adjustments affect output quality, pedagogical clarity, and consistency, along with using retrieval-augmented generation to query methodological data. The evaluation employs automated metrics (GLEU, BERTScore) but also human expert judgments to capture dimensions that purely computational measures miss: linguistic nuance, cultural sensitivity, and instructional appropriateness. While models demonstrate impressive surface-level correction abilities, their explanations often lack the terminological and domain knowledge that effective language teaching requires, suggesting that current enthusiasm for AI-assisted language learning may be outpacing our understanding of these systems'actual pedagogical competence.