Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study validates and extends the eye movement-based approach to assessing language proficiency, moving from single sentences to contextualized passages in English as a second language. The researchers found that while eye movement proficiency scores are effective, they exhibit biases related to the reader's native language, which can undermine their validity. A novel debiasing method was proposed, resulting in more reliable proficiency scores compared to traditional assessments, thereby enhancing the empirical foundation for future language assessment technologies.
Eye movement-based language proficiency scores are not only more reliable than traditional tests but also reveal significant biases linked to the reader's native language.
Standard language proficiency tests rely on linguistic tasks such as vocabulary, grammar and reading comprehension quizzes. An alternative, cognitively motivated approach, introduced in Berzak et al. (2018), proposed instead to predict language proficiency from behavioral traces of eye movements in reading. In this work, we validate and extend this approach from single sentences to more naturalistic reading of contextualized passages in English as a second language, new proficiency measures, prediction models, and reading in an information seeking regime. We find that the approach is effective in all these evaluations. We further address two key open questions on eye movement based proficiency testing: (1) potential scoring biases that reflect the proximity of the reader's native language to English, which may undermine validity, and (2) its reliability. We find that eye movement based proficiency scores are indeed biased towards L1s that are linguistically closer to English. We propose a score debiasing method which effectively remedies this issue. The reliability analyses suggest that eye movement proficiency scores are more reliable than standard language proficiency scores. Overall, our results strengthen and broaden the empirical foundations for future eye movement based language assessment technologies.