Search papers, labs, and topics across Lattice.
This study addresses the limitations of gaze-only models in predicting reading comprehension by introducing LEXIC, a lightweight method that incorporates precomputed word-level difficulty signals into eye-tracking data. By enhancing the AhnCNN baseline with two mechanisms鈥擫EXIC-Concat and LEXIC-Res鈥攔esearchers achieved statistically significant improvements in AUROC scores on the OneStop reading comprehension task, particularly for unseen text and readers. The results indicate that while both mechanisms enhance performance, LEXIC-Concat shows a notable advantage in adapting to unseen reader data.
Gaze-only models can be significantly improved by injecting lightweight language signals, achieving up to a 2.9 percentage point gain in reading comprehension prediction accuracy.
On the recent EyeBench benchmark, predicting reading comprehension from eye movements exposes a stark gap: text-aware models using pretrained language models reach 56--63% AUROC, while gaze-only models operate at chance. We ask how far a gaze-only model can be pushed by lightweight, language-model-free conditioning. Building on the EyeBench AhnCNN baseline, LEXIC-Base, we propose two mechanisms to inject three precomputed word-level difficulty signals, GPT-2 surprisal, word frequency, and word length, into the per-fixation input: direct concatenation, LEXIC-Concat, and a residual mechanism, LEXIC-Res, where a small head predicts typical-reader gaze response and the encoder is conditioned on the deviation. On the OneStop reading comprehension task, with K=5 seed-ensemble training across ten folds, both mechanisms produce statistically consistent AUROC gains on Unseen Text, +1.8 to +2.2 percentage points, Wilcoxon p<= 0.065. LEXIC-Concat additionally lifts Unseen Reader by +2.9 percentage points, p = 0.010. We trace an architectural boundary in LEXIC-Res on Unseen Reader, +1.8 percentage points, p = 0.19, to the prediction head being calibrated to training readers, transferring imperfectly to out-of-distribution readers.
