Search papers, labs, and topics across Lattice.
This study evaluates the effectiveness of Cantonese-adapted language models in predicting reading behavior by analyzing eye-tracking data. The researchers compared two pairs of models鈥擟KIP GPT-2 Tiny and its Cantonese-adapted version JED351, as well as Qwen2.5-7B and CantoneseLLM-7B鈥攗sing information-theoretic measures such as lexical surprisal and entropy. Results indicate that CantoneseLLM-7B outperforms other models in predictive fit, highlighting the importance of extensive Cantonese-specific training for better alignment with human reading expectations.
More extensive Cantonese-specific training significantly enhances model predictions of reading behavior, revealing nuanced insights into language processing.
Information-theoretic measures derived from autoregressive language models are widely used to characterize the expectations that shape human reading, but whether language-variety-specific training improves such psycholinguistic alignment remains unclear. This question is still open for Cantonese, where recent NLP evaluations reported mixed benefits from Cantonese-specific training relative to Mandarin-oriented or general-purpose models. Using naturalistic Cantonese eye-tracking data, we compare two within-family adaptation contrasts: CKIP GPT-2 Tiny versus its lightly Cantonese-adapted JED351 derivative, and Qwen2.5-7B versus CantoneseLLM-7B, which underwent substantially more extensive Cantonese continued pretraining and instruction tuning. From each model, we derive lexical surprisal, POS surprisal, entropy before the target, and entropy reduction. Lexical surprisal and the joint four-metric model consistently favor CantoneseLLM-7B, followed by Qwen2.5-7B, CKIP, and JED351, whereas entropy reduction favors CKIP. These results suggest that more extensive Cantonese-specific training can be associated with stronger predictive fit, while model rankings also depend on the information-theoretic measure being evaluated.
