Search papers, labs, and topics across Lattice.
This paper investigates how curriculum learning (CL) affects the inductive bias of language models (LMs) regarding typological tendencies in language. They extend prior LM-based exploration by implementing a simple CL variant, where LMs are trained on simpler sentences first. The key finding is that CL significantly alters the apparent inductive bias of LMs, suggesting that the order of training data strongly influences which language structures are learned more easily.
Curriculum learning flips the script on what language structures LMs find "easy," suggesting that training order is a critical factor in shaping their inductive biases.
Many of the thousands of attested languages share common configurations of features, creating a spectrum from typologically very rare (e.g., object-verb-subject word order) or impossible languages to very common combinations of features (e.g., subject-object-verb word order). One central question is under what conditions such typological tendencies can be predicted, and specifically whether the learning bias of language models (LMs) is sufficient to reproduce such patterns. In this study, we add one dimensionality to such analysis -- the learning scenario for LMs -- to explore its interaction with the inductive bias of LMs. Specifically, as a first study, we examine the effect of curriculum learning (CL), as a developmentally motivated learning scenario, i.e., starting with simpler sentences rather than randomly-ordered input. We expand existing LM-based exploration (El-Naggar et al., 2025a,b) with a simple CL variant and find that CL substantially impacts the apparent inductive bias of LMs.
