Search papers, labs, and topics across Lattice.
This study explores whether language models can generalize human-like preferences for noun phrase modifier orders despite being trained on data devoid of direct evidence for such structures. The results reveal that models consistently favor scope-homomorphic orders when evaluated on multiple modifier sentences, indicating a capacity for linguistic generalization akin to human learners. Notably, the strength of these preferences varies by modifier type, suggesting nuanced underlying mechanisms at play.
Language models can exhibit human-like preferences for modifier order even when trained on data that lacks direct evidence for such structures.
A central question in language acquisition is whether linguistic biases can emerge from general learning mechanisms operating over underdetermined input. Artificial Language Learning (ALL) studies have shown that human learners reliably generalize beyond the evidence provided, including by preferring scope-homomorphic noun phrase modifier orders. In this work, we investigate whether language models exhibit the same bias under similar conditions. We create a controlled learning environment in which models are trained on a corpus where all noun phrases containing multiple modifiers have been removed, eliminating direct evidence about modifier ordering, and are then evaluated on multiple modifier sentences. Across three model sizes, we find that they consistently prefer scope-homomorphic orders despite never observing them during training. These preferences vary in strength by modifier type. To investigate the source of these preferences, we examine noun-modifier association strength using pointwise mutual information (PMI). While PMI reflects known modifier-ordering patterns, it does not explain the models'ordering preferences. These findings demonstrate that LMs can recover human-like linguistic generalizations from impoverished input and provide a controlled framework for investigating the mechanisms underlying such biases.
