Search papers, labs, and topics across Lattice.
This paper analyzes distribution shifts in language models, decomposing them into observable and unobservable components and highlighting the omitted variable bias arising from the latter. They demonstrate that this bias compromises both evaluation and optimization. To mitigate this, they introduce a framework that maps the strength of omitted variables to bounds on worst-case generalization performance, using these bounds to improve OOD evaluation and optimization.
Language models' brittle behavior under distribution shift isn't just about the data you see; unobservable variables are creating an "omitted variable bias" that compromises both evaluation and optimization.
Despite their impressive performance on a wide variety of tasks, modern language models remain susceptible to distribution shifts, exhibiting brittle behavior when evaluated on data that differs in distribution from their training data. In this paper, we describe how distribution shifts in language models can be separated into observable and unobservable components, and we discuss how established approaches for dealing with distribution shift address only the former. Importantly, we identify that the resulting omitted variable bias from unobserved variables can compromise both evaluation and optimization in language models. To address this challenge, we introduce a framework that maps the strength of the omitted variables to bounds on the worst-case generalization performance of language models under distribution shift. In empirical experiments, we show that using these bounds directly in language model evaluation and optimization provides more principled measures of out-of-distribution performance, improves true out-of-distribution performance relative to standard distribution shift adjustment methods, and further enables inference about the strength of the omitted variables when target distribution labels are available.
