Search papers, labs, and topics across Lattice.
This study investigates whether language models can learn to distinguish between principled and statistical properties from language data, a distinction that humans intuitively grasp. The research reveals that while earlier models are sensitive to statistical prevalence, they fail to accurately represent the principled-vs-statistical distinction. Notably, GPT-4 demonstrates the capability to successfully learn this distinction, suggesting that advanced language models can bootstrap core conceptual understandings from linguistic input.
GPT-4 is the first language model shown to successfully learn the critical distinction between principled and statistical truths from language data.
Generic statements like "tigers are striped" and "cars have radios" communicate information that is, in general, true. However, while the first statement is true in principle, the second is true only statistically. People are exquisitely sensitive to this principled-vs-statistical distinction. It has been argued that this ability to distinguish between something being true by virtue of it being a category member versus being true because of mere statistical regularity, is a general property of people's conceptual machinery and cannot itself be learned. We investigate whether the distinction between principled and statistical properties can be learned from language itself. If so, it raises the possibility that language experience can bootstrap core conceptual distinctions and that it is possible to learn sophisticated causal models directly from language. We find that language models are all sensitive to statistical prevalence, but struggle with representing the principled-vs-statistical distinction controlling for prevalence. Until GPT-4, which succeeds.
