Search papers, labs, and topics across Lattice.
This study investigates the consistency of factual knowledge retrieval in language models (LMs) by analyzing their parameters as task-specific knowledge bases. Through behavioral and mechanistic analyses, the authors find that facts learned in one task often do not co-occur in others, indicating that LMs encode knowledge in a manner that is heavily dependent on the task context. The results reveal that distinct subsets of parameters are responsible for different tasks, challenging the notion of LMs as universal knowledge bases and highlighting implications for the reliability of their factual knowledge.
Language models encode knowledge in a task-specific manner, leading to inconsistent retrieval of facts across different tasks.
Language models (LMs) capture large amounts of factual knowledge applicable to a wide range of tasks, motivating the view of their parameters as a knowledge base. An important property of knowledge bases is that different queries for the same fact return consistent results, drawing on a single source of truth. We investigate whether LMs satisfy this property through behavioral and mechanistic analyses. Our results suggest that they encode knowledge in a task-specific manner. Behaviorally, facts acquired on one task frequently fail to co-emerge on others during training. Parameter localization experiments suggest a mechanistic explanation, revealing distinct parameter subsets underlying different tasks for the same fact. Finally, we show that chain-of-thought reasoning draws part of its effectiveness from engaging task-specific parameters beyond those tied to the evaluation task. Our findings suggest that what the model knows and how it is asked are intertwined in parameter space, undermining the "knowledge base" analogy and carrying implications for the reliability and controllability of factual knowledge in LMs.
Google Research