Search papers, labs, and topics across Lattice.
This study explores a novel approach to knowledge retrieval by compiling a knowledge graph into a set of LoRA adapters, which allows for zero-cost context retrieval at query time. The authors demonstrate that these adapters can encode context-free factual knowledge, significantly outperforming a baseline model on the MetaQA dataset, particularly in terms of exact-match scores for single-valued relations. However, the findings reveal a critical limitation: knowledge stored in these adapters does not transfer through similarity-based retrieval methods, highlighting the need for alternative mechanisms to select and compose the appropriate adapters for effective querying.
Compiling knowledge graphs into LoRA adapters allows for zero-cost context retrieval, but knowledge stored is not transferable through similarity, posing a challenge for effective querying.
Graph retrieval-augmented generation places retrieved subgraphs into the model's context window at query time, paying a recurring token cost and exposing source data on every call. We study an alternative: compiling a knowledge graph offline into a bank of LoRA adapters, one per entity, that serve as a parametric knowledge layer queried by injecting weights rather than text, at zero query-time context cost. On the MetaQA dataset, we find that subgraph-trained adapters encode context-free factual knowledge that generalizes to unseen questions: on single-valued relations the adapter gains $+0.243$ exact-match score over a base model that is nearly blind closed-book ($0.007$), and only the correct adapter recovers this knowledge (an oracle gap of $+0.283$ over the base model). However, the stored knowledge is not recoverable by similarity: given a query with no subgraph, embedding-based and weight-space geometry retrieval both perform at chance, because a semantically neighbouring entity's adapter does not contain the answer - knowledge is stored locally and does not transfer. Weight geometry correlates with subgraph semantics ($\rho = +0.329$) but not with functional retrievability. We quantify the byte and context-token costs against graph retrieval-augmented generation and discuss deployment implications. Our results establish that parametric knowledge graph memory is feasible for storing knowledge, and identify selecting and composing the right adapters by a mechanism other than semantic similarity as the central open problem - motivating a learned, query-conditioned composition mechanism.
