Search papers, labs, and topics across Lattice.
This study investigates an innovative approach to knowledge retrieval by compiling a knowledge graph into a set of LoRA adapters, allowing for efficient querying without incurring token costs at runtime. The findings reveal that these adapters can encode context-free factual knowledge, significantly outperforming a baseline model in terms of exact-match scores on the MetaQA dataset. However, the research also uncovers a critical limitation: knowledge stored in these adapters is not retrievable through similarity-based methods, indicating a need for alternative mechanisms to select and compose the appropriate adapters for effective querying.
Knowledge graph memory can achieve superior factual recall without the token costs of traditional retrieval methods, but its effectiveness hinges on overcoming the challenge of adapter selection.
Graph retrieval-augmented generation places retrieved subgraphs into the model's context window at query time, paying a recurring token cost and exposing source data on every call. We study an alternative: compiling a knowledge graph offline into a bank of LoRA adapters, one per entity, that serve as a parametric knowledge layer queried by injecting weights rather than text, at zero query-time context cost. On the MetaQA dataset, we find that subgraph-trained adapters encode context-free factual knowledge that generalizes to unseen questions: on single-valued relations the adapter gains $+0.243$ exact-match score over a base model that is nearly blind closed-book ($0.007$), and only the correct adapter recovers this knowledge (an oracle gap of $+0.283$ over the base model). However, the stored knowledge is not recoverable by similarity: given a query with no subgraph, embedding-based and weight-space geometry retrieval both perform at chance, because a semantically neighbouring entity's adapter does not contain the answer - knowledge is stored locally and does not transfer. Weight geometry correlates with subgraph semantics ($\rho = +0.329$) but not with functional retrievability. We quantify the byte and context-token costs against graph retrieval-augmented generation and discuss deployment implications. Our results establish that parametric knowledge graph memory is feasible for storing knowledge, and identify selecting and composing the right adapters by a mechanism other than semantic similarity as the central open problem - motivating a learned, query-conditioned composition mechanism.
