Search papers, labs, and topics across Lattice.
This paper investigates how retrieved context in RAG systems shapes the internal representations of LLMs, going beyond analysis of output behavior. By analyzing hidden states across layers in controlled single- and multi-document retrieval settings on four QA datasets and three LLMs, the authors quantify the impact of context relevancy on internal representations. They then correlate these representation shifts with downstream generation performance, offering insights into information integration within RAG.
Forget tweaking prompts – understanding how retrieved context warps an LLM's hidden states is the key to unlocking better RAG performance.
Retrieval-augmented generation (RAG) enhances large language models (LLMs) by conditioning generation on retrieved external documents, but the effect of retrieved context is often non-trivial. In realistic retrieval settings, the retrieved document set often contains a mixture of documents that vary in relevance and usefulness. While prior work has largely examined these phenomena through output behavior, little is known about how retrieved context shapes the internal representations that mediate information integration in RAG. In this work, we study RAG through the lens of latent representations. We systematically analyze how different types of retrieved documents affect the hidden states of LLMs, and how these internal representation shifts relate to downstream generation behavior. Across four question-answering datasets and three LLMs, we analyze internal representations under controlled single- and multi-document settings. Our results reveal how context relevancy and layer-wise processing influence internal representations, providing explanations on LLMs output behaviors and insights for RAG system design.
