Search papers, labs, and topics across Lattice.
This paper introduces GRADRAG, a novel framework for cross-component prompt adaptation in Retrieval-Augmented Generation (RAG) systems, which coordinates improvements across multiple LLM agents rather than optimizing them in isolation. By modeling the RAG pipeline as a computational graph and utilizing an Evaluator to provide structured feedback, GRADRAG enables iterative updates to upstream components like retrievers and answerers. The framework demonstrates significant performance enhancements on the SQUALITY and QMSUM benchmarks, achieving a 12-15 percentage point improvement in LLM-judged preferences compared to traditional one-step refinement methods.
Coordinated feedback across RAG components leads to a remarkable 12-15 percentage point improvement in performance, challenging the notion that isolated optimization suffices.
Retrieval-Augmented Generation (RAG) systems increasingly employ multiple LLM agents. Yet, most prior work optimizes components in isolation rather than coordinating improvements across the pipeline. We introduce GRADRAG, a framework for cross-component prompt adaptation that models the RAG pipeline as a computational graph and propagates structured evaluation feedback to update upstream agents. An Evaluator critiques downstream answers and supporting evidence, producing actionable feedback that a Prompt Optimizer uses to iteratively update adaptive agents, such as retrievers, graph constructors, and answerers. The Evaluator also triggers early stopping when the output is deemed satisfactory. We evaluate GRADRAG on the SQUALITY and QMSUM benchmarks under two retrieval paradigms: flat chunk-based retrieval using IRCoT-style query refinement (Trivedi et al., 2023), and graph-based retrieval that constructs and iteratively enriches an entity-relation graph from the document. Across both settings, GRADRAG consistently outperforms one-step refinement baselines that update only the final generator, achieving a 12-15 percentage point net preference margin in LLM-judged pairwise comparisons, with most gains realized within two refinement iterations.
