Search papers, labs, and topics across Lattice.
Agentic AutoRAG optimizes RAG pipelines by employing an LLM-agent that reasons about retrieval versus generation failures. This approach attributes errors to specific pipeline stages, enabling targeted configuration adjustments and Pareto frontier tracing for accuracy-cost trade-offs. The method achieves superior LLM-judge accuracy on multi-hop QA benchmarks and significantly reduces query costs while maintaining high performance on a real-world healthcare corpus.
LLM agents can autonomously optimize RAG pipelines by diagnosing retrieval vs. generation failures, outperforming traditional hyperparameter tuning and achieving Pareto efficiency.
Retrieval-augmented generation (RAG) is a widely used approach for grounding large language models (LLMs) in external knowledge. However, configuring a pipeline is an expensive hyperparameter optimization problem over many interacting choices, from chunking and embedding model to reranking and generation. Existing optimizers, from greedy search to Bayesian optimization, reduce each trial to an aggregate score and search without modeling why a configuration performed as it did, even though the retrieved chunks already provide evidence about whether each failure occurred during retrieval or after it. We introduce Agentic AutoRAG, an LLM-agent optimizer for multi-objective RAG hyperparameter optimization with retrieval-versus-generation failure attribution. It proposes configurations scored on a frozen exam from the corpus: after each trial a Diagnoser attributes each failed question to retrieval or generation, and a Proposer, grounded in a knowledge base of model rankings and pricing, selects the next configuration, weighing accuracy against cost to trace a Pareto frontier. On three multi-hop QA benchmarks it reaches higher LLM-judge accuracy than every baseline we compare, and within its first 10 trials it matches or beats the statistical baselines'full 30-trial judge accuracy. In its cost-aware mode on a real-world healthcare corpus it reaches a median exam accuracy of 77%, above the strongest baseline's 71.5%, at about 58% of that baseline's cost per query, and it matches that 71.5% at about 22% of the cost.
ETH