Search papers, labs, and topics across Lattice.
This paper systematically analyzes how large language models (LLMs) can adapt their query formulation strategies for different retrieval systems using reinforcement learning (RL). The study reveals that distinct retrievers require unique optimal query styles, with strategies effective for one retriever often failing for another. Incorporating human guidance and scaling model size further enhances performance, supported by a novel branching-based rollout technique that stabilizes training over multi-retrieval-step trajectories.
LLMs can learn to tailor their queries for different retrievers, revealing that one-size-fits-all strategies are ineffective in retrieval-augmented generation.
Recent retrieval-augmented generation (RAG) approaches have demonstrated strong capability in handling complex queries, yet current research overlooks a critical challenge: different retrievers require fundamentally different query formulation strategies for optimal performance. In this work, we present the first systematic analysis of how LLMs can learn to adapt their query formulation strategies for different retrievers via reinforcement learning (RL). Our empirical study reveals that RL effectively teaches an LLM to tailor its queries to specific retriever characteristics. We discover that different retrievers exhibit surprisingly distinct optimal query styles (e.g., descriptive vs. question-like), suggesting strategies learned for one retriever ineffective for another. We further show that performance can be enhanced by incorporating retriever-specific human guidance and by scaling model size. To facilitate learning over multi-retrieval-step trajectories, we introduce a branching-based rollout technique that improves training stability. Our work provides the first empirical evidence and actionable insights for building truly retriever-aware RAG systems. Code and resources are available at https://github.com/LCO-Embedding/Envs-aware-Information-Retrieval.
