Search papers, labs, and topics across Lattice.
This paper introduces RL-Index, a novel framework that reformulates retrieval index reasoning as a reinforcement learning problem, enabling reasoning to occur during the indexing stage rather than at query time. By augmenting documents with LLM-generated rationales that capture the implicit relationships between queries and knowledge, the method significantly enhances retrieval effectiveness while reducing online latency. Experimental results on the BRIGHT benchmark show that RL-Index not only improves retrieval and downstream question-answering performance but also demonstrates robustness across various retrieval systems.
Shifting reasoning to the indexing stage, RL-Index reduces online inference latency while boosting retrieval performance through LLM-generated rationales.
Retrieving external knowledge is essential for solving real-world tasks, yet it remains challenging when the relationship between a query and its relevant knowledge involves implicit and complex reasoning beyond surface-level semantic or lexical matching (e.g., mathematical problems relying on the same theorem or coding requiring deep reasoning). Existing approaches primarily rely on query-side reasoning (e.g., query rewriting), which introduces significant online latency and underutilizes the opportunity to perform reasoning over the knowledge corpus itself (i.e., index-side reasoning). In this paper, we propose RL-Index, an agentic indexing framework that formulates retrieval index reasoning as a reinforcement learning problem. Instead of performing reasoning at query time, RL-Index shifts reasoning to the indexing stage by augmenting documents with LLM-generated rationales that explicitly encode the latent query-knowledge relationship. To optimize the quality of these rationales, we employ Group Relative Policy Optimization (GRPO) and use retrieval similarity as a verifiable reward signal, enabling direct optimization of indexing decisions for retrieval effectiveness. Extensive experiments on the BRIGHT benchmark demonstrate that RL-Index consistently improves both retrieval and downstream question-answering performance, while significantly reducing online inference latency. Moreover, the learned rationale augmentation generalizes across diverse retrievers and generators, highlighting its robustness as a plug-and-play indexing strategy across different retrieval systems.
