Track new results across pure and applied mathematics.
Search papers, labs, and topics across Lattice.
This paper introduces RL-Index, a novel framework that transforms retrieval index reasoning into a reinforcement learning problem, enabling reasoning to occur during the indexing stage rather than at query time. By augmenting documents with LLM-generated rationales that capture the implicit relationships between queries and knowledge, the method significantly enhances retrieval effectiveness and reduces online latency. Experiments on the BRIGHT benchmark show that RL-Index not only improves retrieval and downstream question-answering performance but also generalizes well across various retrieval systems, marking a substantial advancement in retrieval strategies.
Shifting reasoning to the indexing stage can drastically reduce query latency while enhancing retrieval effectiveness through LLM-generated rationales.
Retrieving external knowledge is essential for solving real-world tasks, yet it remains challenging when the relationship between a query and its relevant knowledge involves implicit and complex reasoning beyond surface-level semantic or lexical matching (e.g., mathematical problems relying on the same theorem or coding requiring deep reasoning). Existing approaches primarily rely on query-side reasoning (e.g., query rewriting), which introduces significant online latency and underutilizes the opportunity to perform reasoning over the knowledge corpus itself (i.e., index-side reasoning). In this paper, we propose RL-Index, an agentic indexing framework that formulates retrieval index reasoning as a reinforcement learning problem. Instead of performing reasoning at query time, RL-Index shifts reasoning to the indexing stage by augmenting documents with LLM-generated rationales that explicitly encode the latent query-knowledge relationship. To optimize the quality of these rationales, we employ Group Relative Policy Optimization (GRPO) and use retrieval similarity as a verifiable reward signal, enabling direct optimization of indexing decisions for retrieval effectiveness. Extensive experiments on the BRIGHT benchmark demonstrate that RL-Index consistently improves both retrieval and downstream question-answering performance, while significantly reducing online inference latency. Moreover, the learned rationale augmentation generalizes across diverse retrievers and generators, highlighting its robustness as a plug-and-play indexing strategy across different retrieval systems.