Search papers, labs, and topics across Lattice.
The paper introduces CoSearch, a reinforcement learning framework that jointly trains a multi-step reasoning agent and a generative document ranking model for agentic search. CoSearch uses Group Relative Policy Optimization (GRPO) and a semantic grouping strategy to cluster sub-queries, enabling effective ranker training without extra rollouts. Experiments on QA benchmarks demonstrate that CoSearch outperforms existing methods, highlighting the importance of joint training for reasoning and retrieval.
Closing the gap between fixed retrieval systems and oracle performance in agentic search yields a +26.8% F1 improvement, demonstrating that jointly training the reasoning agent and retrieval system is a game-changer.
Agentic search -- the task of training agents that iteratively reason, issue queries, and synthesize retrieved information to answer complex questions -- has achieved remarkable progress through reinforcement learning (RL). However, existing approaches such as Search-R1, treat the retrieval system as a fixed tool, optimizing only the reasoning agent while the retrieval component remains unchanged. A preliminary experiment reveals that the gap between an oracle and a fixed retrieval system reaches up to +26.8% relative F1 improvement across seven QA benchmarks, suggesting that the retrieval system is a key bottleneck in scaling agentic search performance. Motivated by this finding, we propose CoSearch, a framework that jointly trains a multi-step reasoning agent and a generative document ranking model via Group Relative Policy Optimization (GRPO). To enable effective GRPO training for the ranker -- whose inputs vary across reasoning trajectories -- we introduce a semantic grouping strategy that clusters sub-queries by token-level similarity, forming valid optimization groups without additional rollouts. We further design a composite reward combining ranking quality signals with trajectory-level outcome feedback, providing the ranker with both immediate and long-term learning signals. Experiments on seven single-hop and multi-hop QA benchmarks demonstrate consistent improvements over strong baselines, with ablation studies validating each design choice. Our results show that joint training of the reasoning agent and retrieval system is both feasible and strongly performant, pointing to a key ingredient for future search agents.
