Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to document selection for deep research agents by employing search-oriented rubrics that define the criteria for high-quality document sets. By training a document reranker, RubricRanker, through a two-stage framework involving supervised fine-tuning and reinforcement learning, the authors address the limitations of traditional relevance matching in retrieval systems. The results show that RubricRanker significantly outperforms existing methods, achieving a 2.6-point improvement on four deep research benchmarks while also demonstrating strong generalization across five RAG benchmarks.
Search-oriented rubrics redefine how we evaluate document sets, leading to a 2.6-point performance boost in deep research tasks.
Retrieval systems help deep research agents generate high-quality answers by providing relevant documents. However, existing retrievers typically select documents through relevance matching, while individually well-matched top-$k$ documents may not form a \textit{set} that satisfies the complex information needs of an agent query (\eg, diverse, concise and authoritative documents). In this paper, we propose search-oriented rubrics that \textit{explicitly} define the requirements that high-quality document sets should satisfy for each agent query. Our search rubrics are organized into a hierarchical structure and synthesized using a powerful LLM. Based on these search rubrics, we further train a document reranker \textbf{RubricRanker} to select a high-quality subset from retrieved documents. We design a two-stage training framework that consists of rubrics-guided supervised fine-tuning and rubric-based reinforcement learning. Extensive experiments demonstrate that RubricRanker outperforms the strongest baseline by 2.6 points on four deep research benchmarks and generalizes well to five RAG benchmarks.
Mila