Search papers, labs, and topics across Lattice.
This paper investigates the effectiveness of various text ranking methods within the context of deep research, where LLM agents iteratively query and reason over retrieved web content. The study systematically analyzes retrieval units (documents vs. passages), pipeline configurations (retrievers, re-rankers, re-ranking depths), and query characteristics (agent-issued vs. training queries) using the BrowseComp-Plus dataset. The key finding is that agent-issued queries favor lexical, learned sparse, and multi-vector retrievers due to their web-search-style syntax, passage-level retrieval is more efficient, and translating agent queries into natural language improves performance.
LLM agents' web search queries favor lexical and sparse retrievers, challenging the assumption that dense retrieval is always superior in deep research.
Deep research has emerged as an important task that aims to address hard queries through extensive open-web exploration. To tackle it, most prior work equips large language model (LLM)-based agents with opaque web search APIs, enabling agents to iteratively issue search queries, retrieve external evidence, and reason over it. Despite search's essential role in deep research, black-box web search APIs hinder systematic analysis of search components, leaving the behaviour of established text ranking methods in deep research largely unclear. To fill this gap, we reproduce a selection of key findings and best practices for IR text ranking methods in the deep research setting. In particular, we examine their effectiveness from three perspectives: (i) retrieval units (documents vs. passages), (ii) pipeline configurations (different retrievers, re-rankers, and re-ranking depths), and (iii) query characteristics (the mismatch between agent-issued queries and the training queries of text rankers). We perform experiments on BrowseComp-Plus, a deep research dataset with a fixed corpus, evaluating 2 open-source agents, 5 retrievers, and 3 re-rankers across diverse setups. We find that agent-issued queries typically follow web-search-style syntax (e.g., quoted exact matches), favouring lexical, learned sparse, and multi-vector retrievers; passage-level units are more efficient under limited context windows, and avoid the difficulties of document length normalisation in lexical retrieval; re-ranking is highly effective; translating agent-issued queries into natural-language questions significantly bridges the query mismatch.
