Search papers, labs, and topics across Lattice.
This paper investigates compute allocation strategies for reasoning-intensive retrieval agents, focusing on query expansion and re-ranking stages within LLM-augmented pipelines. Using the BRIGHT benchmark and Gemini 2.5 models, the authors systematically vary model capacity, inference time, and re-ranking depth. They find that re-ranking benefits significantly from stronger models and deeper candidate pools, while query expansion shows diminishing returns beyond lightweight models, suggesting that compute should be concentrated on re-ranking.
Stop wasting compute on query expansion: focusing it on re-ranking with stronger models and deeper candidate pools yields significantly better retrieval performance in reasoning-intensive tasks.
As agents operate over long horizons, their memory stores grow continuously, making retrieval critical to accessing relevant information. Many agent queries require reasoning-intensive retrieval, where the connection between query and relevant documents is implicit and requires inference to bridge. LLM-augmented pipelines address this through query expansion and candidate re-ranking, but introduce significant inference costs. We study computation allocation in reasoning-intensive retrieval pipelines using the BRIGHT benchmark and Gemini 2.5 model family. We vary model capacity, inference-time thinking, and re-ranking depth across query expansion and re-ranking stages. We find that re-ranking benefits substantially from stronger models (+7.5 NDCG@10) and deeper candidate pools (+21% from $k$=10 to 100), while query expansion shows diminishing returns beyond lightweight models (+1.1 NDCG@10 from weak to strong). Inference-time thinking provides minimal improvement at either stage. These results suggest that compute should be concentrated on re-ranking rather than distributed uniformly across pipeline stages.
