Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Adaptive Re-Ranking, a utility-based labeling framework that optimizes the re-ranking process in information retrieval systems by routing queries based on their complexity. By employing a routing classifier trained on three strategies—sparse retrieval, dense re-ranking, and heavy neural re-ranking—the authors demonstrate significant reductions in latency, achieving 1.15-53x lower median latency compared to traditional methods while maintaining competitive nDCG@10 scores. The results highlight the potential of adaptive query routing to enhance efficiency and reduce computational costs in IR systems without sacrificing performance.
Query-specific routing can slash re-ranking latency by over 50x while keeping retrieval quality competitive.
Modern Information Retrieval (IR) systems typically use a "retrieve-then-rerank" pipeline, where a computationally expensive, pre-determined cross-encoder re-ranks the top results from a fast initial retriever. While effective, this approach often applies heavy re-ranking models regardless of query complexity, resulting in high latency and wasted computational resources on simple queries. We propose Adaptive Re-Ranking, an utility-based labeling framework for cost-aware routing and present empirical evidence (via oracle analysis and a trained baseline router) that per-query routing offers large potential gains but is non-trivial to learn from limited supervision. We train a routing classifier with 3 strategies: sparse retrieval (BM25), dense re-ranking (MiniLM-L6-v2), and heavy neural re-ranking (BGE-v2-m3). Compared to BGE our method achieves 1.15-53x lower median latency and 1.11-5.22x lower mean latency across all datasets we have tested, while delivering -17.5% to +4.0% nDCG@10, which is competitive in some datasets. Our findings show that routing queries based on our novel utility function offers a scalable solution for reducing computational costs and latency in a variety of IR systems.