Search papers, labs, and topics across Lattice.
This paper explores the efficiency of long-context language models (LLMs) in passage re-ranking by introducing a novel approach called Whole-Pool Setwise re-ranking. By leveraging the capability of long-context LLMs to process all candidate passages simultaneously, the authors present DualEnd, a method that ranks passages by identifying both the most and least relevant candidates in a single model call. The results demonstrate that DualEnd can effectively rank 100 candidates with only 50 serial LLM calls, significantly improving efficiency compared to traditional methods that require more calls for similar tasks.
Long-context LLMs can drastically reduce the number of model calls needed for passage re-ranking, achieving efficiency without sacrificing effectiveness.
Previous LLM-based passage re-rankers are often expensive and slow because the input context constraints require the LLM to make many dependent model calls. We study how recent long-context LLMs change this problem: when the full set of retrieved candidate passages can be shown to the model at once, ranking no longer has to be reconstructed from many overlapping local comparisons. We propose Whole-Pool Setwise re-ranking, where each call considers all currently unranked candidate passages, and introduce DualEnd, which identifies both the most and least relevant passages in one call. By filling the ranking from both ends, DualEnd ranks 100 candidates with 50 serial LLM calls, compared with 99 calls for comparable one-passage-at-a-time whole-pool methods. Experiments with nine open-weight LLMs on two passage re-ranking benchmarks, measuring effectiveness, call count, token use, runtime, and output reliability shows that long context is not merely more prompt space, but an opportunity to make LLM re-rankers both effective and efficient.
