Search papers, labs, and topics across Lattice.
This paper introduces DoPR, a novel framework that enhances the efficiency of large language model (LLM) reranking by utilizing compressed document prefixes to minimize redundant document processing across queries. By precomputing query-independent document representations and reusing them during online reranking, DoPR significantly reduces both memory usage and latency while maintaining high performance metrics. Experimental results on multiple datasets demonstrate that DoPR achieves up to 8.0脳 reduction in online document-side memory and up to 8.04脳 speedup in latency, with minimal loss in ranking quality.
DoPR slashes online reranking costs by reusing compressed document prefixes, achieving remarkable efficiency without sacrificing performance.
Large language models (LLMs) are effective rerankers, but pointwise reranking repeatedly processes the same document across different queries, causing substantial redundant document-side computation. We propose \textbf{DoPR}, a compressed document prefix framework that decouples offline document processing from online reranking. DoPR first selects query-independent document representations and converts them into compressed document prefix states, which are precomputed offline and reused whenever the document is retrieved. During online reranking, the model scores each query-document pair by processing only the query and scoring token, with document information supplied by the stored prefix states. This design reduces online cost through both document-side compression and cross-query prefix-state reuse. Experiments on TREC DL, BEIR, and BRIGHT with Qwen3 models from $0.6$B to $8$B show that DoPR achieves up to 8.0$\times$ online document-side memory reduction and up to 8.04$\times$ latency speedup, while retaining \textbf{97.1\%-99.5\%} of the average NDCG@10 of matched full-document rerankers.
