Search papers, labs, and topics across Lattice.
This paper introduces SHIFT, a novel retrieval training framework that enhances LLM-based retrievers by employing residual projection and task-oriented bidirectional attention aggregation in the latent space. By addressing the mismatch between contrastive learning and implicit reasoning through fine-grained next-token-prediction-based reconstruction, SHIFT significantly improves the efficiency and effectiveness of information retrieval systems. Experimental results demonstrate that SHIFT consistently outperforms existing retrieval methods on reasoning-intensive benchmarks, showcasing its potential for advancing retrieval capabilities.
SHIFT transforms LLMs into reasoning-efficient retrievers, achieving superior performance on complex retrieval tasks by rethinking how we align retrieval objectives with implicit reasoning.
LLM-based retrievers have become a fundamental component of modern information retrieval systems. The paradigm of"rewrite-then-retriev"introduces explicit reasoning before retrieval. In addition, implicit-reasoning retrievers such as GIRCSE and LaSER improve efficiency by replacing explicit reasoning with soft tokens. Although these methods demonstrated competitive performance on reasoning-intensive retrieval benchmarks, they struggle to address the mismatch between the objectives of retrieval and generation. In this work, we propose SHIFT ($\textbf{S}$elf-reconstruction $\textbf{H}$arnesses $\textbf{I}$mplicit $\textbf{F}$ine-grained $\textbf{T}$hinking for Retrieval), a retrieval training framework based on LLMs. Firstly, we transfer LLMs into reasoning-efficient retrievers with residual projection and task-oriented bidirectional attention aggregation in the latent space. Secondly, we alleviate the mismatch between contrastive learning and implicit reasoning using fine-grained next-token-prediction-based reconstruction. Extensive experiments on reasoning-intensive retrieval benchmarks show that SHIFT consistently outperforms other widely used retrievers. We also carried out a detailed analysis to illustrate how our method works.
