Search papers, labs, and topics across Lattice.
This paper addresses the limitations of traditional fine-tuning methods in conversational information retrieval, which often lead to catastrophic forgetting and require costly retraining. By employing model merging techniques, specifically Model Soup and Slerp, the authors create a unified retrieval model that effectively operates in both ad-hoc and conversational contexts without the need for additional fine-tuning. The results show a significant improvement in ad-hoc search performance, achieving up to 15% higher NDCG@3 in zero-shot scenarios, highlighting the effectiveness of this training-free approach.
Merging models can boost ad-hoc search performance in conversational retrieval by up to 15% without any retraining costs.
Conversational information retrieval is challenging since it requires the consideration of the conversation history which potentially gives rise to topic shifts and coreference resolution across previous turns. To address these challenges, previous work mainly rely on traditional fine-tuning of ad-hoc retrievers on conversational datasets or extrapolates their generalizability through multi-tasking. However, this mainstream approach is costly - since it requires model re-training - and exhibits catastrophic forgetting, where the model loses its foundational ad-hoc retrieval performance. In this paper, we fill this gap by introducing model merging as a training-free strategy enabling the design of a single retrieval model that operates across both ad-hoc and conversational settings with no additional fine-tuning. We conduct experiments using linear and non-linear parameter-wise merging strategies - namely Model Soup and Slerp - on standard ad-hoc search and conversational retrieval datasets. Our results demonstrate that model merging significantly enhances the ad-hoc search capabilities of conversational retrievers while improving generalizability across task-specific datasets, achieving up to 15% higher NDCG@3 under zero-shot conditions.
