Search papers, labs, and topics across Lattice.
This paper introduces a novel method for enhancing conversational recommender systems (CRS) by quantifying the effectiveness of multi-turn interactions through entropy reduction, which measures the assistant's uncertainty over recommendations. By using this entropy reduction as a reward for fine-tuning large language models (LLMs), the authors eliminate the need for ground-truth recommendations, which are often scarce in real-world applications. Empirical evaluations on the INSPIRED and ReDial datasets demonstrate significant improvements in both the quality of recommendations and the efficiency of conversations compared to existing methods.
Reducing uncertainty in LLM interactions can dramatically enhance recommendation quality without needing ground-truth data.
Recent advances in large language models (LLMs) have enabled their use as conversational recommender systems (CRS), demonstrating strong recommendation accuracy and natural dialogue. However, guiding multi-turn interactions to elicit user preferences effectively remains challenging. Existing approaches either use separate reinforcement learning agents with templated interactions or optimize for interactivity judged by another LLM, without measuring how much useful information is actually gained. We propose a new approach that quantifies the effectiveness of each interaction by the reduction in the assistant's uncertainty, measured via entropy over recommendations. We apply this entropy reduction as a reward---without relying on ground-truth recommendations, which are often unavailable in real-world scenarios---to fine-tune the LLM, enabling strategic interaction generation. Empirical results with supervised fine-tuning (SFT) and direct preference optimization (DPO) on the INSPIRED and ReDial datasets show that our method improves both recommendation quality and conversational efficiency.
