Search papers, labs, and topics across Lattice.
This paper introduces the situated preference reasoning (SiPeR) framework for situated conversational recommendation (SCR), which enhances the understanding of dynamic and implicit user preferences influenced by visual scenes and dialogue. By integrating scene transition estimation and Bayesian inverse inference, SiPeR effectively predicts user needs and preferences, adapting recommendations in real-time. Experimental results on two benchmarks reveal that SiPeR significantly outperforms existing methods in both recommendation accuracy and response generation quality.
SiPeR reveals how integrating scene dynamics with Bayesian inference can dramatically enhance the relevance of conversational recommendations in real-world contexts.
Situated conversational recommendation (SCR), which utilizes visual scenes grounded in specific environments and natural language dialogue to deliver contextually appropriate recommendations, has emerged as a promising research direction due to its close alignment with real-world scenarios. Compared to traditional recommendations, SCR requires a deeper understanding of dynamic and implicit user preferences, as the surrounding scene often influences users' underlying interests, while both may evolve across conversations. This complexity significantly impacts the timing and relevance of recommendations. To address this, we propose situated preference reasoning (SiPeR), a novel framework that integrates two core mechanisms: (1) Scene transition estimation, which estimates whether the current scene satisfies user needs, and guides the user toward a more suitable scene when necessary; and (2) Bayesian inverse inference, which leverages the likelihood of multimodal large language models (MLLMs) to predict user preferences about candidate items within the scene. Extensive experiments on two representative benchmarks demonstrate SiPeR's superiority in both recommendation accuracy and response generation quality. The code and data are available at https://github.com/DongdingLin/SiPeR.
