Search papers, labs, and topics across Lattice.
This paper introduces PR2, a reinforcement learning framework for personalized question answering that learns adaptive retrieval-reasoning policies to retrieve relevant personal context. PR2 optimizes multi-turn reasoning trajectories using a personalized reward function, enabling the model to determine when and what evidence to retrieve from user profiles. Experiments on the LaMP-QA benchmark demonstrate that PR2 outperforms existing retrieval-augmented generation methods, achieving significant improvements in personalized QA accuracy across three LLMs.
Key contribution not extracted.
Personalization in Question Answering (QA) requires answers that are both accurate and aligned with users' background, preferences, and historical context. Existing state-of-the-art methods primarily rely on retrieval-augmented generation (RAG) solutions that construct personal context by retrieving relevant items from the user's profile. Existing methods use the user's query directly to retrieve personal documents, and such strategies often lead to surface-level personalization. We propose PR2 (Personalized Retrieval-Augmented Reasoning), a reinforcement learning framework that integrates reasoning and retrieval from personal context for personalization. PR2 learns adaptive retrieval-reasoning policies, determining when to retrieve, what evidence to retrieve from user profiles, and how to incorporate it into intermediate reasoning steps. By optimizing multi-turn reasoning trajectories under a personalized reward function, the framework reinforces reasoning paths that better align with user-specific preferences and contextual signals reflected by the reward model. Extensive experiments on the LaMP-QA benchmark using three LLMs show that PR2 consistently outperforms strong baselines, achieving an average relative improvement of 8.8%-12% in personalized QA.
UW