Search papers, labs, and topics across Lattice.
This paper introduces PrefReward, a novel framework that explicitly models user preferences through a structured preference matrix to enhance personalized text generation. By integrating this matrix into the decoding process as a reward signal, PrefReward effectively captures individual stylistic tendencies and addresses long-context dependencies that traditional methods struggle with. Experimental results on the LongLaMP dataset demonstrate that PrefReward significantly improves both the quality of generated content and the interpretability of personalization compared to existing approaches.
PrefReward reveals how an explicit user preference matrix can drastically enhance personalization in text generation, outperforming traditional methods in both quality and interpretability.
Large Language Models (LLMs) have demonstrated remarkable ability in generating personalized content by leveraging user histories and contextual cues. However, most existing personalization approaches rely on implicit representations within model parameters, making it difficult to interpret user-specific preferences or effectively handle long-context dependencies. To address these challenges, we propose PrefReward, a novel preference-aware generative framework that explicitly models user styles through a structured preference matrix and integrates it into the decoding process as a reward signal. PrefReward consists of two stages: (1) extracting a user-specific preference matrix that summarizes individual stylistic tendencies, and (2) using the matrix to guide generation via a KL-divergence-based reward function. Experiments on the LongLaMP dataset show that PrefReward outperforms non-personalized and retrieval-based baselines in both generation quality and personalization interpretability.
NUS