Search papers, labs, and topics across Lattice.
This paper introduces a model-agnostic framework for optimizing long-term user engagement in recommendation systems by addressing the challenges of sparse and delayed feedback signals. By formulating the downstream reward learning problem and implementing an offline screening method to identify early predictive behaviors, the authors derive several reward signals that can be applied across various models without extensive customization. The results from online A/B testing show significant improvements in user engagement and retention metrics across multiple platforms at Pinterest, highlighting the framework's practical applicability and effectiveness.
A unified, model-agnostic reward framework boosts long-term user engagement metrics significantly across multiple recommendation surfaces.
As recommender systems mature in the past few years, their optimization objectives have evolved from a primary focusing on short-term behavioral signals to a broader emphasis on long-term user engagement and retention. However, directly optimizing retention is difficult because return signals are sparse, delayed, and only partially attributable to earlier recommendations. Prior work has addressed this challenge with sequential modeling and reinforcement learning, but these approaches typically require task specific reward engineering, substantial computational overhead, and surface specific implementations that are difficult to generalize. In this paper, we present a unified, model-agnostic downstream reward framework for optimizing long-term user value in large-scale recommendation systems. First, we formulate the downstream reward learning problem and develop an offline screening framework to identify session level behaviors that are both observable early and predictive of future retention. We then propose several model-agnostic downstream rewards signals derived from observed user action patterns across multiple sources. We further discuss the engineering effort to productionize the proposed rewards derivations and challenges we faced when adding them to our ranking models. Online A/B experiments demonstrate consistent improvements in engagement and retention-related metrics, and the framework has been deployed across multiple Pinterest surfaces, including Homefeed, Related Pins, Search, and Notifications.
