Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces Personalized Agents from Human Feedback (PAHF), a framework that enables AI agents to continually personalize their behavior based on live interaction and explicit per-user memory. PAHF uses a three-step loop involving pre-action clarification, memory-grounded action selection, and post-action feedback integration to adapt to evolving user preferences. Evaluations on embodied manipulation and online shopping benchmarks demonstrate that PAHF, leveraging dual feedback channels and explicit memory, significantly outperforms baselines in learning initial preferences and adapting to preference shifts.
Agents can rapidly adapt to individual and evolving user preferences by explicitly memorizing and acting on both pre- and post-action feedback.
Modern AI agents are powerful but often fail to align with the idiosyncratic, evolving preferences of individual users. Prior approaches typically rely on static datasets, either training implicit preference models on interaction history or encoding user profiles in external memory. However, these approaches struggle with new users and with preferences that change over time. We introduce Personalized Agents from Human Feedback (PAHF), a framework for continual personalization in which agents learn online from live interaction using explicit per-user memory. PAHF operationalizes a three-step loop: (1) seeking pre-action clarification to resolve ambiguity, (2) grounding actions in preferences retrieved from memory, and (3) integrating post-action feedback to update memory when preferences drift. To evaluate this capability, we develop a four-phase protocol and two benchmarks in embodied manipulation and online shopping. These benchmarks quantify an agent's ability to learn initial preferences from scratch and subsequently adapt to persona shifts. Our theoretical analysis and empirical results show that integrating explicit memory with dual feedback channels is critical: PAHF learns substantially faster and consistently outperforms both no-memory and single-channel baselines, reducing initial personalization error and enabling rapid adaptation to preference shifts.