Search papers, labs, and topics across Lattice.
This paper introduces PerMemBench, a new benchmark for evaluating personalized memory systems in LLM-based agents across multi-year, multi-domain interaction histories and diverse user personas. They propose session-level storage gating, a framework that selectively bypasses memory operations for transient sessions, as a method for personalization. Experiments demonstrate that personalization yields significant retention gains with perfect gating, but accurate gating remains a key challenge.
LLM agents waste memory on transient interactions because they lack personalized memory policies, and this benchmark shows just how much better things could be.
Existing large language model (LLM) based memory systems apply universal, static policies that overlook a fundamental reality: the contexts that are worth storing in memory are different across users. This misalignment wastes limited memory budget on transient interactions while failing to preserve critical context for long horizon tasks. To address this gap, we investigate an underexplored question: can LLM based memory systems learn personalized memory policies? We introduce PerMemBench, the first benchmark for evaluating personalized memory systems, featuring multi year, multi domain interaction histories across diverse user personas. We further present the first empirical study of memory personalization, proposing session level storage gating, a lightweight framework that selectively bypasses memory operations for transient sessions. Our study confirms that personalization yields substantial retention gains under perfect gating, yet reveals that accurate gating remains an open and critical challenge.
