Search papers, labs, and topics across Lattice.
This paper introduces Memory as a Controlled Process (MemCon), a novel framework that optimizes memory management in Large Language Model (LLM) agents by modeling memory operations as a Markov Decision Process. By learning an adaptive policy that determines when and how to retrieve information, inject plans, and consolidate memory, MemCon addresses the limitations of static memory access methods that hinder agentic learning. The framework demonstrates significant improvements across various benchmarks, achieving up to 15.2 points higher task success rates and reducing token consumption by 5-20% compared to existing memory management approaches.
MemCon reveals that adaptive memory management can boost task success rates by over 15 points while cutting token usage significantly.
Large Language Model (LLM) agents increasingly rely on external memory systems to accumulate experience across tasks. Yet nearly all existing approaches, from graph-structured memories to reflective insight stores, access memory through fixed, hand-designed heuristics. We argue that this static view of memory is a core bottleneck for agentic learning because optimal memory behavior is fundamentally context-dependent. The early stages of the tasks, benefit from minimal retrieval because memory is sparse; recurring goal types benefit from plan reuse rather than generic nearest-neighbor lookup; stuck agents benefit from re-retrieval with alternative queries; and across long task streams, the memory store itself must be consolidated and pruned to remain useful. We present Memory as a Controlled Process (MemCon), a framework that models memory operations as a Markov Decision Process and learns an online policy that adaptively decides when, what, and how much to retrieve, when to inject a distilled plan, and when to consolidate or forget. MemCon is backend-agnostic: it wraps any existing memory implementation, learns from task-by-task binary feedback with no pretraining and no additional LLM calls, and uses a lightweight tabular contextual bandit with UCB exploration that converges within tens of tasks. Across 6 benchmarks, 3 agent frameworks, and 3 LLM backbones, MemCon consistently outperforms multiple memory baselines by up to 15.2 points in task success while reducing token consumption by 5--20%.
UW