Search papers, labs, and topics across Lattice.
Tsinghua AIInstitute for Advanced Algorithms ResearchMemTensor Technology Co.SJTUJun 23, 2026arXiv:2606.24775This paper systematically evaluates agent memory systems by decomposing them into four core modules: memory representation and storage, extraction, retrieval and routing, and maintenance. The authors conducted extensive experiments on 12 memory systems across five benchmark workloads, revealing that no single architecture excels universally; instead, performance is contingent on the alignment of memory structure with workload bottlenecks. Key findings include the identification of cost-performance trade-offs, highlighting that localized maintenance is more efficient than global reorganization, thus paving the way for more effective agent-native memory systems.
No single memory architecture is best for all tasks; performance hinges on how well memory structures align with specific workload challenges.
Memory for large language model (LLM) agents has rapidly evolved from simple retrieval-augmented mechanisms into a data management system that supports persistent information storage, retrieval, update, consolidation, and dynamic lifecycle governance throughout agent execution. Despite this evolution, existing evaluations still benchmark agent memory mainly through end-to-end task success metrics (e.g., F1, BLEU), while treating the underlying system as a monolithic black box. As a result, critical system-level concerns, including operational costs, architectural trade-offs across memory modules, and robustness under dynamic knowledge updates, remain insufficiently explored. In this paper, we present a systematic experimental study of agent memory from a data management perspective. We propose an analytical framework that decomposes agent memory into four core modules: memory representation and storage, extraction, retrieval and routing, and maintenance. Under this framework, we evaluate 12 representative memory systems and two reference baselines across five benchmark workloads spanning 11 datasets. Our extensive end-to-end evaluation shows that no single architecture dominates across all scenarios; instead, effectiveness depends heavily on how well the memory structure aligns with the workload bottleneck. Furthermore, through fine-grained ablation studies, we quantify their individual effects on representation fidelity, retrieval precision, update correctness, and long-horizon stability. Finally, we reveal cost-performance trade-offs under realistic workloads, showing localized maintenance is more cost-efficient than global reorganization. Based on these findings, we identify promising directions towards building truly agent-native memory systems. The code is publicly available at https://github.com/OpenDataBox/MemoryData.
