Search papers, labs, and topics across Lattice.
This paper conducts a comprehensive evaluation of various memory substrates for memory-augmented agents, assessing their performance across multiple models and benchmark tasks. The findings reveal that no single memory substrate excels universally; while broad retrieval enhances long-context factual question answering, it can detrimentally affect sequential decision-making by diverting attention from critical actions. The study highlights the importance of substrate routing to optimize memory usage based on specific operational contexts, offering valuable insights for the design of adaptive memory systems in LLM agents.
No memory substrate is a one-size-fits-all solution; the right choice depends on the task, with broad retrieval boosting QA but hindering decision-making.
Memory is becoming core infrastructure for long-horizon LLM agents, yet existing evaluations offer limited guidance on which memory substrate, namely the underlying medium in which memory is represented and stored, should be used under different operating regimes. We present a controlled harness evaluation of memory substrates for memory-augmented agents, covering dense and sparse indices, text records, structural stores, hierarchical stores, refinement-based memories, parametric updates, and activation-compatible context mechanisms. Across three backbone models and four benchmark suites spanning user-centric question answering and agent-centric decision-making, we instrument 26 performance and efficiency metrics under a unified harness. Our results show that no single substrate consistently dominates: broad retrieval benefits long-context factual QA, while excessive retrieval can harm sequential decision-making by shifting attention away from action-critical context. Scalability introduces a further routing axis, as substrates that perform well at moderate history lengths can become costly or brittle at longer horizons. These findings motivate substrate routing as a necessary component of adaptive agent memory systems and provide empirical guidance for designing efficient, reliable, and regime-aware long-term memory for LLM agents. Code will be made available upon acceptance.
