Search papers, labs, and topics across Lattice.
This paper addresses the limitations of existing memory systems in LLM-based agents by introducing the concept of state tracking, which ensures that responses reflect the current state of the world rather than outdated information. The authors present StateMemBench, a benchmark designed to evaluate state tracking across 234 multi-session scenarios, revealing that current memory systems struggle with this task. They introduce StateMem, a memory method that effectively tracks evolving states, achieving significant improvements in accuracy over existing baselines while maintaining competitive performance with long-context models.
StateMem improves current-state accuracy by up to 1.8x, revealing that existing memory systems are ill-equipped to handle evolving contexts in LLM interactions.
As LLM-based agents are deployed for longer and higher-stakes tasks, their memory systems continue to have crucial gaps. While existing memory benchmarks focus largely on recall-shaped tasks, we argue an effective memory system must track the evolving state of the world; as facts, constraints, and decisions are revised over a long interaction, answers must reflect the current state and not a superseded one. We define this capability as state tracking and instantiate it in StateMemBench, a benchmark of 234 multi-session scenarios spanning two conversation-length regimes. Its closed-pool grading scores whether an answer reflects the current state, the superseded state, or fails otherwise, separating state-tracking failures from other errors by construction. Our analysis shows that this task is challenging for existing memory systems, retrieval-augmented baselines, and long-context baselines. We then present StateMem, a state-first memory method that explicitly tracks supersession and relational dependencies, and show it improves current-state accuracy over the strongest same-backbone baseline by 1.8x (0.205 ->0.363) on DeepSeek-V4-Flash and over the strongest memory system by 1.6x (0.149 ->0.233) on Qwen-3.5-9B, while remaining competitive with the long-context baselines. Finally, we show the same state approach can be applied as a lightweight single-call wrapper over existing memory systems, lifting current-state accuracy by +32 to +67 points on StateMemBench across six memory and retrieval backends. A length- and cost-matched control attributes +15 to +32 of those points to state structure rather than added context.
