Search papers, labs, and topics across Lattice.
This paper introduces GateMem, a benchmark designed to evaluate memory governance in multi-principal shared-memory agents, addressing the inadequacies of existing benchmarks that focus on single-user scenarios. The study reveals that current methods struggle to balance utility, access control, and reliable forgetting, with long-context prompting providing the best governance scores at a high token cost. Ultimately, the findings highlight significant gaps in the performance of memory agents, indicating they are not yet suitable for reliable deployment in shared environments such as hospitals and workplaces.
Current memory agents fail to provide reliable governance in shared settings, with no method achieving a balance between utility, access control, and forgetting.
Memory benchmarks for LLM agents largely assume single-user settings, leaving shared assistants for hospitals, workplaces, campuses, and households understudied. In these deployments, multiple principals write to a common memory pool and query it under different roles, scopes, and relationships, so memory quality requires governance as well as recall. We introduce GateMem, a benchmark for multi-principal shared-memory agents. GateMem jointly evaluates utility for legitimate long-horizon requests with state updates, access control across contextual authorization boundaries, and agent-facing active forgetting after explicit deletion requests. It spans medical, office, education, and household domains, with long-form multi-party episodes, incremental memory injection, hidden checkpoints, structured judging, and leak-target annotations. Across diverse baselines and backbone models, no method simultaneously achieves strong utility, robust access control, and reliable forgetting. Long-context prompting often yields the best governance score at high token cost, while retrieval-based and external-memory methods reduce cost yet still leak unauthorized or deleted information. These results show current memory agents remain far from reliable shared institutional deployment.