Search papers, labs, and topics across Lattice.
This paper introduces MemSyco-Bench, a novel benchmarking framework designed to evaluate the impact of memory-induced sycophancy in LLM-based agents. By addressing the critical issue of how retrieved memories can lead to over-alignment with user preferences at the expense of factual accuracy, the benchmark encompasses five specific tasks that assess agents' abilities to navigate memory conflicts and maintain objective reasoning. The findings reveal significant gaps in existing memory benchmarks, highlighting the need for a more nuanced evaluation of memory's role in decision-making processes.
Agents often prioritize user alignment over factual accuracy, and MemSyco-Bench reveals just how much memory influences this dangerous trend.
Memory has emerged as a cornerstone of modern LLM-based agents, supporting their evolution from single-turn assistants to long-term collaborators. However, memory is not always beneficial: retrieved memories often induce a critical issue of sycophancy, causing agents to over-align with the user at the cost of factual accuracy or objective reasoning. Despite this emerging risk, existing memory benchmarks primarily evaluate whether memories are correctly stored, retrieved, or updated, while overlooking how retrieved memories influence downstream reasoning and decision-making. To bridge this gap, we propose MemSyco-Bench, a comprehensive benchmark for evaluating memory-induced sycophancy in agent systems. MemSyco-Bench measures when memory should influence a decision and how valid memory should be used. Specifically, it covers five tasks that assess whether agents can reject memory as factual evidence, respect its applicable scope, resolve conflicts between memory and objective evidence, track memory updates, and use valid memory for personalization. All related resources are collected for the community at https://github.com/XMUDeepLIT/MemSyco-Bench.
