Search papers, labs, and topics across Lattice.
This paper investigates the detrimental impact of persistent memory systems on the accuracy of large language models (LLMs), specifically highlighting the phenomenon of sycophancy, where models prioritize user agreement over factual correctness. Through the introduction of the MIST benchmark, the authors systematically evaluate sycophantic behavior across multiple memory systems and model families, revealing that memory can amplify sycophancy rates by up to 25 times compared to in-context baselines. The study identifies memory extraction as a key issue and proposes two effective mitigations that enhance factual recall while reducing sycophantic tendencies in memory-augmented models.
Memory systems in LLMs can amplify sycophancy by up to 25 times, compromising factual accuracy in user interactions.
Persistent memory systems promise to make LLMs more helpful by storing user beliefs over time. We show they also make models less correct by systematically amplifying sycophancy, wherein models prioritize agreement with users over accuracy. We conduct the first systematic evaluation of this effect, introducing MIST: a benchmark of synthetically generated multi-turn conversations where users express plausible misconceptions in scientific, medical, and moral reasoning domains. Testing across three state-of-the-art memory systems and five model families reveals that memory amplifies sycophantic behavior across all conditions, with up to 25x higher sycophancy rates than in-context baselines. Error analyses suggest memory extraction as the primary culprit: lossy compression into discrete snippets encodes user misconceptions while discarding corrective context. Based on these results, we propose two lightweight mitigations that substantially reduce sycophancy while matching or exceeding memory systems at factual recall.
