Search papers, labs, and topics across Lattice.
This study reveals that a language model's memory can be detrimental when it retains incorrect conclusions while discarding the underlying reasoning, leading to confidently erroneous outputs. Through a novel evaluation method called reclaim evaluation, the authors demonstrate that the ability to correct mistakes is fundamentally limited by whether the original source of the answer is preserved, rather than the model's overall capabilities. The findings indicate that implementing a source-first policy can significantly enhance correctability, achieving high scores in memory systems and real dialogue scenarios, while highlighting the risks of memory errors propagating through subsequent interactions.
Retaining incorrect conclusions in memory can be more harmful than having no memory at all, leading to a cascade of uncorrectable errors in language models.
A language model's memory can be worse than having no memory at all. Give a model a memory that kept a wrong conclusion but dropped the work behind it, and it emits that stale value as a confident answer; give the same model an empty memory and it abstains. Across seven models this direction never reverses, a clean kill condition that none breaks. We call this brittle memory: behavioral, not the near-immediate information bound beneath it; only its magnitude is disposition- and task-dependent, not its direction. We measure it with reclaim evaluation: compress a drifted interaction at a fixed budget, then test whether a correction recovers the known answer, scored against ground truth with no judge. Correctability is bottlenecked by whether the answer-determining source survives, not by capability. A one-line source-first policy (keep the recomputable source, drop the re-derivable conclusion) restores correctability at equal budget where that source is compact and identifiable; a length-matched control rules out added text as the cause. The hand-built oracle reaches 1.00; a one-prompt deployable version reclaims 0.49-0.88. The stake compounds: chained through a memory loop, a single dropped-source error corrupts a growing span of downstream steps and stays uncorrectable, while source-first holds to a bounded budget horizon. The wall and fix replicate across three deployed memory systems and on real dialogue (MultiWOZ), and past the budget where the source no longer fits, the fix fails silently unless the note records completeness. This is a controlled study of a mechanism, not a benchmark: judge-free exact scoring, matched-budget controls, and validators built to come out false. We release the harness, conditions, and validators.
