Search papers, labs, and topics across Lattice.
This paper addresses the significant drop in accuracy of large language models (LLMs) when task-critical information is revealed over multiple conversation turns, with a noted decrease of up to 65%. By employing a memory-augmented reinforcement learning approach and a novel low-cost sharding pipeline, the authors train models to maintain a compact rolling memory, which enhances multi-turn reasoning. The results show that models trained on sharded datasets not only improve accuracy in multi-turn scenarios but also generalize better to more complex tasks, outperforming traditional full-history baselines even when provided with complete context at test time.
Training LLMs with a compact rolling memory can lead to more robust reasoning, outperforming models that rely on full historical context.
When a user reveals task-critical information across several conversation turns, LLM accuracy drops by up to 65% despite full context availability. We show that this Lost in Conversation degradation can be substantially mitigated by training models to maintain a compact rolling memory instead of attending to a growing history. To make such training scalable, we introduce a low-cost sharding pipeline that converts single-turn QA datasets into multi-turn fragmented-information episodes, eliminating the need for hours of manual annotation. Training only on sharded GSM8K, our memory-augmented policy significantly improves multi-turn accuracy and generalises zero-shot to harder math and out-of-domain long-context QA. Moreover, memory-trained models outperform full-history baselines even when given the full history at test time, suggesting that learning to compress induces more robust incremental reasoning than full-context exposure alone.
Google Research