Search papers, labs, and topics across Lattice.
This paper introduces ECHO, a selective turn-memory framework designed to enhance long-horizon decision-making in agentic reinforcement learning (RL) by addressing the limitations of existing context-management methods. ECHO compresses completed environment turns into compact memory records, allowing for the reconstruction of bounded policy contexts while maintaining source-addressable historical evidence for outcome-based RL updates. The approach significantly improves performance on the BrowseComp-Plus benchmark, achieving 43.4% held-out accuracy, surpassing previous methods while utilizing fewer turns and lower trajectory volume.
ECHO enables RL agents to retain and leverage fine-grained historical evidence, achieving a 43.4% accuracy on complex tasks while using less context than prior methods.
Long-horizon language agents must repeatedly interact with tools, accumulate evidence, and make decisions under bounded context windows. Existing context-management methods make such rollouts feasible by truncating distant history, folding past turns into summaries, or selecting compact memory states. However, these breakthroughs introduce two coupled limitations. First, as the number of turns grows, historical observations are progressively removed or collapsed into compressed states, making it harder for the policy to reuse fine-grained evidence. Second, once the original turns are no longer source-addressable, outcome-based RL loses an explicit path for aligning policy updates with the evidence that supported a successful final answer. To this end, we propose ECHO, a selective turn-memory framework that jointly addresses history collapse and traceable learning through source-indexed reconstruction. Specifically, ECHO compresses each completed environment turn into a compact memory record, reconstructs bounded policy contexts by selecting from these records, and reuses the selected source indices to route positive outcome credit to the evidence and selection actions that support successful answers. On BrowseComp-Plus, ECHO reaches 43.4% held-out accuracy, outperforming GRPO (28.9%) and the rolling-summary baseline SUPO (36.1%), while using fewer turns and lower trajectory volume than SUPO (Figure 1). Additionally, the trained policy improves zero-shot generalization across multi-objective QA, code generation, and deep information-seeking benchmarks on both dense and MoE backbones.
