Search papers, labs, and topics across Lattice.
This paper introduces Maven, a reinforcement learning framework designed to enhance long-context reasoning by utilizing an editable evidence memory that tracks and rewards intermediate actions affecting the evidence state. By defining an answer-conditioned evidence-state value, Maven provides nuanced feedback on actions such as adding, linking, and dropping evidence, leading to improved evidence set sufficiency and reduced distractor retention. Experiments on Llama and Qwen models across multiple benchmarks demonstrate that Maven significantly outperforms traditional outcome-only RL methods and static evidence extraction approaches.
Optimizing stateful evidence navigation in long-context reasoning leads to more effective evidence synthesis and lower distractor retention than traditional methods.
Long-context reasoning requires models to locate, revise, and synthesize evidence distributed across lengthy inputs. Existing long-context RL methods usually reward final answers or static evidence extraction, offering little feedback on how intermediate actions change the model's evidence state. We propose Maven, a reinforcement learning framework with an editable evidence memory. Maven defines an answer-conditioned evidence-state value and rewards action-level state transitions: add actions are credited by marginal gain and hindsight contribution, link actions by evidence synergy, and drop actions by improved answer support after removing misleading evidence. These rewards are assigned to the corresponding action spans in GRPO. Across Llama and Qwen models on LongBench v2, LongReason, and RULER, Maven outperforms outcome-only RL and evidence-identification baselines, producing more sufficient evidence sets and lower distractor retention. Our results show that long-context RL benefits from optimizing stateful evidence navigation rather than one-shot evidence extraction.
