Search papers, labs, and topics across Lattice.
This paper introduces EviSD, an evidence-conditioned self-distillation framework that enhances outcome-based reinforcement learning for search-augmented language agents by utilizing instance-level supporting evidence and golden answers. By allowing the model to sample actions from the original context while re-scoring them under an action-aligned context, EviSD effectively narrows the teacher-student gap and refines the learning process without requiring additional distillation objectives. The framework demonstrates significant improvements across seven question-answering benchmarks, achieving the highest macro-average Exact Match scores and outperforming leading methods by 1.3–2.3 points with minimal modulation of response tokens.
EviSD achieves state-of-the-art performance in question-answering tasks by leveraging privileged evidence, outperforming existing methods while maintaining efficiency in response generation.
Outcome-based reinforcement learning enables search-augmented language agents to learn from verifiable final answers, but its trajectory-level credit cannot distinguish the contributions of individual actions in a multi-turn search process. We propose EviSD, an evidence-conditioned self-distillation framework that uses instance-level supporting evidence as privileged information for search actions and golden answers as complementary privilege for answer actions. During training, the student samples actions from the original context, while the same model re-scores them as a privileged teacher under an action-aligned context. EviSD converts the detached teacher--student gap into a bounded correction to the outcome-derived GRPO advantage and applies it only to generated action spans. This design localizes privileged guidance while preserving the update direction determined by the outcome reward, without an auxiliary distillation objective or any change at inference time. Across seven question-answering benchmarks and three backbones spanning model scales and generations, EviSD achieves the highest macro-average Exact Match in all evaluated settings, outperforming the strongest compared methods by 1.3--2.3 points while modulating only 6.7%--15.1% of response tokens. Code is available at https://github.com/JiananXie/EviSD.
