Search papers, labs, and topics across Lattice.
This paper addresses the reward-credit mismatch in reinforcement learning for deep-search agents by introducing STAMP, a method that utilizes a reference-based verifier to assess the support of cited documents in a training-time evidence graph. By implementing first-exposure attribution, STAMP effectively traces citations back to the actions that first revealed them, redistributing credit through sign-preserving advantage modulation. The approach significantly enhances performance on multiple benchmarks, achieving notable improvements over the GRPO baseline while maintaining trajectory-level reward integrity.
STAMP redefines credit assignment in deep search agents, leading to up to 5.5-point gains in performance by effectively linking actions to their supporting evidence.
Reinforcement learning for deep-search agents has largely focused on trajectory-level scoring -- outcome correctness, citation-aware rewards, and evidence coverage. Yet the actions that expose supporting documents receive no targeted credit, a gap we call the reward-credit mismatch. We propose STAMP, in which a reference-based verifier judges whether each cited document supports an entity or relation in a training-time evidence graph, and first-exposure attribution traces each supported citation back to the action that first surfaced it. This step credit is injected through sign-preserving advantage modulation, which redistributes advantage across steps without changing the trajectory-level reward or the relative ranking of trajectories within each group. On BrowseComp, BrowseComp-ZH, and xbench-DS, STAMP improves the GRPO baseline by +2.0/+5.5/+3.0 points under matched SFT initialization, training data, and search tools, and composes with both outcome-only and citation-rubric base rewards. Component ablations confirm that the provenance-based credit signal and the sign-preserving advantage modulation each contribute to the gains.
