Search papers, labs, and topics across Lattice.
This paper introduces SCOPE-RL, a two-stage framework designed to enhance reinforcement learning with verifiable rewards (RLVR) for large language models (LLMs) by optimizing reasoning paths both before and after achieving success. The method incorporates Adaptive Scaffolded RL, which provides prefix-decomposed verifiable rewards for sub-question chains prior to success, and Quality-Aware Process RL, which refines trajectories post-success using correctness-gated process-shape rewards. Experimental results show that SCOPE-RL achieves an average accuracy improvement of up to 11.2 percentage points and reduces reasoning tokens by up to 27.1% compared to traditional outcome-only approaches, demonstrating the effectiveness of reward-signal densification in LLM training.
SCOPE-RL boosts LLM accuracy by over 11% while cutting reasoning tokens significantly, reshaping how we optimize reasoning paths in AI.
Reinforcement learning with verifiable rewards (RLVR) optimizes LLMs using sparse verifiable final-answer rewards. This sparse anchor reliably verifies whether a trajectory succeeds but provides no direct feedback on the reasoning path that produced it. Before success, prerequisite progress on hard problems receives no reward signal; after success, outcome rewards cannot distinguish well-organized correct trajectories from redundant or locally flawed ones. We introduce SCOPE-RL (Scaffolded Chain Optimization with Process Efficiency), a two-stage framework that densifies this anchor while retaining the GRPO update: Adaptive Scaffolded RL adds prefix-decomposed verifiable rewards on answer-hidden sub-question chains before success, and Quality-Aware Process RL applies correctness-gated process-shape rewards to refine correct trajectories after success. An expert-validated Step-Quality Evaluation Protocol evaluates useful-step density, error localization, and token efficiency beyond final-answer accuracy. On Qwen3-8B-Instruct trained on DAPO-Math and Big-Math, SCOPE-RL improves average accuracy by up to 11.2 pp and reduces reasoning tokens by up to 27.1% over outcome-only GRPO; the gains hold under GSPO and on Qwen3-0.6B-Instruct, indicating that reward-signal densification is complementary to policy-update-level RLVR advances. Code and data are available at https://github.com/tokencraft-lab/SCOPE-RL.
