Search papers, labs, and topics across Lattice.
This paper introduces a two-stage framework for improving the accuracy-efficiency trade-off in large reasoning models (LRMs) by enabling stable adaptive thinking. The framework first uses Hybrid Fine-Tuning to expose the model to both thinking and no-thinking behaviors, and then employs adaptive reinforcement learning with Correctness-Preserving Advantage Shaping (CPAS) and Length-Aware Gradient Regulation (LAGR). Experiments on Qwen2.5-1.5B and 7B demonstrate that the proposed approach achieves significant improvements in accuracy and token reduction compared to existing methods, while also exhibiting robustness across varying problem difficulties and out-of-distribution tasks.
Stop wasting tokens: a novel RL framework slashes LRM token generation by 40% without sacrificing accuracy by adaptively controlling reasoning length.
Large reasoning models (LRMs) achieve strong performance through extended reasoning traces, but they often exhibit overthinking behavior for low-complexity queries. Existing efforts to mitigate this issue are fundamentally limited by unstable accuracy-efficiency trade-offs and poor robustness to heterogeneous reasoning behaviors. To address these challenges, we propose a two-stage framework for stable adaptive thinking in LRMs. The framework first applies Hybrid Fine-Tuning to expose the model to both thinking and no-thinking behaviors, establishing well-conditioned initialization. It then performs adaptive reinforcement learning with Correctness-Preserving Advantage Shaping (CPAS) to avoid suppressing correct long-chain reasoning, and Length-Aware Gradient Regulation (LAGR) to stabilize optimization under severe reasoning-length heterogeneity. Extensive experiments on Qwen2.5-1.5B and 7B show consistent improvements over strong baselines, achieving up to +3.7/+3.6 accuracy points while reducing generated tokens by 40.6%/43.9%. Further analyses across varying problem difficulties and out-of-distribution tasks confirm the robustness and generalization of our approach.
