Search papers, labs, and topics across Lattice.
The paper introduces SmartThinker, a Group Relative Policy Optimization (GRPO) method that dynamically calibrates the chain-of-thought (CoT) length of large reasoning models (LRMs) to improve efficiency. SmartThinker estimates the optimal CoT length during training and modulates the length reward coefficient to avoid penalizing correct reasoning paths, leading to reduced redundancy and improved accuracy. Experiments demonstrate that SmartThinker achieves up to 52.5% length compression with improved accuracy and up to 16.6% accuracy improvement on challenging benchmarks.
Achieve up to 52.5% compression in LLM chain-of-thought reasoning *while improving* accuracy by dynamically calibrating CoT length.
Large reasoning models (LRMs) like OpenAI o1 and DeepSeek-R1 achieve high accuracy on complex tasks by adopting long chain-of-thought (CoT) reasoning paths. However, the inherent verbosity of these processes frequently results in redundancy and overthinking. To address this issue, existing works leverage Group Relative Policy Optimization (GRPO) to reduce LRM output length, but their static length reward design cannot dynamically adapt according to the relative problem difficulty and response length distribution, causing over-compression and compromised accuracy. Therefore, we propose SmartThinker, a novel GRPO-based efficient reasoning method with progressive CoT length calibration. SmartThinker makes a two-fold contribution: First, it dynamically estimates the optimal length with peak accuracy during training and guides overlong responses toward it to reduce response length while sustaining accuracy. Second, it dynamically modulates the length reward coefficient to avoid the unwarranted penalization of correct reasoning paths. Extensive experiment results show that SmartThinker achieves up to 52.5% average length compression with improved accuracy, and achieves up to 16.6% accuracy improvement on challenging benchmarks like AIME25. The source code can be found at https://github.com/SJTU-RTEAS/SmartThinker.
