Search papers, labs, and topics across Lattice.
This study investigates the ability of reasoning language models to adaptively allocate their computational resources by selecting from three distinct response modes: NoThink, Short, and Long. Utilizing Group Relative Policy Optimization (GRPO) with a shaped reward system, the model learns to optimize its reasoning effort based on problem difficulty, achieving a 41% reduction in mean response length while maintaining accuracy comparable to the base model. Notably, this adaptive approach also generalizes to other benchmarks, demonstrating significant token savings and improved performance on easier problems without requiring retraining.
Adaptive reasoning in language models can reduce token usage by 41% while maintaining high accuracy, reshaping efficiency in AI computations.
Reasoning language models trained with reinforcement learning typically operate under a fixed token budget rather than an explicitly adaptive one, which can lead to over-computation on easy problems and insufficient computation on difficult ones. We study whether a model can learn to allocate its own reasoning effort by choosing, as the first token of its response, one of three modes: \textsc{NoThink} (answer as quickly as possible), \textsc{Short} (brief reasoning), or \textsc{Long} (extended reasoning). The choice is learned inside Group Relative Policy Optimization (GRPO) with no separate router, through a shaped reward that makes each mode worthwhile at a different response length, together with hard per-mode token caps that keep the modes distinct. On a 1.5B distilled model trained on MATH, the three modes emerge without collapsing to a single choice, and the brief modes end up more accurate than \textsc{Long}, which shows that the router sorts problems by difficulty rather than at random. Averaged over three seeds, the resulting policy stays close to the base model's accuracy on the held-out MATH500 ($0.782$ vs.\ $0.796$) while cutting the mean response length from $4{,}796$ to $2{,}811$ tokens (a $41\%$ reduction). Interestingly, it also transfers to other benchmarks without retraining, with the largest savings where problems are easier, with for instance 76\% token reduction on GSM8K and at higher accuracy than the baselines at similar response length. In short, we build a reasoning model that adaptively chooses how much to reason for each problem.
