Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study critically evaluates next-chunk reasoning reinforcement learning (RL) against a simpler Mixed supervised fine-tuning (SFT) approach using no-CoT data. The findings reveal that Mixed SFT not only surpasses next-chunk reasoning RL in performance but also requires over 60 times less training compute, challenging the assumption that complex RL methods are inherently superior. Additionally, the research underscores the importance of assessing training strategies within the entire post-training pipeline, as higher pre-RLVR accuracy does not guarantee better outcomes post-RLVR.
Mixed SFT outperforms next-chunk reasoning RL while consuming over 60 times less compute, reshaping our understanding of effective training strategies with no-CoT data.
Recent work proposes next-chunk reasoning RL for leveraging no-CoT data---corpora such as worked solutions and textbook derivations that contain reasoning-rich content but lack explicit chain-of-thought annotations. The method trains a model to generate implicit reasoning traces and rewards them by their ability to predict the next chunk of text. While promising, existing evaluations primarily compare against conventional SFT baselines, leaving open whether the gains come from the RL formulation itself or from more effectively exposing the model to no-CoT data. We address this question with a controlled study of next-chunk reasoning RL and a simple but previously overlooked alternative: Mixed SFT, a single supervised fine-tuning stage that jointly trains on no-CoT and long-CoT data. Despite its simplicity, Mixed SFT achieves a clearly higher post-RLVR performance ceiling than next-chunk reasoning RL while requiring over 60 times less training compute. The advantage is consistent across in-domain mathematical reasoning and out-of-domain reasoning tasks. Moreover, we show that higher pre-RLVR accuracy does not necessarily translate into higher post-RLVR accuracy, highlighting the need to evaluate no-CoT training strategies in the context of the full post-training pipeline.