Search papers, labs, and topics across Lattice.
This paper introduces Contrastive Reasoning Path Synthesis (CRPS), a method for extracting more efficient supervision signals from MCTS by contrasting successful and unsuccessful search trajectories. CRPS identifies strategic pivots and failure modes by analyzing differences between high- and low-reward paths, then synthesizes reasoning chains that incorporate successful patterns while avoiding pitfalls. Fine-tuning models on CRPS-synthesized examples achieves comparable or superior performance to models trained on 10x larger datasets generated via rejection sampling, and improves out-of-domain generalization.
Forget sifting through mountains of MCTS trajectories – contrasting successful and failed reasoning paths distills 10x more effective training data for reasoning models.
Monte Carlo Tree Search (MCTS) has been widely used for automated reasoning data exploration, but current supervision extraction methods remain inefficient. Standard approaches retain only the single highest-reward trajectory, discarding the comparative signals present in the many explored paths. Here we introduce \textbf{Contrastive Reasoning Path Synthesis (CRPS)}, a framework that transforms supervision extraction from a filtering process into a synthesis procedure. CRPS uses a structured reflective process to analyze the differences between high- and low-quality search trajectories, extracting explicit information about strategic pivots and local failure modes. These insights guide the synthesis of reasoning chains that incorporate success patterns while avoiding identified pitfalls. We show empirically that models fine-tuned on just 60K CRPS-synthesized examples match or exceed the performance of baselines trained on 590K examples derived from standard rejection sampling, a 20$\times$ reduction in dataset size. Furthermore, CRPS improves generalization on out-of-domain benchmarks, demonstrating that learning from the contrast between success and failure produces more transferable reasoning capabilities than learning from success alone.
