Search papers, labs, and topics across Lattice.
This paper introduces EvoThink, a novel framework designed to enhance the reasoning efficiency and capability of Large Reasoning Models (LRMs) by addressing the issue of redundant verification steps. The framework incorporates Self-Pruning Training (SPT) to iteratively eliminate unnecessary reasoning steps and Aha-Moment Preference Optimization (AMPO) to leverage valuable failed reasoning attempts for improved learning. Experimental results show that EvoThink significantly reduces inference-time token usage while simultaneously enhancing the reasoning performance of LRMs on mathematical reasoning and code generation tasks.
EvoThink reduces inference-time token usage by pruning redundant reasoning steps while boosting LRM performance through the optimization of valuable failed attempts.
Large Reasoning Models (LRMs) often suffer from overthinking due to redundant verification steps. Existing approaches for mitigating overthinking, such as fast-slow thinking switching and reasoning trajectory compression, fail to make a fine-grained distinction between beneficial and redundant steps within the LRM's reasoning process, and may thus impair reasoning capability in their pursuit of efficiency. To simultaneously improve reasoning efficiency and capability, we propose EvoThink, a framework that reduces redundant verification and encourages the exploration of new reasoning paths. EvoThink comprises two key components: Self-Pruning Training (SPT), an unsupervised method that iteratively prunes redundant reasoning steps and self-trains on the concise trajectories; and Aha-Moment Preference Optimization (AMPO), which, inspired by genetic algorithms, identifies valuable failed reasoning attempts, synthesizes from-wrong-to-right aha-moment data, and optimizes the model to internalize this reasoning pattern. Extensive evaluations across mathematical reasoning and code generation benchmarks demonstrate that EvoThink not only substantially reduces inference-time token usage but also improves the reasoning capability of LRMs.
