Search papers, labs, and topics across Lattice.
This paper addresses the problem of redundant reasoning steps ("overthinking") in Chain-of-Thought (CoT) prompting, which arises from inefficient reflection patterns induced by RL training. They propose a graph-based CoT optimization framework that converts linear CoTs into directed acyclic graphs (DAGs) and employs branch-level and depth-level pruning to remove weakly contributing or repetitive reflection steps. Experiments demonstrate a 42% reduction in reasoning tokens with maintained or improved accuracy, achieved through a three-stage SFT, DPO, and GRPO distillation pipeline.
LLMs can slash 42% of their reasoning tokens without losing accuracy by pruning redundant reflections in their chain-of-thought process.
Extending CoT through RL has been widely used to enhance the reasoning capabilities of LLMs. However, due to the sparsity of reward signals, it can also induce undesirable thinking patterns such as overthinking, i.e., generating redundant intermediate reasoning content. In this work, we argue that a major source of such redundancy is inefficient reflection, which often manifests in two problematic patterns: Indiscriminate Reflection, where the model performs broad, low-impact checks throughout reasoning, and Repetitive Reflection, where it repeatedly re-verifies an already established conclusion. To address this, we introduce a graph-based CoT optimization framework. Specifically, we convert each linear CoT into a directed acyclic graph (DAG) with explicit dependency edges, and design a dual pruning strategy: branch-level pruning removes weakly contributing reflection branches, while depth-level pruning eliminates late-stage re-verification. We distill this behavior via a three-stage pipeline: (1) SFT to initialize the policy on pruned concise traces, (2) DPO to prefer correct but less redundant trajectories, and (3) GRPO with length penalty to jointly optimize answer correctness and efficiency. Experiments show that our approach reduces the average reasoning tokens by 42\% while maintaining or improving accuracy.
