Search papers, labs, and topics across Lattice.
The paper introduces High-Entropy Sum (HES), a training-free metric that estimates the quality of LLM reasoning samples by summing the entropy of the highest-entropy tokens. Validated across SFT, RFT, and RL, HES consistently identifies high-quality reasoning data, enabling effective training with significantly reduced computational cost. Experiments show that training on the top 20% of HES-ranked data matches full-dataset performance in SFT, outperforms baselines in RFT, and enables superior reasoning patterns in RL.
Forget expensive data curation: a simple, training-free entropy metric lets you train LLMs on just 20% of your reasoning data without sacrificing performance.
Effectively training Large Language Models (LLMs) for complex, long-CoT reasoning is often bottlenecked by the need for massive high-quality reasoning data. Existing methods are either computationally expensive or fail to reliably distinguish high- from low-quality reasoning samples. To address this, we propose High-Entropy Sum (HES), a training-free metric that quantifies reasoning quality by summing only the entropy of the top (e.g., 0.5\%) highest-entropy tokens in each reasoning sample. We validate HES across three mainstream training paradigms: Supervised Fine-tuning (SFT), Rejection Fine-tuning (RFT), and Reinforcement Learning (RL), with extensive results demonstrating its consistent effectiveness and significantly reduced computational overhead. In SFT, training on the top 20\% HES-ranked data matches full-dataset performance, while using the lowest-HES data degrades it. In RFT, our HES-based training approach significantly outperforms baseline methods. In RL, HES-selected successful trajectories enable the model to learn strong reasoning patterns, significantly surpassing other compared methods. Our findings establish HES as a robust, training-free metric that enables a unified, effective, and efficient method for developing advanced reasoning in LLMs.
DAMO
NUS