Search papers, labs, and topics across Lattice.
This paper introduces Flexible Self-Consistency (FSC), a novel decoding strategy that dynamically adjusts the number of reasoning paths sampled based on a continuous estimation of problem difficulty. By predicting output entropy as a measure of model uncertainty, FSC effectively allocates computational resources, leading to significant token savings without sacrificing accuracy. Experimental results demonstrate that FSC can achieve up to 76% token savings while maintaining performance comparable to traditional Self-Consistency methods across various models and benchmarks.
Dynamic adjustment of reasoning paths based on continuous difficulty estimates can save up to 76% in token consumption without losing accuracy in complex reasoning tasks.
Self-Consistency (SC) is a decoding strategy that samples diverse reasoning paths and selects the most consistent answer, demonstrating strong performance on complex reasoning problems. However, the excessive token consumption incurred by generating multiple reasoning paths has been identified as a major limitation of SC. To improve computational efficiency, several studies have proposed strategies that adjust the number of reasoning paths or allocate resources differentially according to problem difficulty. Nevertheless, most existing methods categorize difficulty into a few fixed levels, failing to fully capture the continuously varying nature of reasoning complexity. In this work, we propose Flexible Self-Consistency (FSC), which estimates problem difficulty as a continuous signal and dynamically adjusts the number of generated reasoning paths accordingly. FSC predicts the output entropy of an input question using a pre-trained probe and leverages it as an indicator of model uncertainty to flexibly control the sampling budget. Experimental results show that, across various models and benchmarks, FSC maintains accuracy comparable to SC while achieving token savings of up to 76%.
