Search papers, labs, and topics across Lattice.
This paper formulates the problem of inference budget allocation for Large Language Models (LLMs) as a global constrained optimization issue, leveraging economic principles to enhance performance under strict computational constraints. By introducing the Constrained Latent-utility Equilibrium Allocation for Reasoning (CLEAR) method, the authors optimize resource distribution by reallocating from insolvent to solvable queries based on a derived shadow price. Experimental results show that CLEAR can improve global accuracy by up to 3x in resource-scarce environments compared to traditional uniform allocation methods, significantly enhancing the efficiency of LLM deployment.
CLEAR reallocates resources in LLMs to boost accuracy by up to 3x in resource-limited settings, challenging traditional uniform allocation strategies.
Inference-time scaling has emerged as a critical avenue for enhancing Large Language Models'performance, yet real-world deployment is constrained by strict computational budgets. In this work, we formulate inference budget allocation as a global constrained optimization problem governed by economic principles. By modeling per-query reasoning utility with a shifted-surge function, we derive an optimal allocation policy based on a global shadow price that equilibrates marginal utility under resource scarcity. Based on this theory, we propose Constrained Latent-utility Equilibrium Allocation for Reasoning (CLEAR). It performs rational abandonment and reallocates resources from insolvent queries to solvable queries near their emergence thresholds. Extensive experiments on several reasoning tasks with different traffic streams demonstrate that CLEAR significantly improves the Pareto frontier of total token cost versus mean accuracy. In resource-scarce regimes, CLEAR achieves up to a 3x improvement in global accuracy compared to uniform allocation.
