Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces an inference auction mechanism to address compute capacity constraints in LLM inference by allowing users to bid for faster service, thereby economically allocating priority without sacrificing latency. The authors develop efficient algorithms for truthful bidding and an autobidding agent that maximizes user utility within a budget. Experimental validation demonstrates that the proposed auction increases system welfare while preserving the performance benefits of the SGLang inference serving framework.
LLM inference auctions unlock economic efficiency and user utility by enabling dynamic bidding for compute priority, outperforming static pricing tiers.
When inference demand exceeds available compute capacity, model providers must decide which requests should be served first. Users have different tolerances for delay from an LLM API, but current priority pricing schemes compress these differences into coarse fixed-price service tiers. We design an inference auction that allows users to bid for faster service. Our auction allocates priority in an economically efficient way without sacrificing latency, and we develop fast algorithms for implementing prices that incentivize truthful bidding. We also design an autobidding agent for our inference auction, where users specify an inference budget and the autobidder dynamically adjusts its bids over time to maximize user utility subject to the budget constraint. Experiments validate the practicality of our auction: it increases system welfare while maintaining the cache utilization and latency advantages of SGLang, a state-of-the-art inference serving framework.
Google Research