Search papers, labs, and topics across Lattice.
This paper introduces BudCache, a novel caching strategy for diffusion models that optimizes for final output quality by fixing the compute budget in advance rather than relying on heuristic error thresholds. By employing a combination of Simulated Annealing and deterministic Hill Climbing, BudCache efficiently identifies high-quality cache policies that minimize inference latency while maintaining generation quality. Experimental results demonstrate that BudCache outperforms existing heuristic caching methods under the same computational constraints, particularly when the compute budget is limited.
BudCache redefines caching for diffusion models by prioritizing output quality over traditional heuristic approaches, achieving superior results within fixed compute budgets.
Step-level caching accelerates diffusion models by exploiting temporal redundancy across denoising steps. Existing methods make per-step cache decisions using threshold-based heuristics, without directly optimizing for final output quality. As a result, their inference latency varies across inputs and is difficult to control at deployment. In this work, we propose BudCache, which inverts this formulation: rather than letting per-step error thresholds dictate the runtime cost, we fix the compute budget in advance and search for the cache policy that best preserves the final output. To tackle the combinatorial complexity of step selection, we combine Simulated Annealing with deterministic Hill Climbing. This offline search identifies high-quality cache policies within minutes and introduces no online search or thresholding overhead during inference. When the compute budget is very tight, we further introduce cache-aware schedule alignment, which adapts the time discretization to the selected cache policy to reduce cache-induced trajectory mismatch. Experiments on FLUX.1-dev and Wan2.1 show that BudCache achieves better generation quality than heuristic caching baselines under the same inference budgets. Code is available at https://github.com/Westlake-AGI-Lab/BudCache
