Search papers, labs, and topics across Lattice.
This paper introduces OmniCache, a hierarchical caching framework designed to optimize inference efficiency in high-resolution image and video diffusion models by leveraging redundancy in intermediate diffusion features. By identifying four sources of redundancy鈥攊ntra-frame, inter-frame, motion, and denoising-step鈥攖he authors implement a multidimensional feature reuse strategy that enhances computational efficiency without altering model weights or requiring retraining. The results demonstrate significant reductions in inference latency of up to 35% across various models, while preserving visual fidelity and motion coherence.
Caching intermediate features can slash inference latency by up to 35% in diffusion models without sacrificing quality or requiring retraining.
High-resolution image and video diffusion models, including SD3, FLUX, and recent video diffusion transformers, have substantially improved generative quality but remain expensive at inference time because they repeatedly evaluate attention-heavy denoisers over many sampling steps. We address this inefficiency by exploiting redundancy in intermediate diffusion features rather than changing model weights or retraining. We identify four complementary redundancy sources in image and video generation: intra-frame, inter-frame, motion, and denoising-step redundancy. Based on this analysis, we propose OmniCache, a unified hierarchical caching framework that performs multidimensional feature reuse through Token Cache, Frame Cache, Block Cache, and Layered Cache. Unlike token-merging baselines that average matched features, OmniCache uses similarity matching to select cacheable features, skips redundant computation, and restores positionally consistent cached activations, preserving feature order and spatial-temporal structure. The resulting framework reuses spatial features in temporal layers and temporal features in spatial layers, while Layered Cache captures cross-step redundancy at the model-layer level. Across SD3, SVD-XT, and Latte, OmniCache reduces inference latency by up to 35%, 25%, and 28%, respectively, while maintaining visual fidelity and motion coherence in a training-free setting.
