Search papers, labs, and topics across Lattice.
The paper introduces Chorus, a novel caching approach to accelerate video diffusion transformer (DiT) model serving by exploiting similarities across different requests. Chorus employs a three-stage caching strategy, including full latent feature reuse and inter-request caching in specific latent regions, combined with Token-Guided Attention Amplification to improve semantic alignment. Experiments on industrial 4-step distilled models show Chorus achieves up to 45% speedup, outperforming existing intra-request caching methods.
Chorus unlocks 45% speedups in video diffusion inference by cleverly reusing computations across user requests, even in highly optimized 4-step models where traditional caching fails.
Video Diffusion Transformer (DiT) models are a dominant approach for high-quality video generation but suffer from high inference cost due to iterative denoising. Existing caching approaches primarily exploit similarity within the diffusion process of a single request to skip redundant denoising steps. In this paper, we introduce Chorus, a caching approach that leverages similarity across requests to accelerate video diffusion model serving. Chorus achieves up to 45\% speedup on industrial 4-step distilled models, where prior intra-request caching approaches are ineffective. Particularly, Chorus employs a three-stage caching strategy along the denoising process. Stage 1 performs full reuse of latent features from similar requests. Stage 2 exploits inter-request caching in specific latent regions during intermediate denoising steps. This stage is combined with Token-Guided Attention Amplification to improve semantic alignment between the generated video and the conditional prompts, thereby extending the applicability of full reuse to later denoising steps.
