Search papers, labs, and topics across Lattice.
This paper introduces SyncCache, a training-free caching acceleration method designed for Diffusion Transformers (DiTs) in audio-driven portrait animation, addressing the high computational costs and inference latency associated with existing techniques. By leveraging asymmetric dynamics, SyncCache prioritizes caching of high-frequency audio-driven dynamics in human regions while efficiently managing low-frequency visual backgrounds. Experimental results show that SyncCache achieves up to 4.12x acceleration with near-lossless visual fidelity and precise audio alignment, significantly improving the speed-quality trade-off in portrait animation tasks.
SyncCache accelerates audio-driven portrait animation by up to 4.12x while maintaining near-lossless visual quality, transforming the efficiency of DiT-based systems.
Diffusion Transformers (DiTs) have significantly advanced audio-driven portrait animation, but their high computational cost leads to substantial inference latency. Although training-free diffusion caching accelerates inference significant, existing methods are primarily developed for text-conditioned generation and overlook the spatial and modality imbalances inherent in audio-driven portrait animation. In this paper, we propose SyncCache, a training-free caching acceleration method tailored for DiT-based portrait animation that explicitly exploits asymmetric dynamics. Specifically, high-frequency dynamics driven by audio conditions and concentrated in human regions are more challenging and critical to cache and reuse than the low-frequency visual background in portrait animation. First, we introduce Spatially-Asymmetric Probing to prioritize error sensitivity in dynamic human region. Second, through Modality-Decoupled Caching, we bypass heavy DiT block by reusing stable inter-block residuals, while continuously recomputing lightweight audio blocks to preserve precise lip synchronization. Furthermore, we introduce a cache ratio to control cache capacity and formulate memory-adaptive cache selection as an offline dynamic programming problem without online overhead. Extensive experiments demonstrate that SyncCache achieves superior speed-quality trade-offs, delivering up to 4.12x acceleration on HunyuanVideo-Avatar and 3.75x on Wan-S2V with near-lossless visual fidelity and precise audio alignment.
