Search papers, labs, and topics across Lattice.
This paper introduces CODA, an innovative architecture that employs Compute-Cache Operator Disaggregation to enhance the efficiency of Video Diffusion Models (VDMs) on edge devices. By separating compute and cache operations and optimizing cache activity, CODA significantly reduces the memory footprint and communication overhead associated with cross-timestep caching. Experimental results demonstrate that CODA achieves up to 1.80x speedup and 1.74x improved energy efficiency while maintaining competitive generation quality compared to existing methods.
CODA redefines edge video diffusion by achieving up to 1.80x faster inference and 1.74x better energy efficiency without sacrificing output quality.
Deploying Video Diffusion Models (VDMs) on edge devices is appealing for localized and privacy-preserving generation, but their iterative Transformer-based denoising remains too slow for practical local inference. Cross-Timestep Caching (CTC) has emerged as a promising direction for reducing redundant computation, reusing activations across adjacent denoising steps rather than modifying model weights, while largely preserving generation fidelity. However, on memory-constrained edge GPUs, CTC requires a massive cache footprint that quickly exceeds on-device VRAM and forces the cache into host memory. More fundamentally, cache operators remain tightly interleaved and chain-dependent with native compute operators, so naive near-memory offloading still incurs repeated PCIe exchanges for residual and fusion computations, turning cache reuse into a communication- and serialization-bound execution flow. We therefore propose CODA, an algorithm-hardware co-designed architecture centered on Compute-Cache Operator Disaggregation. CODA separates dense compute paths and memory-bound cache paths across the xPU and a lightweight DIMM-side near-memory engine, reorganizes fragmented cache activity into hardware-friendly coalesced segments, and exploits Classifier-Free Guidance (CFG) branch independence to overlap xPU compute with cache-side execution. Experiments show that CODA achieves up to 1.80x end-to-end speedup and 1.74x higher energy efficiency, while preserving competitive generation quality compared with a state-of-the-art caching algorithm.
DAMO