Search papers, labs, and topics across Lattice.
This paper introduces EcoVideo, an innovative framework that enhances video generation by dynamically decoupling inter-frame dependencies through entropy-based keyframe selection and motion-aware interpolation. By leveraging early-stage self-attention entropy to assess frame information density, EcoVideo allows a cloud model to efficiently denoise keyframes while an edge model reconstructs the remaining frames, adapting to real-time resource constraints. Experimental results demonstrate that EcoVideo achieves significant improvements in quality-efficiency trade-offs, achieving up to a 2.9x speedup in low-bandwidth, compute-limited environments.
EcoVideo achieves up to 2.9x faster video generation in resource-constrained environments by intelligently orchestrating cloud-edge dynamics.
DiT video generation is latency-intensive due to iterative full-frame denoising, while prior cloud-edge methods largely rely on static inter-step decoupling and cannot leverage inter-frame similarity or adapt to system dynamics. We propose EcoVideo, an entropy-orchestrated framework for dynamic inter-frame decoupling: early-stage self-attention entropy provides a training-free estimate of frame-wise information density for frame selection; a cloud large model denoises sparse high-entropy keyframes; and an edge lightweight model reconstructs the remaining frames via motion-aware interpolation with refinement for temporal stability. EcoVideo further adapts the keyframe budget and edge refinement depth to real-time bandwidth and compute availability, optimizing end-to-end latency under constraints. Experiments on representative DiT video generators show improved quality--efficiency trade-offs and up to 2.9x end-to-end speedup in low-bandwidth, compute-limited edge settings. Code is available at https://github.com/IF-LAB-PKU/EcoVideo.
