Search papers, labs, and topics across Lattice.
The paper introduces VoRTeC, a novel video compression framework that leverages a foundational flow model to overcome the limitations of traditional neural and diffusion-based video compression methods. By effectively encoding latent video representations and utilizing multi-scale priors, VoRTeC achieves high perceptual fidelity while significantly reducing bit consumption by 58% compared to previous approaches. Additionally, it enhances decoding speed dramatically, achieving rates of 13 FPS at 720p and 32 FPS at 480p, thus addressing both quality and efficiency in real-time video compression.
VoRTeC slashes bit consumption by 58% while boosting decoding speeds up to 197 times, revolutionizing real-time video compression.
Ultra-low bitrate video compression still faces critical challenges: traditional neural video compression inevitably introduces blurring artifacts, while diffusion-based generative video compression suffers from excessive decoding latency and poor temporal consistency. To address these issues, we propose $\mathtt{VoRTeC}$, a Video Compression framework built upon a foundational flow model (Wan2.1). By compactly encoding latent video representations, predicting the positions of compressed representations along flow trajectories, and integrating multi-scale priors, $\mathtt{VoRTeC}$ enables the compressor to harness generative video flow priors effectively. Without accessing the parameters or gradients of flow matching networks, our framework achieves one-step decoding and reconstructions with high perceptual fidelity. Meanwhile, we maintain consistency across frame groups via tail-frame reuse and prior caching. Extensive experiments demonstrate that our method reduces bit consumption by 58\% compared to prior diffusion-based approaches, with decoding speed boosted by 3 to 197 times: $\mathtt{VoRTeC}$ achieves a decoding speed of 13 FPS at 720p and 32 FPS at 480p.
