Search papers, labs, and topics across Lattice.
This paper introduces GVCHR, a novel generative video compression method that leverages hierarchical referencing to enhance both latent coding and generative reconstruction. By organizing latent frames hierarchically and employing a Hierarchical Temporal Context Mining approach, GVCHR effectively allocates coding resources and minimizes artifact propagation during denoising. Experimental results demonstrate that GVCHR outperforms the state-of-the-art by achieving significant BD-rate gains of 50.5% and 54.0% in LPIPS and DISTS metrics, respectively, while also improving visual quality.
Hierarchical referencing in video compression can lead to over 50% gains in perceptual quality metrics, revolutionizing generative reconstruction.
Diffusion-based generative video compression has emerged as a promising paradigm to improve perceptual quality, where latent frames are required to be encoded efficiently while serving as denoising conditions. However, existing methods neither carefully design reference and quality structures during latent coding nor account for the impact of frame-level quality variation on denoising procedure, which limits coding efficiency and aggravates artifact propagation during generative reconstruction. In this paper, we propose GVCHR, Generative Video Compression based on Hierarchical Referencing. The key idea is to organize latent frames hierarchically, where the selected high-quality references benefit both latent coding and generative reconstruction. In latent coding, GVCHR couples a hierarchical reference structure with a hierarchical quality structure, assigning more bits to lower-layer frames that are reused more frequently as references. Built on this design, we introduce Hierarchical Temporal Context Mining to exploits complementary short- and long-term temporal context for effective latent coding. In generative reconstruction, the coding-side hierarchy is incorporated into a Hierarchical Attentive Adapter which is attached to a video diffusion transformer. This adapter uses hierarchical attention to restrict each latent frame to attend only to the same- or lower-layer references, thereby reducing artifact propagation during denoising. Experiments validate GVCHR on multiple benchmarks. Compared with the previous state-of-the-art method, GVCHR achieves 50.5% and 54.0% BD-rate gains in terms of LPIPS and DISTS, respectively, while also delivering clearly improved visual quality.
