Search papers, labs, and topics across Lattice.
The paper introduces Generative Video Compression with One-Dimensional Latent Representation (GVC1D), a novel approach to generative video compression that encodes video into compact 1D latent tokens conditioned on both short- and long-term contexts. By eliminating the rigid 2D spatial correspondence of previous methods, GVC1D adaptively attends to semantic regions, facilitating token reduction and reducing spatial redundancy. Experiments demonstrate that GVC1D achieves significant bitrate reductions compared to previous video compression methods, achieving reductions of 60.4% under LPIPS and 68.8% under DISTS on the HEVC Class B dataset.
Ditching the 2D latent grid unlocks 60%+ bitrate reductions in generative video compression by encoding videos into adaptable 1D latent tokens.
Recent advancements in generative video codec (GVC) typically encode video into a 2D latent grid and employ high-capacity generative decoders for reconstruction. However, this paradigm still leaves two key challenges in fully exploiting spatial-temporal redundancy: Spatially, the 2D latent grid inevitably preserves intra-frame redundancy due to its rigid structure, where adjacent patches remain highly similar, thereby necessitating a higher bitrate. Temporally, the 2D latent grid is less effective for modeling long-term correlations in a compact and semantically coherent manner, as it hinders the aggregation of common contents across frames. To address these limitations, we introduce Generative Video Compression with One-Dimensional (1D) Latent Representation (GVC1D). GVC1D encodes the video data into extreme compact 1D latent tokens conditioned on both short- and long-term contexts. Without the rigid 2D spatial correspondence, these 1D latent tokens can adaptively attend to semantic regions and naturally facilitate token reduction, thereby reducing spatial redundancy. Furthermore, the proposed 1D memory provides semantically rich long-term context while maintaining low computational cost, thereby further reducing temporal redundancy. Experimental results indicate that GVC1D attains superior compression efficiency, where it achieves bitrate reductions of 60.4\% under LPIPS and 68.8\% under DISTS on the HEVC Class B dataset, surpassing the previous video compression methods.Project: https://gvc1d.github.io/
