Search papers, labs, and topics across Lattice.
This paper introduces a novel generative framework for video compression that utilizes pre-trained Diffusion Transformer (DiT) priors to optimize perceptual quality at extremely low bitrates. By implementing a Group-of-Latents (GoL) strategy, the authors partition the latent space into intra and inter latents, allowing for efficient encoding of perceptual anchors while minimizing overhead. The proposed method achieves state-of-the-art perceptual fidelity at bitrates below 0.005 bpp, demonstrating robust temporal consistency and rich spatial details, which is a significant advancement in the field of video compression.
Achieving high perceptual quality in video compression at bitrates below 0.005 bpp could redefine the limits of efficient video transmission.
Most existing video compression algorithms follow a paradigm of transformation and quantization, optimizing the trade-off between distortion and bitrate. However, extremely low-bitrate compression remains an underexplored frontier where perceptual quality optimization under severely constrained coding resources has not been adequately addressed. In this paper, we propose a unified generative framework that leverages pre-trained Diffusion Transformer (DiT) priors to achieve high perceptual quality at extremely low bitrates. We first introduce a flexible Group-of-Latents (GoL) strategy within the latent space of a causal tokenizer, explicitly partitioning the latent stream into intra $I$-latents and inter $P$-latents. The Deep Compression Module (I-DCM) then encodes key $I$-latents to preserve perceptual anchors with minimal overhead. Building upon these anchors, the DiT-based Unified Latent Denoising Module (U-LDM) refines intra-frame textures and synthesizes $P$-latents from noise, reconstructing temporal dynamics at zero additional bitrate cost. Extensive experiments demonstrate that our method uniquely operates in the extreme-low-bitrate regime (e.g., (<0.005) bpp), achieving state-of-the-art perceptual fidelity with rich spatial details and robust temporal consistency. The code will be made publicly available.
