Search papers, labs, and topics across Lattice.
This paper introduces Content Adaptive Diffusion-based Generative Image Compression (CADC), a novel approach to improve diffusion-based image compression at ultra-low bitrates by making the compression process content-adaptive. CADC addresses limitations in existing methods by introducing uncertainty-guided adaptive quantization, auxiliary decoder-guided information concentration, and bitrate-free adaptive textual conditioning. Experiments demonstrate that CADC achieves improved reconstruction quality and compression efficiency compared to existing diffusion-based image compression techniques.
Achieve better image compression without sacrificing quality by adapting diffusion models to image content using uncertainty-guided quantization and bitrate-free textual conditioning.
Diffusion-based generative image compression has demonstrated remarkable potential for achieving realistic reconstruction at ultra-low bitrates. The key to unlocking this potential lies in making the entire compression process content-adaptive, ensuring that the encoder's representation and the decoder's generative prior are dynamically aligned with the semantic and structural characteristics of the input image. However, existing methods suffer from three critical limitations that prevent effective content adaptation. First, isotropic quantization applies a uniform quantization step, failing to adapt to the spatially varying complexity of image content and creating a misalignment with the diffusion model's noise-dependent prior. Second, the information concentration bottleneck -- arising from the dimensional mismatch between the high-dimensional noisy latent and the diffusion decoder's fixed input -- prevents the model from adaptively preserving essential semantic information in the primary channels. Third, existing textual conditioning strategies either need significant textual bitrate overhead or rely on generic, content-agnostic textual prompts, thereby failing to provide adaptive semantic guidance efficiently. To overcome these limitations, we propose a content-adaptive diffusion-based image codec with three technical innovations: 1) an Uncertainty-Guided Adaptive Quantization method that learns spatial uncertainty maps to adaptively align quantization distortion with content characteristics; 2) an Auxiliary Decoder-Guided Information Concentration method that uses a lightweight auxiliary decoder to enforce content-aware information preservation in the primary latent channels; and 3) a Bitrate-Free Adaptive Textual Conditioning method that derives content-aware textual descriptions from the auxiliary reconstructed image, enabling semantic guidance without bitrate cost.
