Search papers, labs, and topics across Lattice.
Microsoft ResearchCommunication University of ChinaUniversity of Science and TechnologyApr 14, 2026arXiv:2604.12525This paper investigates lightweight diffusion models for real-time image compression, finding that compression-oriented pre-training outperforms generation-oriented pre-training at small model scales. They also demonstrate that lightweight convolutions, when combined with distillation, can replace transformers without sacrificing performance in compression tasks. The resulting one-step convolution diffusion codec achieves real-time encoding/decoding at 1080p and reduces bitrate by 85% compared to MS-ILLM while maintaining comparable FID.
Real-time, lightweight image compression is now possible with diffusion models, thanks to a novel architecture that swaps transformers for convolutions and prioritizes compression-focused pre-training.
Recent advanced diffusion methods typically derive strong generative priors by scaling diffusion transformers. However, scaling fails to generalize when adapted for real-time compression scenarios that demand lightweight models. In this paper, we explore the design of real-time and lightweight diffusion codecs by addressing two pivotal questions. First, does diffusion pre-training benefit lightweight diffusion codecs? Through systematic analysis, we find that generation-oriented pre-training is less effective at small model scales whereas compression-oriented pre-training yields consistently better performance. Second, are transformers essential? We find that while global attention is crucial for standard generation, lightweight convolutions suffice for compression-oriented diffusion when paired with distillation. Guided by these findings, we establish a one-step lightweight convolution diffusion codec that achieves real-time $60$~FPS encoding and $42$~FPS decoding at 1080p. Further enhanced by distillation and adversarial learning, the proposed codec reduces bitrate by 85\% at a comparable FID to MS-ILLM, bridging the gap between generative compression and practical real-time deployment. Codes are released at https://github.com/microsoft/GenCodec/CoD_Lite
