Search papers, labs, and topics across Lattice.
This paper introduces PixSGR, a novel pixel diffusion framework that enhances image generation by implementing Sparse Global Refinement to overcome the limitations of existing models that either sacrifice detail or restrict refinement to individual patches. By utilizing a low-channel bottleneck to capture the low-dimensional manifold of natural images, PixSGR progressively increases channel dimensionality and spatial resolution, allowing for more detailed structural recovery. The model demonstrates significant improvements in image fidelity, achieving an FID of 1.51 at 256脳256 and maintaining competitive performance at 512脳512 with an FID of 1.60, showcasing its efficiency and effectiveness in high-fidelity image generation.
PixSGR achieves unprecedented image fidelity by enabling non-local refinement in pixel diffusion without the computational burden of dense attention.
Pixel-space diffusion has recently emerged as a promising direction for high-fidelity image generation by modeling images directly in the original pixel domain. However, pixel-space diffusion is computationally demanding due to the extremely high dimensionality of natural images. For efficiency, existing pixel diffusion models either compromise fine details with large-patch tokenization or confine subsequent refinement within individual patches. Such intra-patch refinement inevitably restricts structural continuity across patch boundaries and long-range token interactions, limiting refinement quality. To address these issues, we propose PixSGR, a novel Pixel diffusion framework with Sparse Global Refinement tailored for modeling the distribution of natural images directly in pixel space. PixSGR starts from a supervised low-channel bottleneck to efficiently capture the low-dimensional manifold of natural images. It then progressively expands the channel dimensionality and spatial resolution to recover increasingly fine-grained structures. At the spatial refinement stage, coarse-scale attention maps preselect globally relevant interactions to pre-sparsify fine-scale attention, enabling non-local refinement beyond isolated patches without the quadratic cost of dense attention. Extensive experiments on ImageNet validate the effectiveness of PixSGR. It achieves an FID of 1.51 at 256$\times$256 and maintains performance when scaled to 512$\times$512, attaining an FID of 1.60.
