Search papers, labs, and topics across Lattice.
This paper presents UltraDiffEdit, a tuning-free framework that enhances latent diffusion models for ultrahigh-resolution image editing, addressing the limitations of existing methods that struggle with resolutions above 1K. By employing a multiscale progressive editing strategy and multipatch encoding, UltraDiffEdit effectively blends edited and unedited areas while preserving fine details, achieving high-quality results even at 8K resolution. Extensive experiments reveal that UltraDiffEdit outperforms traditional methods in editing quality and flexibility, all while operating on a single NVIDIA GeForce RTX 3090 GPU.
UltraDiffEdit achieves stunning 8K image editing without the need for extensive tuning, bridging the gap between generative models and real-world photographic demands.
Recent diffusion-based generative models have shown impressive performance in image generation and editing. However, due to memory limitations and the high cost of collecting high-resolution training images, existing methods are typically restricted to inputs with linear resolutions below 1K. On the contrary, photographs captured by modern mobile devices often reach linear resolutions up to 8K, revealing a significant gap between current capabilities and real-world demands. Simply upscaling low-resolution edited results often results in visually enlarged but blurry images that lack fine details. This article introduces UltraDiffEdit, a novel, tuning-free image editing framework that extends off-the-shelf latent diffusion models (LDMs) to ultrahigh resolutions. UltraDiffEdit employs a multiscale progressive editing strategy, iteratively blending high-resolution edited content with unedited areas in a coarse-to-fine manner. We employ multipatch encoding to preserve both edited and unedited visual details within the latent space. To mitigate editing artifacts, our global-local consistency denoising technique consistently integrates edited and unedited latent features, ensuring smooth transition at editing boundaries from the latent representation to the final image. We also introduce a patch-based hybrid sampling approach that captures local, intermediate, and global features, ensuring semantic coherence and enhancing fine detail during denoising. We conduct extensive experiments demonstrating UltraDiffEdit's superior editing quality and flexibility: it can handle image resolutions up to 8K using only a single NVIDIA GeForce RTX 3090 GPU.
