Search papers, labs, and topics across Lattice.
This paper introduces ResEdit, a novel approach that enhances conditional diffusion image generators for precise image editing by integrating a residual image encoding as additional conditioning. This method addresses the challenges of maintaining image identity and global consistency during edits, which are often compromised by weakly conditioned inversions that introduce conflicting features. The results demonstrate significant improvements in high-fidelity image editing, including intrinsic-based modifications and text-guided manipulation, while effectively preserving the original image identity.
Incorporating a residual encoding into image editing workflows can drastically improve both identity preservation and editability in generative models.
Conditional diffusion image generators can be repurposed for editing through inversion, without the need for large-scale paired fine-tuning data. However, producing high-quality, targeted edits while maintaining image identity and global consistency remains challenging, as weakly conditioned inversion often embeds conflicting image features into the noise. We demonstrate that incorporating a residual image encoding as additional conditioning enables both improved identity preservation and better editability. We optimize this residual encoding to provide a strong conditioning signal for reconstruction, thereby reducing the reliance on inversion and susceptibility to its aforementioned pitfalls. To ensure this residual does not interfere with desired edits, we incorporate a gradient reversal-based optimization strategy that disentangles the residual from the edited condition. We illustrate our method's ability to produce high-fidelity results across precise intrinsic-based editing and relighting, and show proof-of-concept text-guided manipulation.
