Search papers, labs, and topics across Lattice.
This paper introduces In-Context Region-based Drag (ICRDrag), a novel method for region-based drag editing in diffusion models, addressing the limitations of traditional point-based approaches. By leveraging in-context learning and implementing two innovative attention regularization techniques鈥攊mage-mask attention consistency and source-target attention correspondence鈥擨CRDrag enhances the accuracy and visual fidelity of image editing tasks. Extensive experiments demonstrate that ICRDrag outperforms existing methods, as evidenced by both quantitative metrics and user studies, highlighting its effectiveness in producing high-quality dragged images.
ICRDrag achieves superior editing accuracy and visual fidelity by redefining region-based drag in diffusion models, leaving point-based methods in the dust.
Diffusion models have shown promise in drag-style editing. Previous works mainly focus on point-based drag, which is inherently ambiguous. This paper focuses on region-based drag and introduces a novel In-Context Region-based Drag (ICRDrag) method. Under the in-context learning framework, ICRDrag consumes a source image, a source region mask, and a target region mask, producing the target dragged image. Built upon the basic in-context learning model, we introduce two novel attention regularization: 1) image-mask attention consistency to ensure that a target region attends to similar source regions for image and mask modalities; 2) source-target attention correspondence to ensure the mutual correspondence between source and target regions. To facilitate region-based drag, we also construct Paired Region Dataset (PRD), a large-scale dataset with paired masks and images. Extensive experiments show that ICRDrag significantly outperforms existing methods in both quantitative metrics and user studies, achieving superior editing accuracy and visual fidelity. The dataset, code, and model are available at https://github.com/bcmi/ICRDrag-Region-Drag-Editing.
