Search papers, labs, and topics across Lattice.
This paper tackles the over-editing problem in instruction-based image editing (IIE) by introducing a task-aware edit localization framework. The method leverages attention-based edit cues from both source and target image streams to partition tokens into edit and non-edit regions. A unified mask construction strategy then selectively uses these streams based on the specific editing task (addition, removal, replacement) to improve localization.
Task-aware localization, using attention cues from both source and target image streams, significantly reduces over-editing in instruction-based image editing, even when applied to strong diffusion transformer backbones.
Instruction-based image editing (IIE) aims to modify images according to textual instructions while preserving irrelevant content. Despite recent advances in diffusion transformers, existing methods often suffer from over-editing, introducing unintended changes to regions unrelated to the desired edit. We identify that this limitation arises from the lack of an explicit mechanism for edit localization. In particular, different editing operations (e.g., addition, removal and replacement) induce distinct spatial patterns, yet current IIE models typically treat localization in a task-agnostic manner. To address this limitation, we propose a training-free, task-aware edit localization framework that exploits the intrinsic source and target image streams within IIE models. For each image stream, We first obtain attention-based edit cues, and then construct feature centroids based on these attentive cues to partition tokens into edit and non-edit regions. Based on the observation that optimal localization is inherently task-dependent, we further introduce a unified mask construction strategy that selectively leverages source and target image streams for different editing tasks. We provide a systematic analysis for our proposed insights and approaches. Extensive experiments on EdiVal-Bench demonstrate our framework consistently improves non-edit region consistency while maintaining strong instruction-following performance on top of powerful recent image editing backbones, including Step1X-Edit and Qwen-Image-Edit.
