Search papers, labs, and topics across Lattice.
This paper explores the feasibility of conducting reinforcement learning (RL) for image editing without relying on specific editing rewards by leveraging the existing reward ecosystem from text-to-image (T2I) generation. The authors introduce Lever-Edit, a two-stage framework that creates a reward-aligned captioner for generating counterfactual target descriptions, which are then used to optimize the editing policy based solely on T2I rewards. Experimental results demonstrate that this approach achieves competitive performance in editing alignment and source preservation compared to traditional methods that depend on editing-specific rewards.
Lever-Edit shows that you can effectively optimize image editing policies using T2I rewards, bypassing the need for costly editing-specific rewards altogether.
Reinforcement learning (RL) enables direct preference optimization for image editing through editing-specific rewards, which remain less developed due to costly triplet supervision and complex task-dependent calibration. In contrast, text-to-image (T2I) generation benefits from a mature and diverse reward ecosystem spanning semantic alignment, aesthetics, realism, glyph shape, and other visual preferences. Extending this ecosystem to image editing would substantially broaden the range of visual preferences accessible to RL-based optimization, prompting the central question: \emph{Can We Perform Image Editing RL without Editing Rewards?} In this paper, we argue that the standard image editing dimensions have potential to be mapped to the T2I reward space: image quality can transfer directly, prompt following can be aligned through a description of the desired visual state, and reference consistency admits a coarse semantic conversion by encoding the source content to preserve. However, editing instructions specify relative changes, whereas T2I rewards require self-contained target descriptions; moreover, semantically valid captions from generic vision-language models may be incompatible with the frozen reward. Hence, we further introduce Lever-Edit, a two-stage framework that learns a reward-aligned captioner for counterfactual target descriptions, freezes it, and optimizes the editing policy solely with the transferred T2I reward. Experiments show competitive editing alignment and source preservation against editing-reward-based fine-tuning, while outperforming intuitive transfer baselines.
