Search papers, labs, and topics across Lattice.
This paper introduces Bootstrap Your Generator (ByG), a novel framework for unpaired training of flow matching editing models that circumvents the need for extensive paired datasets. By leveraging instruction-following cues from a frozen base model and employing cycle-consistency for structure preservation, ByG achieves state-of-the-art performance in both image and video editing tasks, even in data-scarce environments. The method's innovative gradient routing from downstream losses to noisy training states effectively bridges the train-inference gap, demonstrating superior generalization to unseen domains compared to traditional supervised approaches.
Unpaired training can now rival supervised methods in image and video editing, achieving state-of-the-art results with significantly less data.
Modern generative models possess a deep understanding of visual content, yet training them for image editing typically requires massive datasets of paired examples. This limits scalability, especially for video editing where collecting paired data is prohibitively expensive. We propose Bootstrap Your Generator (ByG), a general framework for unpaired training of flow matching editing models. It leverages the base model's knowledge without any external signal. Our approach pairs instruction-following cues extracted from the frozen model with cycle-consistency for structure preservation. To make this tractable, we propose to route gradients from downstream losses over clean predictions to noisy training states. We demonstrate state-of-the-art results on challenging data-scarce image and video editing scenarios. Extensive evaluations and user studies show that our method effectively generalizes to unseen domains and outperforms supervised baselines trained on millions of samples. Analysis reveals that our gradient routing bridges the train-inference gap, and extracting semantic cues from a base model provides a robust training signal that obviates the need for external reward models.
