Search papers, labs, and topics across Lattice.
This paper introduces Energy-Guided Flow Matching (EG-FM), a novel approach that enhances pixel-space generative models by explicitly modeling a coarse-to-fine generative trajectory through a dynamically evolving endpoint. By utilizing a heat-kernel-filtered endpoint that transitions from low-frequency to clean images, EG-FM effectively retargets the velocity in flow matching without requiring adaptations to the backbone or training data. Experimental results demonstrate that EG-FM achieves significantly lower FID scores on the ImageNet class-conditional image generation task, outperforming previous methods in fewer training epochs and showing strong performance in text-to-image generation as well.
Achieving an FID of 1.45 in just 600 epochs, Energy-Guided Flow Matching redefines efficiency in high-quality image generation without the need for extensive model adaptations.
Pixel-space generative models bypass lossy latent compression, yet necessitate joint learning of global structure and fine-grained details in a high-dimensional space. Standard flow matching interpolates noise toward a fixed clean-image endpoint, leaving the spectral evolution to be learned implicitly. In this paper, we introduce Energy-Guided Flow Matching(EG-FM) that explicitly models a coarse-to-fine generative trajectory by moving endpoint. Specifically, EG-FM replaces the fixed endpoint with a heat-kernel-filtered endpoint that evolves smoothly from low-frequency image to clean image. The fraction of high-frequency signal in moving endpoint is released by an image-specific energy-guided scheduling, leading to the re-targeting of velocity in flow matching. Our framework requires no adaptation of the backbone and training data, bringing negligible cost on the training and inference stages. In our experiment, EG-FM consistently achieves lower FID on the ImageNet class-conditional image generation task at $256 \times 256$ with fewer epochs, reaching an FID of 1.55 at 200 epochs and 1.45 at 600 epochs. We continue training the generation task on the setting of $512 \times 512$ resolution, yielding a FID of 1.58 after only 40 high-resolution adaptation epochs. Furthermore, we transfer EG-FM on text-to-image generation and achieve 0.85 on GenEval score and 83.9 on DPG-Bench. Code is available at https://github.com/ysng123/EG-FM.
