Search papers, labs, and topics across Lattice.
This paper introduces Flow-Map GRPO, an online reinforcement learning framework designed to enhance deterministic few-step flow-map generators by incorporating stochastic elements without altering their original parameterization. The method utilizes Anchored Stochastic Flow Map Composition (ASFMC) to introduce randomness through anchor-based conditional resampling, allowing for effective optimization with RL techniques. Experimental results show significant improvements in performance metrics for text-to-image generation tasks, indicating that deterministic models can be effectively aligned with RL post-training strategies.
Flow-Map GRPO transforms deterministic flow-map generators into powerful stochastic models, boosting their performance in generative tasks without retraining.
Few-step flow-map generators, such as consistency models and MeanFlow, accelerate sampling by directly learning long-range transport maps between noise and data. However, these models are typically deterministic, which makes them difficult to optimize with reinforcement learning (RL) post-training methods that require stochastic trajectories and well-defined likelihood ratios. Existing SDE-based stochasticization techniques are designed for velocity-based samplers with infinitesimal or finely discretized transitions, and therefore do not directly apply to long-range flow maps. In this work, we propose Flow-Map GRPO, an online RL post-training framework for deterministic few-step flow-map generators. The key component is Anchored Stochastic Flow Map Composition (ASFMC), a path-preserving stochasticization mechanism that introduces randomness through anchor-based conditional resampling while preserving the original marginal probability path of the deterministic flow map. We derive GRPO objectives for both single-time and two-time flow-map parameterizations. Experiments on few-step FLUX-based text-to-image generators, including MeanFlow and sCM, show that Flow-Map GRPO improves pretrained deterministic flow-map models across reward-based, perceptual, and task-level evaluation metrics. Our results demonstrate that deterministic few-step flow-map generators can be effectively aligned with RL post-training without modifying their original model parameterization or retraining them as native stochastic models.
