Search papers, labs, and topics across Lattice.
Multi-View GRPO (MV-GRPO) addresses the limitations of standard Group Relative Policy Optimization (GRPO) in text-to-image flow models by augmenting the condition space to create a dense multi-view reward mapping. MV-GRPO uses a Condition Enhancer to generate semantically adjacent captions from a single prompt, enabling multi-view advantage re-estimation and capturing diverse semantic attributes. Experiments show MV-GRPO achieves superior alignment performance compared to state-of-the-art methods by incorporating the probability distribution of original samples conditioned on the new captions without sample regeneration.
Text-to-image flow models can achieve superior preference alignment by augmenting the condition space, creating a "dense" reward mapping that better captures inter-sample relationships.
Group Relative Policy Optimization (GRPO) has emerged as a powerful framework for preference alignment in text-to-image (T2I) flow models. However, we observe that the standard paradigm where evaluating a group of generated samples against a single condition suffers from insufficient exploration of inter-sample relationships, constraining both alignment efficacy and performance ceilings. To address this sparse single-view evaluation scheme, we propose Multi-View GRPO (MV-GRPO), a novel approach that enhances relationship exploration by augmenting the condition space to create a dense multi-view reward mapping. Specifically, for a group of samples generated from one prompt, MV-GRPO leverages a flexible Condition Enhancer to generate semantically adjacent yet diverse captions. These captions enable multi-view advantage re-estimation, capturing diverse semantic attributes and providing richer optimization signals. By deriving the probability distribution of the original samples conditioned on these new captions, we can incorporate them into the training process without costly sample regeneration. Extensive experiments demonstrate that MV-GRPO achieves superior alignment performance over state-of-the-art methods.
Stanford HAI