Search papers, labs, and topics across Lattice.
This paper addresses the issue of diversity collapse in pretrained flow models, which typically generate less varied outputs when given the same input. The authors propose a novel self-guidance mechanism that disperses internal features during batch generation while maintaining alignment with input conditions through manifold regularization. Their approach not only enhances diversity significantly across various conditional flow models but also incurs minimal additional inference cost, making it a practical solution for real-world applications.
A training-free self-guidance method can dramatically enhance output diversity in flow models without the overhead of external reward systems.
State-of-the-art flow models generate stunning images from text or image prompts. However, they suffer from diversity collapse when generating multiple samples under the same conditioning. Existing methods address this issue via either latent guidance, which has limited effectiveness, or sample selection, which relies on external reward models that incur significant inference-time overhead. In this work, we introduce an efficient, training-free self-guidance mechanism to mitigate diversity collapse without requiring additional reward models. Specifically, we disperse the internal features of the flow model during batch generation with feature self-guidance. Further, to keep the features close to the manifold, we introduce a manifold regularization step that projects these dispersed features back onto the data manifold, ensuring diverse generation without sacrificing alignment with the input conditions. Our method integrates seamlessly as a plug-and-play module into pretrained flow models, adding only a marginal inference cost. Experiments demonstrate significant improvements in diversity while preserving fidelity across several conditional flow models, including multi-step and few-step text-to-image, depth-to-image, and reference image generation.
Stanford HAI