Search papers, labs, and topics across Lattice.
This paper addresses the limited diversity in images generated by text-to-image models for the same prompt, which can lead to demographic skew, particularly in person-centric prompts. The authors introduce a novel reinforcement learning objective, multi-axis max@K, that enhances target-mode coverage by assigning positive weights to samples that improve the maximum score across predefined categories. Experimental results demonstrate that this approach significantly improves fairness scores by 0.23-0.36 without compromising image quality or text alignment.
Multi-axis max@K boosts image diversity and fairness in text-to-image generation, enhancing representation without sacrificing quality.
Text-to-image (T2I) models can synthesize realistic, prompt-aligned images, yet samples generated for the same prompt often cover only a small subset of visually distinct modes. This limits the diversity of images, and for person-centric prompts, can reflect or amplify demographic skew. We formalize this problem as coverage of a predefined set of semantically specified modes, which we call target-mode coverage. We then propose multi-axis max@K, a group-based reinforcement learning objective for improving such coverage in diffusion-based T2I models. Given a group of samples and one score per target category, multi-axis max@K first takes the maximum score across samples for each category and then sums these category-wise maxima. The resulting credit assignment gives a sample positive weight on a category only when it increases that category's group-wise maximum, allowing different samples to contribute to different categories. We first validate the credit-assignment mechanism on a synthetic mixture and on SD3.5-M using deterministic pixel-based color rewards. We then evaluate the same objective on perceived-appearance fairness. Across three automatic evaluators on held-out prompts, multi-axis max@K improves the Fairness Score by 0.23-0.36 relative to the base model, while maintaining image quality and text alignment.
