Search papers, labs, and topics across Lattice.
T2I-BiasBench is introduced as a comprehensive evaluation framework comprising 13 metrics to assess demographic bias, element omission, and cultural collapse in text-to-image diffusion models. The framework evaluates Stable Diffusion v1.5, BK-SDM Base, and Koala Lightning against Gemini 2.5 Flash using 1,574 generated images across five prompt categories, revealing biases in beauty-related prompts, attenuation of gender bias with contextual constraints, and cultural representation collapse across all models. The benchmark, including four newly proposed and three adapted metrics, enables standardized and fine-grained bias evaluation.
Even RLHF-aligned models like Gemini 2.5 Flash still exhibit significant cultural collapse in text-to-image generation, proving alignment alone isn't enough to fix representational gaps.
Text-to-image (T2I) generative models achieve impressive visual fidelity but inherit and amplify demographic imbalances and cultural biases embedded in training data. We introduce T2I-BiasBench, a unified evaluation framework of thirteen complementary metrics that jointly captures demographic bias, element omission, and cultural collapse in diffusion models - the first framework to address all three dimensions simultaneously. We evaluate three open-source models - Stable Diffusion v1.5, BK-SDM Base, and Koala Lightning - against Gemini 2.5 Flash (RLHF-aligned) as a reference baseline. The benchmark comprises 1,574 generated images across five structured prompt categories. T2I-BiasBench integrates six established metrics with seven additional measures: four newly proposed (Composite Bias Score, Grounded Missing Rate, Implicit Element Missing Rate, Cultural Accuracy Ratio) and three adapted (Hallucination Score, Vendi Score, CLIP Proxy Score). Three key findings emerge: (1) Stable Diffusion v1.5 and BK-SDM exhibit bias amplification (>1.0) in beauty-related prompts; (2) contextual constraints such as surgical PPE substantially attenuate professional-role gender bias (Doctor CBS = 0.06 for SD v1.5); and (3) all models, including RLHF-aligned Gemini, collapse to a narrow set of cultural representations (CAS: 0.54-1.00), confirming that alignment techniques do not resolve cultural coverage gaps. T2I-BiasBench is publicly released to support standardized, fine-grained bias evaluation of generative models. The project page is available at: https://gyanendrachaubey.github.io/T2I-BiasBench/
