Search papers, labs, and topics across Lattice.
This study introduces INCLUDE-BENCH, a comprehensive benchmark designed to evaluate disability-related biases in text-to-image (T2I) models, addressing a significant gap in the assessment of social biases. By generating 119K images across various prompts and contexts, the research reveals that T2I models predominantly depict mobility-impaired individuals as wheelchair users, exhibit reduced diversity in disability portrayals, and align closely with real-world stereotypes. The findings underscore the urgent need for more nuanced evaluation practices that reflect sociological definitions of stereotyping, highlighting the limitations of current methodologies in assessing representational harms toward people with disabilities.
T2I models overwhelmingly depict disability through the lens of stereotypes, with wheelchair imagery dominating representations of mobility impairment.
Text-to-image (T2I) models have been shown to exhibit social biases. Prior work has mainly focused on gender, skin tone, and cultural representation within restricted occupational associations, and emerging benchmarks increasingly incorporate these dimensions. However, disability remains systematically underexplored. Current evaluation practices often fail to align with sociologically grounded definitions of stereotyping, limiting principled assessment of representational harms toward people with disabilities (PWD). To address this, we introduce INCLUDE-BENCH, the first large-scale benchmark for evaluating disability-related bias in T2I models. INCLUDE-BENCH comprises 119K generated images based on prompt design across multiple bias dimensions and both static and dynamic contexts. We evaluate 15 open-source and 2 closed-source models. Our key findings reveal that: (1) mobility-impaired and default disability prompts predominantly yield wheelchair depictions across all models; (2) disability-conditioned generations consistently exhibit less diversity; (3) stereotypical portrayals demonstrate stronger disability-text alignment; and (4) we introduce the Stereotype Content Model (SCM) Score, demonstrating that T2I models reflect real-world stereotypical associations.
