Search papers, labs, and topics across Lattice.
The paper addresses the lack of stereotype resources for evaluating generative AI model safety in Sub-Saharan Africa by creating a multilingual stereotype dataset covering Ghana, Kenya, Nigeria, and South Africa. They employ community-engaged methods, including telephonic surveys moderated in native languages, to capture socioculturally-situated stereotypes. The resulting SAFARI dataset contains 3,534 stereotypes in English and 3,206 stereotypes in 15 native languages, balanced across diverse ethnic and demographic backgrounds.
AI safety evaluations get a much-needed dose of Sub-Saharan African perspectives with the release of SAFARI, a stereotype dataset built using community-engaged methods across 15 native languages.
Stereotype repositories are critical to assess generative AI model safety, but currently lack adequate global coverage. It is imperative to prioritize targeted expansion, strategically addressing existing deficits, over merely increasing data volume. This work introduces a multilingual stereotype resource covering four sub-Saharan African countries that are severely underrepresented in NLP resources: Ghana, Kenya, Nigeria, and South Africa. By utilizing socioculturally-situated, community-engaged methods, including telephonic surveys moderated in native languages, we establish a reproducible methodology that is sensitive to the region's complex linguistic diversity and traditional orality. By deliberately balancing the sample across diverse ethnic and demographic backgrounds, we ensure broad coverage, resulting in a dataset of 3,534 stereotypes in English and 3,206 stereotypes across 15 native languages.
