Search papers, labs, and topics across Lattice.
This paper introduces Slang-Q, a curated dataset designed to evaluate language models' understanding of queer slang, consisting of user-generated sentences paired with 118 queer terms and their definitions. The authors conduct an exploratory evaluation to assess how well various language models comprehend and define these terms under different prompting conditions. The findings reveal significant gaps in the models' ability to accurately interpret queer slang, highlighting the need for more inclusive training data in NLP.
Language models struggle to accurately interpret queer slang, revealing critical gaps in their understanding of community-specific language.
Despite its cultural relevance and diffusion, queer slang remains underrepresented in Natural Language Processing research. Towards addressing this gap, we introduce Slang-Q, a manually curated dataset of naturally user-generated English sentences paired with queer slang terms and reference definitions, built upon a newly constructed taxonomy of 118 queer terms. We use this resource to conduct a first exploratory evaluation of language models on their ability to understand and define queer slang under varying prompting conditions. Slang-Q is intended as a basis for studying how current models handle sensitive, community-specific language and whether they can provide accurate and reliable information about such forms of identity and linguistic expression.
