Search papers, labs, and topics across Lattice.
This study investigates the bias of large language models (LLMs) towards Standard American English (SAE) over African American English (AAE), revealing that these models systematically rewrite AAE into SAE despite the context. The authors introduce an innovative auditing framework using conditional Dialect Group Invariance (cDGI) to isolate model bias and identify specific AAE markers that trigger this bias, particularly highlighting negative concord as a universal trigger. They also propose a novel method called activation steering, which effectively mitigates this bias at test time, achieving a reduction of 5 to 20 times more than traditional prompting methods while maintaining fluency in SAE.
LLMs are not just misinterpreting AAE; they鈥檙e rewriting it into SAE, but a new method can significantly reduce this bias without sacrificing fluency.
African American English (AAE), a rule-governed dialect spoken by over 30 million people, is routinely misinterpreted and"corrected"by large language models (LLMs). Across six instruction-tuned LLMs (14B to 70B), we show that state-of-the-art models systematically prefer Standard American English (SAE) continuations even when the preceding context is in AAE, effectively rewriting AAE into SAE. We present an end-to-end framework to audit and mitigate this bias. For auditing, we introduce conditional Dialect Group Invariance (cDGI), which isolates true model bias from translator-induced artifacts, and a feature-level localization analysis that identifies which AAE markers most strongly trigger bias; we find that syntactic constructions, especially negative concord (e.g.,"ain't nobody"), are universal triggers across all models. For mitigation, we introduce, to our knowledge, the first application of activation steering to dialect bias: a training-free, test-time method that extracts dialect directions via causal tracing and injects them into bias-relevant layers. Activation steering reduces bias 5 to 20 times more than prompting while preserving SAE fluency. To enable this work, we release REAL-AAE , the largest real-AAE parallel corpus to date: 17,479 AAE/SAE/ AAE_back triplets from natural tweets (2 to 6 times larger than prior real-AAE resources), validated automatically (BERTScore F1 = 0.95) and by three native AAE speakers (83.0% semantic agreement).
