LLMs Silently Correct African American English: Auditing and Mitigating Dialect Bias via Activation Steering

2026-07-07

AI Summary

This work addresses the systematic “silent correction” of African American English (AAE) to Standard American English (SAE) by large language models, which disregards AAE’s legitimate grammatical structures. The authors propose an end-to-end framework that first identifies model bias against AAE through conditional Dialect Group Invariance (cDGI), then mitigates this bias at test time—without retraining—by leveraging causal tracing–guided activation manipulation while preserving SAE fluency. Key contributions include the first application of activation manipulation to dialect bias mitigation, the identification of AAE syntactic features such as negative concord as universal bias triggers across models, and the creation and release of REAL-AAE, the largest real-world AAE dataset to date, comprising 17,479 triplets. Experiments demonstrate a 5–20× reduction in dialect bias, substantially outperforming baseline approaches like prompt engineering.

Abstract

African American English (AAE), a rule-governed dialect spoken by over 30 million people, is routinely misinterpreted and "corrected" by large language models (LLMs). Across six instruction-tuned LLMs (14B to 70B), we show that state-of-the-art models systematically prefer Standard American English (SAE) continuations even when the preceding context is in AAE, effectively rewriting AAE into SAE. We present an end-to-end framework to audit and mitigate this bias. For auditing, we introduce conditional Dialect Group Invariance (cDGI), which isolates true model bias from translator-induced artifacts, and a feature-level localization analysis that identifies which AAE markers most strongly trigger bias; we find that syntactic constructions, especially negative concord (e.g., "ain't nobody"), are universal triggers across all models. For mitigation, we introduce, to our knowledge, the first application of activation steering to dialect bias: a training-free, test-time method that extracts dialect directions via causal tracing and injects them into bias-relevant layers. Activation steering reduces bias 5 to 20 times more than prompting while preserving SAE fluency. To enable this work, we release REAL-AAE , the largest real-AAE parallel corpus to date: 17,479 AAE/SAE/ AAE_back triplets from natural tweets (2 to 6 times larger than prior real-AAE resources), validated automatically (BERTScore F1 = 0.95) and by three native AAE speakers (83.0% semantic agreement).

Problem

• dialect bias
• African American English
• language models
• bias mitigation
• Standard American English

Innovation

• activation steering
• dialect bias
• African American English
• conditional Dialect Group Invariance
• causal tracing