Analysis Of Linguistic Stereotypes in Single and Multi-Agent Generative AI Architectures

2026-03-19

AI Summary

This study addresses systematic dialect-based biases in large language models (LLMs) when processing Standard American English (SAE) versus African American English (AAE), particularly in the assignment of occupations, names, and adjectives. The authors construct eight prompt templates to systematically compare, for the first time, single-agent approaches—including role prompting and chain-of-thought reasoning—with multi-agent architectures featuring generate-critique-revise workflows in mitigating such biases. Using an LLM-as-judge evaluation paradigm across multiple mainstream models, the experiments reveal significant SAE–AAE disparities in all tested models, with Claude Haiku exhibiting the largest bias and Phi-4 Mini the smallest. Chain-of-thought reasoning effectively reduces bias in Haiku, while the multi-agent framework consistently mitigates bias across all models, underscoring the importance of workflow-level interventions for ensuring fairness.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Large Multimodal Models (LMMs)
• Philosophy and Ethics of AI: Bias, Fairness & Equity

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Many works in the literature show that LLM outputs exhibit discriminatory behaviour, triggering stereotype-based inferences based on the dialect in which the inputs are written. This bias has been shown to be particularly pronounced when the same inputs are provided to LLMs in Standard American English (SAE) and African-American English (AAE). In this paper, we replicate existing analyses of dialect-sensitive stereotype generation in LLM outputs and investigate the effects of mitigation strategies, including prompt engineering (role-based and Chain-Of-Thought prompting) and multi-agent architectures composed of generate-critique-revise models. We define eight prompt templates to analyse different ways in which dialect bias can manifest, such as suggested names, jobs, and adjectives for SAE or AAE speakers. We use an LLM-as-judge approach to evaluate the bias in the results. Our results show that stereotype-bearing differences emerge between SAE- and AAE-related outputs across all template categories, with the strongest effects observed in adjective and job attribution. Baseline disparities vary substantially by model, with the largest SAE-AAE differential observed in Claude Haiku and the smallest in Phi-4 Mini. Chain-Of-Thought prompting proved to be an effective mitigation strategy for Claude Haiku, whereas the use of a multi-agent architecture ensured consistent mitigation across all the models. These findings suggest that for intersectionality-informed software engineering, fairness evaluation should include model-specific validation of mitigation strategies, and workflow-level controls (e.g., agentic architectures involving critique models) in high-impact LLM deployments. The current results are exploratory in nature and limited in scope, but can lead to extensions and replications by increasing the dataset size and applying the procedure to different languages or dialects.

Problem

• linguistic stereotypes
• dialect bias
• large language models
• discriminatory behavior
• fairness in AI

Innovation

• dialect bias
• multi-agent architecture
• Chain-of-Thought prompting
• LLM-as-judge
• stereotype mitigation