Out of Sight Out of Mind, Out of Sight Out of Mind: Measuring Bias in Language Models Against Overlooked Marginalized Groups in Regional Contexts

2025-04-17

AI Summary

This study systematically uncovers offensive stereotypical biases in large language models (LLMs) toward 270 regional marginalized groups across Egypt, 21 Arab countries, Germany, the UK, and the US, as well as toward low-resource dialects such as Egyptian Arabic. Method: We introduce a cross-lingual prompting framework, a dialect-adapted evaluation protocol, and a multidimensional intersectional bias metric—enabling the first large-scale, cross-national detection of bias against regional minorities. Contribution/Results: We find that dialectal representations significantly amplify bias—e.g., bias intensity in Egyptian Arabic exceeds that in Modern Standard Arabic by a large margin. Arabic-language LLMs exhibit high religious and ethnic bias against both dominant and marginalized groups. Non-binary, LGBTQIA+, and Black women face heightened intersectional bias. These findings expose critical limitations in existing bias measurement paradigms and establish a novel, dialect-aware framework for fairness assessment in low-resource languages.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Large Multimodal Models (LMMs)
• Philosophy and Ethics of AI: Bias, Fairness & Equity

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

We know that language models (LMs) form biases and stereotypes of minorities, leading to unfair treatments of members of these groups, thanks to research mainly in the US and the broader English-speaking world. As the negative behavior of these models has severe consequences for society and individuals, industry and academia are actively developing methods to reduce the bias in LMs. However, there are many under-represented groups and languages that have been overlooked so far. This includes marginalized groups that are specific to individual countries and regions in the English speaking and Western world, but crucially also almost all marginalized groups in the rest of the world. The UN estimates, that between 600 million to 1.2 billion people worldwide are members of marginalized groups and in need for special protection. If we want to develop inclusive LMs that work for everyone, we have to broaden our understanding to include overlooked marginalized groups and low-resource languages and dialects. In this work, we contribute to this effort with the first study investigating offensive stereotyping bias in 23 LMs for 270 marginalized groups from Egypt, the remaining 21 Arab countries, Germany, the UK, and the US. Additionally, we investigate the impact of low-resource languages and dialects on the study of bias in LMs, demonstrating the limitations of current bias metrics, as we measure significantly higher bias when using the Egyptian Arabic dialect versus Modern Standard Arabic. Our results show, LMs indeed show higher bias against many marginalized groups in comparison to dominant groups. However, this is not the case for Arabic LMs, where the bias is high against both marginalized and dominant groups in relation to religion and ethnicity. Our results also show higher intersectional bias against Non-binary, LGBTQIA+ and Black women.

Problem

• Measuring bias in LMs against overlooked marginalized groups
• Investigating bias in 23 LMs for 270 marginalized groups
• Assessing impact of low-resource languages on LM bias

Innovation

• Measuring bias in 23 LMs for 270 marginalized groups
• Investigating bias impact of low-resource languages and dialects
• Demonstrating higher intersectional bias against specific groups