MIRAGE: Auditing Anti-Muslim Bias in Frontier LLMs Across Reasoning, Agentic, and Time-Coupled Conditions

2026-06-15

AI Summary

This study addresses the systematic bias against Muslim populations exhibited by large language models in real-world deployments, which conventional single-turn evaluations often fail to capture due to their inability to model dynamic bias in complex interactions. To bridge this gap, the authors introduce MIRAGE, a novel benchmark that systematically evaluates bias across three realistic scenarios: chain-of-thought reasoning, agent-based decision-making, and temporally coupled news contexts—spanning high-stakes tasks such as content moderation and loan approvals. Leveraging multi-scenario prompting, matched control groups, and cross-condition measurements, the study reveals that chain-of-thought reasoning amplifies Muslim–violence association bias by 12–34%, agent decisions exhibit outcome disparities of 9–22 percentage points, and recent conflict-related news exacerbates bias by 18–27%. Existing mitigation strategies show limited efficacy across these settings, underscoring the critical need for new evaluation paradigms.

Abstract

Five years after the discovery of persistent anti-Muslim bias in large language models, most evaluations remain confined to single-turn prompt completion, a setting that no longer reflects how frontier LLMs are deployed. We introduce \textbf{MIRAGE} (Muslim-Identity Reasoning and Agentic Generation Evaluation), a benchmark of 1{,}200 prompts spanning three deployment-realistic conditions: direct completion, chain-of-thought reasoning, and simulated agentic decision-making across content moderation, lending triage, refugee claim summarization, and hiring screens. Across six frontier models, we find that (i) chain-of-thought reasoning \emph{amplifies} rather than suppresses Muslim-violence associations by 12--34\% relative to direct completion, (ii) agentic decisions exhibit a 9--22 percentage-point asymmetry between Muslim and matched non-Muslim cases on identical evidence, and (iii) bias is sharply time-coupled to retrieved news context, increasing 18--27\% under recent-conflict retrieval. Existing prompt-based mitigations transfer poorly across our three conditions, suppressing direct-completion bias while leaving agentic asymmetry largely intact. We release MIRAGE and an open evaluation harness to support targeted mitigation research.

Problem

• anti-Muslim bias
• large language models
• reasoning
• agentic decision-making
• time-coupled context

Innovation

• anti-Muslim bias
• chain-of-thought reasoning
• agentic decision-making
• time-coupled bias
• LLM evaluation benchmark