Discriminatory Compliance: How LLMs Answer Queries from Protected Groups

2026-06-19

AI Summary

This study addresses a critical yet underexplored issue in large language models (LLMs): the emergence of “discriminatory compliance,” wherein safety mechanisms disproportionately filter or withhold essential information when responding to queries from users belonging to protected or marginalized groups. The work introduces this concept and systematically evaluates response consistency across multiple mainstream LLMs through role-based prompting experiments that vary user identity attributes and query phrasings. Findings reveal a pervasive tendency among current models to provide insufficient or incomplete information to minority group members, with significant inconsistencies observed across model providers, identity dimensions, and linguistic formulations. These results expose systemic biases embedded within existing safety alignment frameworks, highlighting an urgent need for more equitable and context-sensitive moderation strategies.

Abstract

Chatbots developed using Large Language Models (LLMs) implement various safeguards for sensitive questions and/or scenarios. These safeguards require making certain assumptions about the person asking the question. We define discriminatory compliance as patterns in question answering that disproportionately disadvantage users from minority or protected backgrounds, for instance by omitting information that would be valuable for them. In this paper, we show that state-of-the-art LLMs respond inconsistently to questions from personas from protected identity groups, and that some of these inconsistencies mean that key information that should be provided to minority or protected background personas is missing. We show that this behavior is, additionally, inconsistent across and within model providers as well as across background conditions and ways of phrasing those conditions.

Problem

• discriminatory compliance
• Large Language Models
• protected groups
• information omission
• algorithmic bias

Innovation

• Discriminatory Compliance
• Large Language Models
• Protected Groups
• Algorithmic Fairness
• AI Safeguards