Failure of contextual invariance in gender inference with large language models

2026-03-24

AI Summary

This study demonstrates that large language models violate the assumption of contextual invariance in gender inference tasks, as their outputs are significantly influenced by irrelevant contextual cues, thereby compromising reliability and fairness. By employing a controlled pronoun selection task with minimally altered, theoretically uninformative contexts and applying Contextuality-by-Default analysis, the work reveals for the first time that models exhibit systematic shifts in predictions across nearly identical syntactic structures due solely to extraneous context. Experimental results show that in 19%–52% of cases, model behavior depends on contextual factors that cannot be attributed to repetition or boundary effects. Moreover, culturally ingrained gender-stereotype associations markedly diminish or vanish under contextual conditions, challenging foundational assumptions of current evaluation paradigms.

Technology Category

• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Standard evaluation practices assume that large language model (LLM) outputs are stable under contextually equivalent formulations of a task. Here, we test this assumption in the setting of gender inference. Using a controlled pronoun selection task, we introduce minimal, theoretically uninformative discourse context and find that this induces large, systematic shifts in model outputs. Correlations with cultural gender stereotypes, present in decontextualized settings, weaken or disappear once context is introduced, while theoretically irrelevant features, such as the gender of a pronoun for an unrelated referent, become the most informative predictors of model behaviour. A Contextuality-by-Default analysis reveals that, in 19--52\% of cases across models, this dependence persists after accounting for all marginal effects of context on individual outputs and cannot be attributed to simple pronoun repetition. These findings show that LLM outputs violate contextual invariance even under near-identical syntactic formulations, with implications for bias benchmarking and deployment in high-stakes settings.

Problem

• contextual invariance
• gender inference
• large language models
• bias evaluation
• pronoun selection

Innovation

• contextual invariance
• gender inference
• large language models
• Contextuality-by-Default
• bias evaluation