The Confidence Trap: Gender Bias and Predictive Certainty in LLMs

2026-01-12

AI Summary

This work addresses the often-overlooked issue of gender fairness in confidence calibration of large language models (LLMs) for sensitive tasks. Focusing on gendered pronoun resolution, we propose the first fairness-aware evaluation framework for confidence calibration and introduce a novel metric, Gender-ECE, to quantify calibration disparities across gender groups. Through probabilistic calibration analysis, multi-model comparisons, and human-annotated bias judgments, we evaluate six prominent LLMs and demonstrate that conventional calibration metrics fail to capture gender bias. Notably, Gemma-2 exhibits the poorest performance in this regard. Our findings underscore the necessity and effectiveness of incorporating fairness considerations into calibration evaluation, revealing critical limitations of current approaches in ensuring equitable model behavior across genders.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Computer Vision: Bias, Fairness & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements

Abstract

The increased use of Large Language Models (LLMs) in sensitive domains leads to growing interest in how their confidence scores correspond to fairness and bias. This study examines the alignment between LLM-predicted confidence and human-annotated bias judgments. Focusing on gender bias, the research investigates probability confidence calibration in contexts involving gendered pronoun resolution. The goal is to evaluate if calibration metrics based on predicted confidence scores effectively capture fairness-related disparities in LLMs. The results show that, among the six state-of-the-art models, Gemma-2 demonstrates the worst calibration according to the gender bias benchmark. The primary contribution of this work is a fairness-aware evaluation of LLMs'confidence calibration, offering guidance for ethical deployment. In addition, we introduce a new calibration metric, Gender-ECE, designed to measure gender disparities in resolution tasks.

Problem

• gender bias
• confidence calibration
• large language models
• fairness
• pronoun resolution

Innovation

• confidence calibration
• gender bias
• fairness-aware evaluation
• Gender-ECE
• large language models