Prejudice and Volatility: A Statistical Framework for Measuring Social Discrimination in Large Language Models

2024-02-23

AI Summary

Existing LLM alignment evaluations often overlook stereotype randomness induced by generation inconsistency, leading to biased risk misestimation. This paper proposes the Prejudice-Volatility framework, the first to decouple discrimination risk into two quantifiable dimensions: systematic prejudice and generation volatility. Leveraging token-level probability distributions, contextual modeling, and statistical risk metrics, it enables unified, cross-model and cross-task assessment of diverse inductive biases—including knowledge and social biases. Experiments across 12 mainstream LLMs reveal: (1) prejudice dominates overall discrimination risk; (2) pervasive pro-male occupational stereotyping exists; (3) RLHF reduces prejudice but increases volatility; and (4) discrimination severity significantly correlates with socioeconomic indicators such as occupational salary. The framework thus provides a principled, fine-grained, and empirically grounded approach to bias evaluation in generative language models.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Computer Vision: Bias, Fairness & Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

This study investigates why and how inconsistency in the generation of Large Language Models (LLMs) might induce or exacerbate societal injustice. For instance, LLMs frequently exhibit contrasting gender stereotypes regarding the same career depending on varied contexts, highlighting the arguably harmful unpredictability of LLMs' behavioral patterns. To augment the existing discrimination assessment with the capability to account for variation in LLM generation, we formulate the Prejudice-Volatility Framework (PVF) that precisely defines behavioral metrics for assessing LLMs, which delineate the probability distribution of LLMs' stereotypes from the perspective of token prediction probability. Specifically, we employ a data-mining approach to approximate the possible applied contexts of LLMs and devise statistical metrics to evaluate the corresponding contextualized societal discrimination risk. Further, we mathematically dissect the aggregated discrimination risk of LLMs into prejudice risk, originating from their system bias, and volatility risk, stemming from their generation inconsistency. While initially intended for assessing discrimination in LLMs, our proposed PVF facilitates the comprehensive and flexible measurement of any inductive biases, including knowledge alongside prejudice, across various modality models. We apply PVF to 12 most commonly adopted LLMs and compare their risk levels. Our findings reveal that: i) prejudice risk is the primary cause of discrimination risk in LLMs, indicating that inherent biases in these models lead to stereotypical outputs; ii) most LLMs exhibit significant pro-male stereotypes across nearly all careers; iii) alignment with Reinforcement Learning from Human Feedback lowers discrimination by reducing prejudice, but increases volatility; iv) discrimination risk in LLMs correlates with socio-economic factors like profession salaries.

Problem

• Analyzing stereotypes in LLMs via bias and variation estimation
• Quantifying discrimination risk from bias and volatility in LLM outputs
• Assessing impact of generation inconsistencies on LLM behavior

Innovation

• Novel statistical framework for LLM stereotypes
• Bias-Volatility Framework quantifies discrimination risk
• Decomposes risk into bias and volatility components