Vision-Language Models Generate More Homogeneous Stories for Phenotypically Black Individuals

2024-12-12

AI Summary

This study investigates whether vision-language models (VLMs) exhibit “phenotype-driven homogenization bias” when generating narratives about Black Americans—i.e., whether racial phenotypic cues (e.g., skin tone, facial features) systematically induce stereotyped, less diverse storytelling, particularly exacerbated along gendered intersections. We construct a controlled image dataset with systematically varied phenotypic attributes and evaluate narrative generation across multiple VLMs (LLaVA, Qwen-VL, Fuyu), quantifying output diversity via text similarity metrics. Our empirical analysis reveals, for the first time, that individuals with higher-phenotype Blackness receive significantly more homogeneous narratives; this effect is markedly stronger for Black women, whose narrative diversity suffers disproportionately greater suppression than men’s—and in two models, the suppression operates exclusively on female-identified subjects. These findings establish “intersectional phenotypic bias” as a novel paradigm in AI fairness research.

Technology Category

• Computer Vision: Bias, Fairness & Privacy
• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Vision-Language Models (VLMs) extend Large Language Models' capabilities by integrating image processing, but concerns persist about their potential to reproduce and amplify human biases. While research has documented how these models perpetuate stereotypes across demographic groups, most work has focused on between-group biases rather than within-group differences. This study investigates homogeneity bias-the tendency to portray groups as more uniform than they are-within Black Americans, examining how perceived racial phenotypicality influences VLMs' outputs. Using computer-generated images that systematically vary in phenotypicality, we prompted VLMs to generate stories about these individuals and measured text similarity to assess content homogeneity. Our findings reveal three key patterns: First, VLMs generate significantly more homogeneous stories about Black individuals with higher phenotypicality compared to those with lower phenotypicality. Second, stories about Black women consistently display greater homogeneity than those about Black men across all models tested. Third, in two of three VLMs, this homogeneity bias is primarily driven by a pronounced interaction where phenotypicality strongly influences content variation for Black women but has minimal impact for Black men. These results demonstrate how intersectionality shapes AI-generated representations and highlight the persistence of stereotyping that mirror documented biases in human perception, where increased racial phenotypicality leads to greater stereotyping and less individualized representation.

Problem

• Investigates homogeneity bias in Vision-Language Models (VLMs)
• Examines how racial phenotypicality affects story generation
• Reveals greater stereotyping in AI outputs for Black individuals

Innovation

• VLMs generate stories using phenotypicality-based images
• Text similarity measures assess story homogeneity
• Intersectionality influences AI-generated content variation