Back to the Communities: A Mixed-Methods and Community-Driven Evaluation of Cultural Sensitivity in Text-to-Image Models

2025-10-31

AI Summary

Text-to-image (T2I) models exhibit pervasive Western-centric biases, leading to misrepresentation and sociocultural harm toward non-Western cultural groups. To address the lack of contextualized, participatory methods for assessing cultural sensitivity, this study introduces the first community-driven, first-person participatory mixed-methods evaluation framework. It integrates co-creative workshops across multiple countries, cultural sensitivity scoring, in-depth interviews, and training-data provenance analysis to dynamically uncover the sociocultural roots and impact mechanisms of model bias. Moving beyond techno-centric paradigms, the framework centers cultural diversity, structural power inequities, and Indigenous/local knowledge systems within evaluation. Empirical application demonstrates its efficacy in identifying cross-cultural perceptual disparities, pinpointing root causes of misrepresentation, and delivering actionable governance pathways for developers and policymakers—thereby advancing T2I systems toward fairness, inclusivity, and cultural responsiveness.

Technology Category

• Computer Vision: Bias, Fairness & Privacy
• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Evidence shows that text-to-image (T2I) models disproportionately reflect Western cultural norms, amplifying misrepresentation and harms to minority groups. However, evaluating cultural sensitivity is inherently complex due to its fluid and multifaceted nature. This paper draws on a state-of-the-art review and co-creation workshops involving 59 individuals from 19 different countries. We developed and validated a mixed-methods community-based evaluation methodology to assess cultural sensitivity in T2I models, which embraces first-person methods. Quantitative scores and qualitative inquiries expose convergence and disagreement within and across communities, illuminate the downstream consequences of misrepresentation, and trace how training data shaped by unequal power relations distort depictions. Extensive assessments are constrained by high resource requirements and the dynamic nature of culture, a tension we alleviate through a context-based and iterative methodology. The paper provides actionable recommendations for stakeholders, highlighting pathways to investigate the sources, mechanisms, and impacts of cultural (mis)representation in T2I models.

Problem

• Evaluating cultural sensitivity in text-to-image models is complex
• Western cultural norms dominate text-to-image models causing misrepresentation
• Developing community-driven methods to assess cultural misrepresentation impacts

Innovation

• Mixed-methods community-driven evaluation methodology
• Co-creation workshops with diverse cultural participants
• Context-based iterative assessment of cultural sensitivity