Clustering Discourses: Racial Biases in Short Stories about Women Generated by Large Language Models

2025-09-02

AI Summary

This study investigates the implicit racialized narrative mechanisms underlying LLaMA 3.2-3B’s generation of short stories featuring Black and white Portuguese women, addressing how ostensibly neutral large language models (LLMs) reproduce colonial gender–race frameworks. Method: We propose a mixed-methods approach integrating computational semantic clustering (applied to 2,100 generated texts) with critical discourse analysis—bridging limitations of purely quantitative or qualitative paradigms. Contribution/Results: We identify three dominant discursive patterns—“social transcendence,” “ancestral mythologization,” and “subjective self-actualization”—which systematically differentiate the model’s construction of agency and historical positioning for Black versus white women. Empirical findings confirm that, even in the absence of explicit bias prompts, the model reproduces structural inequities. The study delivers a transferable methodological framework and empirically grounded benchmark for AI fairness evaluation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Philosophy and Ethics of AI: Bias, Fairness & Equity

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

This study investigates how large language models, in particular LLaMA 3.2-3B, construct narratives about Black and white women in short stories generated in Portuguese. From 2100 texts, we applied computational methods to group semantically similar stories, allowing a selection for qualitative analysis. Three main discursive representations emerge: social overcoming, ancestral mythification and subjective self-realization. The analysis uncovers how grammatically coherent, seemingly neutral texts materialize a crystallized, colonially structured framing of the female body, reinforcing historical inequalities. The study proposes an integrated approach, that combines machine learning techniques with qualitative, manual discourse analysis.

Problem

• Investigates racial biases in LLM-generated stories
• Analyzes narrative construction about Black and white women
• Examines how neutral texts reinforce historical inequalities

Innovation

• Used computational methods to group stories
• Combined machine learning with qualitative analysis
• Analyzed racial biases in generated narratives