Simulating Identity, Propagating Bias: Abstraction and Stereotypes in LLM-Generated Text

2025-09-10

AI Summary

This study investigates whether persona-prompting exacerbates societal stereotyping by modulating linguistic abstraction—specifically along concreteness, specificity, and negation. Leveraging the newly constructed Self-Stereo dataset and a language expectation bias framework, we conduct the first systematic, quantitative analysis of how eleven persona prompts affect abstraction levels in outputs from six open-source large language models (LLMs). Results show that persona-prompting fails to reliably control abstraction and instead consistently amplifies stereotypical expressions—even when simulating marginalized identities, it risks reproducing and entrenching biases. Our work uncovers a critical stereotype-amplification risk inherent in identity-simulation mechanisms within LLMs, providing empirical grounding for trustworthy prompt engineering and introducing a novel paradigm for evaluating linguistic abstraction in generative AI.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Persona-prompting is a growing strategy to steer LLMs toward simulating particular perspectives or linguistic styles through the lens of a specified identity. While this method is often used to personalize outputs, its impact on how LLMs represent social groups remains underexplored. In this paper, we investigate whether persona-prompting leads to different levels of linguistic abstraction - an established marker of stereotyping - when generating short texts linking socio-demographic categories with stereotypical or non-stereotypical attributes. Drawing on the Linguistic Expectancy Bias framework, we analyze outputs from six open-weight LLMs under three prompting conditions, comparing 11 persona-driven responses to those of a generic AI assistant. To support this analysis, we introduce Self-Stereo, a new dataset of self-reported stereotypes from Reddit. We measure abstraction through three metrics: concreteness, specificity, and negation. Our results highlight the limits of persona-prompting in modulating abstraction in language, confirming criticisms about the ecology of personas as representative of socio-demographic groups and raising concerns about the risk of propagating stereotypes even when seemingly evoking the voice of a marginalized group.

Problem

• Investigates persona-prompting effects on LLM linguistic abstraction
• Examines stereotype propagation through AI-generated identity simulations
• Measures abstraction levels in socio-demographic attribute associations

Innovation

• Persona-prompting technique for steering LLMs
• Linguistic Expectancy Bias framework analysis
• Self-Stereo dataset for stereotype measurement