Exploring Language Patterns of Prompts in Text-to-Image Generation and Their Impact on Visual Diversity

2025-04-19

AI Summary

User prompt language homogenization constitutes a critical human-factor mechanism constraining diversity in text-to-image (TTI) generation. Method: Analyzing over 6 million real-world user prompts, we quantified linguistic homogeneity using Sentence-BERT semantic similarity and LDA topic modeling, and measured visual output diversity via the Vendi Score—a first-time application of this metric to TTI human factors evaluation. Contribution/Results: We empirically demonstrate that 40–50% of prompts are high-frequency repetitions, yielding significantly lower visual diversity; moreover, linguistic homogenization intensifies with user activity, while semantic topic distributions remain stable. We propose a novel “linguistic experimentalism” framework for user stratification, revealing that user interaction behavior itself functions as a key sociotechnical moderator. These findings establish actionable, human-in-the-loop optimization pathways—centered on prompting practices—to enhance generative diversity in TTI systems.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Humans and AI: User Experience and Usability
• Computer Vision: Language and Vision

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Following the initial excitement, Text-to-Image (TTI) models are now being examined more critically. While much of the discourse has focused on biases and stereotypes embedded in large-scale training datasets, the sociotechnical dynamics of user interactions with these models remain underexplored. This study examines the linguistic and semantic choices users make when crafting prompts and how these choices influence the diversity of generated outputs. Analyzing over six million prompts from the Civiverse dataset on the CivitAI platform across seven months, we categorize users into three groups based on their levels of linguistic experimentation: consistent repeaters, occasional repeaters, and non-repeaters. Our findings reveal that as user participation grows over time, prompt language becomes increasingly homogenized through the adoption of popular community tags and descriptors, with repeated prompts comprising 40-50% of submissions. At the same time, semantic similarity and topic preferences remain relatively stable, emphasizing common subjects and surface aesthetics. Using Vendi scores to quantify visual diversity, we demonstrate a clear correlation between lexical similarity in prompts and the visual similarity of generated images, showing that linguistic repetition reinforces less diverse representations. These findings highlight the significant role of user-driven factors in shaping AI-generated imagery, beyond inherent model biases, and underscore the need for tools and practices that encourage greater linguistic and thematic experimentation within TTI systems to foster more inclusive and diverse AI-generated content.

Problem

• Analyzes how user prompt choices affect TTI output diversity
• Examines linguistic homogenization in prompts and its visual impact
• Highlights need for tools to encourage diverse prompt experimentation

Innovation

• Analyzing user prompts to understand visual diversity
• Categorizing users by linguistic experimentation levels
• Using Vendi scores to link lexical and visual similarity