We're Different, We're the Same: Creative Homogeneity Across LLMs

2025-01-31

AI Summary

This study investigates whether large language models (LLMs) exhibit cross-model creative homogenization when employed as creative assistants. Method: We systematically compare response distributions from 12 state-of-the-art LLMs and human participants on standardized divergent thinking tasks (e.g., the Alternate Uses Task), employing embedding-based similarity analysis, structural control experiments, and quantitative diversity metrics. Contribution/Results: We provide the first systematic evidence that LLMs display significant cross-model homogeneity in creative generation—inter-LLM response similarity substantially exceeds inter-human similarity (p < 0.001). This homogeneity stems from shared generative mechanisms inherent to LLM architectures, rather than idiosyncratic model deficiencies. Consequently, the finding challenges the common heuristic that “swapping models improves creative diversity” and reveals a systemic risk of creative narrowing in LLM-augmented ideation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Numerous powerful large language models (LLMs) are now available for use as writing support tools, idea generators, and beyond. Although these LLMs are marketed as helpful creative assistants, several works have shown that using an LLM as a creative partner results in a narrower set of creative outputs. However, these studies only consider the effects of interacting with a single LLM, begging the question of whether such narrowed creativity stems from using a particular LLM -- which arguably has a limited range of outputs -- or from using LLMs in general as creative assistants. To study this question, we elicit creative responses from humans and a broad set of LLMs using standardized creativity tests and compare the population-level diversity of responses. We find that LLM responses are much more similar to other LLM responses than human responses are to each other, even after controlling for response structure and other key variables. This finding of significant homogeneity in creative outputs across the LLMs we evaluate adds a new dimension to the ongoing conversation about creativity and LLMs. If today's LLMs behave similarly, using them as a creative partners -- regardless of the model used -- may drive all users towards a limited set of"creative"outputs.

Problem

• Large Language Models
• Creative Diversity
• Convergence

Innovation

• Large Language Models
• Creativity Homogeneity
• Population-Level Diversity