Not-quite-human tastes: the stylized omnivorousness of LLM survey surrogates

2026-06-29

AI Summary

This study investigates whether large language models (LLMs) can effectively simulate the authentic structure of human cultural taste and evaluates their reliability as synthetic survey respondents. Introducing silicon-based sampling to cultural taste research for the first time, the authors generated 277,470 synthetic respondents using LLMs from OpenAI, Anthropic, and DeepSeek to replicate data from arts participation surveys, conducting multidimensional comparisons with real-world samples. The findings reveal that LLM-generated responses exhibit a pronounced positive bias in preferences, fail to capture the nuanced interdependencies inherent in genuine taste structures, and inaccurately reproduce cultural taste patterns shaped by age, social class, gender, and race. These results highlight systematic limitations in current LLMs’ capacity to model sociocultural attitudes with fidelity.

Abstract

Large-language models have proven to be remarkable if inconsistent parrots of public attitudes and opinions. The extent to which LLMs are able to produce reasonable approximations of cultural taste remains an open empirical question that becomes more urgent by the day, with market research companies already offering provisional `synthetic' survey panels and the contamination of standard survey data from LLM-generated responses. In this study, we build on past work on silicon sampling by extending considerations of its algorithmic fidelity and alignment to the domain of cultural consumption. We use large-language models from OpenAI, Anthropic, and DeepSeek to each produce 277,470 (30x9249) silicon surrogates of survey respondents from the Survey of Public Participation in the Arts (SPPA). We find these silicon surrogates' tastes to be highly stylized facsimiles of human tastes. (1) Silicon samples have a systematic postive-bias for liking, resulting in inflated ecological estimates of tastes. The individual-level bias of silicon samples are not well-explained by the WEIRD-bias often discussed in the literature. (2) The complex relationality in real taste structures is completely lost among silicon samples. (3) Finally, very little of the known cultural alignment between tastes and social space are preserved. Silicon samples attenuate age-taste associations, resurrect anachronistic class-taste associations, caricaturize gender- and race-taste associations.

Problem

• large language models
• cultural taste
• survey surrogates
• silicon sampling
• taste structure

Innovation

• silicon sampling
• cultural taste
• large language models
• survey surrogates
• algorithmic bias