Silicon sampling answers with country-level assumptions, not individual attitudes: Cross-national evidence from the European Social Survey

2026-09-14

AI Summary

研究使用大型语言模型模拟调查受访者，评估其在恢复跨国差异方面的有效性，发现添加国家名称可提高准确性，但个体层面恢复仍不足。

Abstract

Silicon sampling uses large language models (LLMs) to simulate survey respondents. Whether it recovers cross-national variation, and why, remains unresolved. This study evaluates it against European Social Survey Round 11 (30 countries, 42 items) with two open-weight LLMs under first- and third-person prompts, plus backstory and response-format experiments. Aggregate recovery is moderate and uneven across items. Adding the country name to a three-variable demographic backstory raises the median per-item correlation between simulated and observed country means from -0.03 to 0.52, and the richer profiles tested add no consistent gain. The respondent's country label acts as a country-level assumption that respondent detail does not revise. Naming the response-scale endpoints in words stops the model from ranking countries backwards, so the answer format sets the direction of the ranking. Individual-level recovery remains negligible in every condition and does not track aggregate recovery across countries. An average of neighboring countries, using no LLM, recovers country levels more accurately than every model condition and ranks them about as well. Silicon sampling can thus support exploratory country-ranking comparison after item-level validation and with the response format reported. It does not support individual or distributional inference.

Problem

• Silicon sampling
• cross-national variation
• large language models
• European Social Survey
• country-level assumptions

Innovation

• Silicon sampling
• country-level assumptions
• response format
• cross-national variation
• large language models (LLMs)