Accurate in space, unreliable in time: how LLMs represent national cultural change

2026-09-01

AI Summary

研究使用世界价值观调查数据，评估了四种最新大语言模型在表示国家文化变迁方面的准确性，发现模型虽能大致定位但存在时间滞后等问题。

Abstract

Assessments of cultural alignment have become an important part of the development and improvement of large language models (LLMs). However, the majority of the evaluations treat culture as a single snapshot, investigating only whether a model represents a society accurately at the current time. Research in cultural psychology shows that cultural values change at different rates and directions over time. Therefore, a "culturally aware" model should capture not only where a culture is today but also how it has changed over time. We examine this missing dimension of cultural awareness using more than two decades of the World Values Survey data. We compare the cultural trajectories of 40 countries with the trajectories produced by four state-of-the-art (SOTA) LLMs on the Inglehart-Welzel cultural map. Our findings show that while models generally place countries close to their most recent surveyed positions, these representations tend to lag several years behind that position. They also capture only part of the magnitude of the observed change, introduce movement where little occurred, and rarely reproduce reversals in countries' trajectories. These findings point to temporal flattening and suggest that snapshot accuracy can give an incomplete picture of cultural awareness in LLMs and have implications for model evaluation, representational harms, and the governance of culturally aware AI systems.

Problem

• cultural change
• large language models
• temporal flattening
• cultural awareness
• snapshot accuracy

Innovation

• cultural change
• temporal flattening
• cultural awareness
• large language models (LLMs)