Search papers, labs, and topics across Lattice.
This study evaluates the temporal accuracy of large language models (LLMs) in representing national cultural changes by analyzing over two decades of World Values Survey data. The researchers found that while LLMs accurately place countries in their current cultural context, they often lag behind actual cultural shifts, misrepresent the magnitude of changes, and fail to capture reversals in cultural trajectories. These findings highlight the limitations of snapshot evaluations in assessing cultural awareness and suggest significant implications for model evaluation and governance in AI systems.
LLMs may accurately reflect current cultural positions, but they lag years behind in capturing the dynamic nature of cultural change.
Assessments of cultural alignment have become an important part of the development and improvement of large language models (LLMs). However, the majority of the evaluations treat culture as a single snapshot, investigating only whether a model represents a society accurately at the current time. Research in cultural psychology shows that cultural values change at different rates and directions over time. Therefore, a"culturally aware"model should capture not only where a culture is today but also how it has changed over time. We examine this missing dimension of cultural awareness using more than two decades of the World Values Survey data. We compare the cultural trajectories of 40 countries with the trajectories produced by four state-of-the-art (SOTA) LLMs on the Inglehart-Welzel cultural map. Our findings show that while models generally place countries close to their most recent surveyed positions, these representations tend to lag several years behind that position. They also capture only part of the magnitude of the observed change, introduce movement where little occurred, and rarely reproduce reversals in countries'trajectories. These findings point to temporal flattening and suggest that snapshot accuracy can give an incomplete picture of cultural awareness in LLMs and have implications for model evaluation, representational harms, and the governance of culturally aware AI systems.
