Search papers, labs, and topics across Lattice.
This study investigates the ability of large-language models (LLMs) to replicate cultural tastes by generating 277,470 silicon surrogates based on the Survey of Public Participation in the Arts (SPPA). The findings reveal that these silicon surrogates exhibit a systematic positive bias towards liking, leading to inflated ecological estimates of cultural preferences, while failing to capture the complex relational structures of real human tastes. Additionally, the research highlights significant distortions in how age, class, gender, and race are associated with cultural tastes, raising concerns about the reliability of LLM-generated survey data in market research contexts.
Silicon surrogates from LLMs not only misrepresent cultural tastes but also inflate positive biases, undermining the validity of synthetic survey panels.
Large-language models have proven to be remarkable if inconsistent parrots of public attitudes and opinions. The extent to which LLMs are able to produce reasonable approximations of cultural taste remains an open empirical question that becomes more urgent by the day, with market research companies already offering provisional `synthetic' survey panels and the contamination of standard survey data from LLM-generated responses. In this study, we build on past work on silicon sampling by extending considerations of its algorithmic fidelity and alignment to the domain of cultural consumption. We use large-language models from OpenAI, Anthropic, and DeepSeek to each produce 277,470 (30x9249) silicon surrogates of survey respondents from the Survey of Public Participation in the Arts (SPPA). We find these silicon surrogates' tastes to be highly stylized facsimiles of human tastes. (1) Silicon samples have a systematic postive-bias for liking, resulting in inflated ecological estimates of tastes. The individual-level bias of silicon samples are not well-explained by the WEIRD-bias often discussed in the literature. (2) The complex relationality in real taste structures is completely lost among silicon samples. (3) Finally, very little of the known cultural alignment between tastes and social space are preserved. Silicon samples attenuate age-taste associations, resurrect anachronistic class-taste associations, caricaturize gender- and race-taste associations.
