Search papers, labs, and topics across Lattice.
This study evaluates the effectiveness of large language models (LLMs) as synthetic survey respondents for capturing intersectional identities by comparing simulated responses to real demographic data across 15 waves of Pew's American Trends Panel. The findings reveal that while real respondents exhibit a distinctive interplay of intersecting identities, LLMs fail to accurately simulate this complexity, often reducing multi-dimensional identities to single features. Notably, the models consistently overlook critical factors such as race and religion, which significantly influence real-world opinions, leading to a misrepresentation of intersectional perspectives in synthetic samples.
LLMs misrepresent intersectional identities by reducing complex opinions to single features, undermining their utility as synthetic survey respondents.
Large language models are increasingly used as synthetic survey respondents, promising cheap access to rare intersectional populations. We test standard demographic-persona methods against every real intersectional subgroup across 15 waves of Pew's American Trends Panel -- 21 million simulated response distributions from eight models. In real respondents, subgroup opinion is approximately the additive sum of its single-identity components, yet grows 2.5x more distinctive as identities intersect. Simulated respondents show no such composition: a single feature explains a two-feature persona's responses better than the additive combination in 75-82% of subgroups, and a third feature adds almost nothing. This collapse survives every prompting strategy we test. Additionally, the feature models retain is chosen nearly blindly -- except that they systematically discard race and religion, the strongest real drivers of opinion. Synthetic samples offer intersectional personas but represent one identity at a time.
