Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Persona-Conditioned Informativeness (PCI), an unsupervised metric that quantifies the informative variation in responses generated by persona-conditioned large language models (LLMs). By modeling personas as a similarity graph and employing Local Moran's I, the authors demonstrate that semantically similar personas can yield concordant response shifts, enhancing the reliability of survey simulations. The evaluation reveals that a PCI-selected subset of responses significantly improves construct recovery in comparison to traditional methods, underscoring its utility in refining synthetic respondent screening in survey methodologies.
Persona conditioning in LLMs can be reliably assessed, revealing that semantically similar personas can yield meaningful response variations that enhance survey accuracy.
Persona-conditioned large language models (LLMs) are increasingly used to simulate survey responses across diverse domains. However, apparent response variation can reflect unconditioned model priors or token sampling noise rather than systematic persona conditioning. We argue that persona-conditioned variation is informative when semantically similar personas exhibit concordant response shifts. To operationalize this principle, we introduce Persona-Conditioned Informativeness (PCI), an unsupervised diagnostic metric that measures whether semantically similar personas deviate in concordant directions relative to item-level sample baselines. By modeling personas as a similarity graph, PCI uses Local Moran's I to quantify local spatial coherence and extract compact persona subsets without using construct labels. To evaluate PCI without external human benchmarks, we test its ability to recover established latent value structure using the 57-item Portrait Values Questionnaire-Revised (PVQ-RR). Confirmatory factor analysis (CFA) shows that a PCI-selected 10% subset substantially improves overall construct recovery relative to response-stability and random selection. These findings support PCI as a principled internal diagnostic for screening synthetic respondents in survey pipelines.