Search papers, labs, and topics across Lattice.
This paper introduces audience segmentation to improve the fidelity of LLM-based social simulations, addressing the common issue of collapsed diversity. They compare six segmentation configurations across Llama 3.1-70B and Mixtral 8x22B, varying identifier granularity, parsimony, and selection logic, and evaluate performance using distributional, structural, and predictive fidelity metrics. Results indicate that moderate identifier enrichment improves performance, while data-driven selection best recovers between-group structure, highlighting the importance of heterogeneity-aware evaluation.
LLM-based social simulations can be significantly improved by segmenting audiences, but surprisingly, more granular segmentation doesn't always lead to better results, and data-driven selection of identifiers is key for recovering group structure.
Large Language Models (LLMs) are increasingly used to simulate social attitudes and behaviors, offering scalable "silicon samples" that can approximate human data. However, current simulation practice often collapses diversity into an "average persona," masking subgroup variation that is central to social reality. This study introduces audience segmentation as a systematic approach for restoring heterogeneity in LLM-based social simulation. Using U.S. climate-opinion survey data, we compare six segmentation configurations across two open-weight LLMs (Llama 3.1-70B and Mixtral 8x22B), varying segmentation identifier granularity, parsimony, and selection logic (theory-driven, data-driven, and instrument-based). We evaluate simulation performance with a three-dimensional evaluation framework covering distributional, structural, and predictive fidelity. Results show that increasing identifier granularity does not produce consistent improvement: moderate enrichment can improve performance, but further expansion does not reliably help and can worsen structural and predictive fidelity. Across parsimony comparisons, compact configurations often match or outperform more comprehensive alternatives, especially in structural and predictive fidelity, while distributional fidelity remains metric dependent. Identifier selection logic determines which fidelity dimension benefits most: instrument-based selection best preserves distributional shape, whereas data-driven selection best recovers between-group structure and identifier-outcome associations. Overall, no single configuration dominates all dimensions, and performance gains in one dimension can coincide with losses in another. These findings position audience segmentation as a core methodological approach for valid LLM-based social simulation and highlight the need for heterogeneity-aware evaluation and variance-preserving modeling strategies.
