Search papers, labs, and topics across Lattice.
This paper introduces Population Aligned Language Models (PALMs), which are tailored to reflect the diverse values, beliefs, and cultural norms of specific populations across five countries. By utilizing construct-grounded rationales as latent supervision during preference tuning, PALMs achieve an average of 8.59% improvement over existing culture-specialized models in capturing population-specific preferences. The models not only excel in aligning with user behavior but also demonstrate strong generalization across various downstream applications, outperforming baselines in personalized reward modeling and social reasoning tasks.
Grounding preference learning in psychological and cultural constructs yields a significant boost in population alignment for LLMs, outperforming traditional demographic methods.
Large language models are being extensively used to simulate individual user behavior, yet faithfully representing a population requires capturing the systematic variation in values, beliefs, and cultural norms that distinguish one group from another. We introduce Population Aligned Language Models (PALMs), a suite of models each aligned to specific populations, covering five countries: USA, India, Brazil, France and Italy. PALMs are created by synthesizing rationales grounded in psychological and cultural constructs and using these as latent supervision during preference tuning for population-specific alignment. Evaluated across four dimensions: personality, values and beliefs, cultural norms, and morality, PALMs consistently outperform baselines, including culture-specialized models, achieving an average of 8.59% relative improvement over the best baseline across all five populations. Notably, construct-grounded rationales outperform both demographic prompting and survey-based fine-tuning, suggesting that grounding preference learning in psychology and culture provides a richer inductive signal than surface-level response distributions. We further demonstrate strong generalization to downstream applications with- out task-specific supervision: outperforming best baselines by 5.19% in personalized reward modeling, 6.34% in population simulation, and showing strong transfer to social reasoning tasks. Datasets and code are available at: https://github.com/limenlp/PALMs.
