Search papers, labs, and topics across Lattice.
This study investigates the endorsement of persuasion-infused messages by six large language models (LLMs) across different geographic regions, focusing on the impact of partisan persona prompting. The results reveal that LLMs typically do not endorse persuasive messages when prompted neutrally, but endorsement increases significantly with left- or right-leaning political cues, highlighting model-level differences and variations based on persuasion techniques and topics. These findings underscore the risks associated with deploying LLMs in politically sensitive contexts, as they may not reliably simulate human political behavior due to their susceptibility to bias and polarization.
Partisan prompts can dramatically skew LLMs' endorsement of persuasive messages, revealing potential biases that threaten their reliability in political contexts.
Large Language Models (LLMs) are increasingly employed as proxies for human behavior in computational social science. However, their tendency to internalize biases from training data raises concerns about their reliability in politically sensitive domains, specifically in regard to their susceptibility to persuasive language. In this work, we examine whether LLMs endorse persuasion-infused messages and whether partisan persona prompting modulates such endorsement. We evaluate six LLMs from different geographic regions on content annotated with persuasion techniques drawn from real-world media sources, measuring the likelihood of endorsement using a five-point Likert scale. The models are prompted as either a neutral social media user or as a user with left- or right-leaning political views. Results show that without political conditioning, LLMs generally do not endorse messages containing persuasion techniques, though model-level differences emerge, and that partisan persona prompting increases polarization of endorsement, particularly for persuasion-infused content. Endorsement further varies by persuasion technique and topic. These findings raise concerns about agentic LLM deployments in politically sensitive environments and complicate their use as reliable simulators of human political cognition.
