Search papers, labs, and topics across Lattice.
This paper investigates the ability of Large Language Models (LLMs) to infer political alignment from online text, leveraging data from Debate.org and Reddit. They found that LLMs significantly outperform traditional machine learning models in predicting political leaning based on seemingly innocuous linguistic cues within online discussions. The study further demonstrates that aggregating text-level inferences and utilizing politics-adjacent domains enhances prediction accuracy, revealing the models' capacity to exploit subtle socio-cultural correlations.
LLMs can guess your political affiliation with surprising accuracy just by reading your online chatter, even when you're not explicitly talking politics.
Due to the correlational structure in our traits such as identities, cultures, and political attitudes, seemingly innocuous preferences such as following a band or using a specific slang, can reveal private traits. This possibility, especially when combined with massive, public social data and advanced computational methods, poses a fundamental privacy risk. Given our increasing data exposure online and the rapid advancement of AI are increasing the misuse potential of such risk, it is therefore critical to understand capacity of large language models (LLMs) to exploit it. Here, using online discussions on Debate.org and Reddit, we show that LLMs can reliably infer hidden political alignment, significantly outperforming traditional machine learning models. Prediction accuracy further improves as we aggregate multiple text-level inferences into a user-level prediction, and as we use more politics-adjacent domains. We demonstrate that LLMs leverage the words that can be highly predictive of political alignment while not being explicitly political. Our findings underscore the capacity and risks of LLMs for exploiting socio-cultural correlates.
