Search papers, labs, and topics across Lattice.
This study investigates the sensitivity of large language model (LLM)-based stance simulations to variations in conversational context, employing counterfactual context revision as an auditing framework. By inferring a target user's stance and applying both text-only and multimodal revision strategies, the authors assess the effectiveness of these methods through metrics like average directional stance shift and stance transition rate. The findings indicate that both strategies yield robust stance transitions, underscoring the dual potential for LLMs to accurately simulate opinion dynamics while also revealing inherent risks in context sensitivity.
LLMs can robustly shift simulated stances in online discussions, but their sensitivity to context changes raises critical questions about reliability in opinion dynamics.
Large language models are increasingly used to simulate social media users and infer how individuals may respond to online discussions. However, it remains unclear whether these simulations reflect precise user-specific beliefs or whether they are highly sensitive to semantically independent changes in conversational contexts. In this work, we study counterfactual context revision as a framework for auditing LLM-based stance simulation. Given an original online conversation, we first infer a target user's stance toward a specific topic. We then apply controlled revision strategies to the conversational context and simulate the user's stance again under the revised context. We compare text-only revision strategies with a multimodal one that incorporates meme-based context and evaluate two main effectiveness metrics, i.e., average directional stance shift and stance transition rate. The results reveal effective and robust stance transitions in both text-only and multimodal strategies across different polarization-preference mechanisms. Our study contributes an evaluation framework for understanding the context sensitivity of LLM-based stance simulation. More broadly, it highlights both the promise and risk of using LLMs to simulate online opinion dynamics.
