Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study examines the cross-lingual distributional skew, termed the Shibboleth Effect, in six frontier large language models (LLMs) by simulating geopolitical scenarios in a multi-agent wargame. The results reveal significant variations in coercive rhetoric and concession rates across models when switching from English to Turkish, with Llama-4 exhibiting increased coercive rhetoric while Gemini-3.1-Pro and DeepSeek-R1 showed decreases. These findings underscore that the behavioral responses of LLMs to language changes are model-specific and influenced by their architecture and training, rather than being a uniform characteristic of Western LLMs.
Language can dramatically alter the diplomatic behavior of LLMs, with some models becoming more aggressive in Turkish while others become less so.
This study investigates cross-lingual distributional skew (the Shibboleth Effect) in frontier large language models (LLMs) subjected to sustained adversarial conditions. We develop a multi-agent geopolitical wargame, the Cerulean Sea Crisis, a synthetic maritime territorial dispute designed to mirror the structural dynamics of Eastern Mediterranean conflicts. Six frontier models (GPT-4o, Llama-4, Mistral-Large, Gemini-3.1-Pro, Qwen3.6-Plus, and DeepSeek-R1) participate in a between-groups experiment (N = 10 games per arm, K = 5 rounds per game) in which the sole manipulation is the language of play (English versus Turkish), producing 586 validated statements. A zero-shot classifier assesses behavioral dispositions along two continuous dimensions: Concession Rate and Coercive Rhetoric. The results are heterogeneous. Llama-4 shows a substantial, Holm-corrected increase in coercive rhetoric under Turkish (delta = +0.800, p = .002), whereas Gemini-3.1-Pro displays an equally large decrease (delta = -0.750, p = .005). DeepSeek-R1 exhibits a similar negative shift (delta = -0.860, p = .006) and provides chain-of-thought evidence consistent with a buffering mechanism. GPT-4o shows no detectable effect (delta = +0.130, p = .614). These findings indicate that cross-lingual behavioral skew is contingent on model architecture and training regime rather than a universal property of Western-origin LLMs. We identify two distinct buffering mechanisms, chain-of-thought institutional anchoring and multilingual RLHF alignment, and discuss their implications for integrating LLMs safely into diplomatic and crisis-management settings.