Search papers, labs, and topics across Lattice.
To decouple intrinsic ideological bias from elicitation artifacts, the authors systematically benchmarked eight Gemma 3 and Qwen 3 models across a 300-configuration perturbation space spanning eight dimensions of prompt phrasing, answer formatting, quantization, and 14 languages on the Political Compass Test. Standard political questionnaire evaluations frequently conflate model dispositions with structural benchmark flaws and coordinate drift rather than distinct cultural reasoning. Although models lean Libertarian-Left overall, apparent "centrism" in smaller models often merely reflects degenerate response collapse toward the origin caused by test axis-weighting imbalances rather than true ideological moderation.
Apparent "centrist" alignment in smaller language models often masks degenerate response collapse rather than genuine neutrality, exposing deep measurement artifacts in standard political benchmark evaluations.
Large Language Models are increasingly deployed as information intermediaries, yet measuring their political behavior remains fragile because questionnaire results mix model dispositions with measurement artifacts and response-elicitation biases. We introduce a robust Political Compass Test evaluation framework that samples 300 configurations across an eight-dimensional perturbation space varying language, framing, instructions, answer format, option order, and persona wording. We evaluate eight Gemma 3 and Qwen 3 models across 14 languages and three quantization levels, obtaining design-averaged political coordinates with quantified uncertainty. Most models lean Libertarian-Left on average, but instruction phrasing, language, and answer format significantly affect recovered coordinates. Cross-lingual differences primarily reflect coordinate drift rather than distinct cultural reasoning. Reverse-engineering the test also exposes axis-weighting imbalances and the collapse of degenerate responses toward the center, so near-origin estimates for the smallest models can reflect weak signal rather than centrism. Free-text reasoning and chat-then-classify elicitation alter recovered coordinates, and larger models show clearer persona separation, with a specific failure of the Authoritarian-Left persona to move most models in the intended social direction. In downstream tasks, persona effects are modest relative to model size and target group for hate-speech detection, while base and centrist prompts give the highest agreement for topic-level sentiment. Political role prompting therefore has measurable but task- and dataset-specific downstream effects.
