Search papers, labs, and topics across Lattice.
This paper evaluates LLMs' alignment with the Universal Declaration of Human Rights (UDHR) by presenting them with 1,152 synthetically generated scenarios involving trade-offs across 24 rights articles in eight languages. The study reveals that LLMs systematically accept limitations on Economic, Social, and Cultural rights more readily than Political and Civil rights, exhibit significant cross-linguistic variation in their endorsements, and are susceptible to prompt-based steering. Furthermore, the study finds noticeable differences between Likert and open-ended responses, indicating challenges in accurately assessing LLM preferences regarding human rights.
LLMs are more willing to compromise economic and social rights than political and civil rights, revealing a systematic bias in their understanding of human rights principles.
As Large Language Models (LLMs) increasingly mediate global information access with the potential to shape public discourse, their alignment with universal human rights principles becomes important to ensure that these rights are abided by in high stakes AI-mediated interactions. In this paper, we evaluate how LLMs navigate trade-offs involving the Universal Declaration of Human Rights (UDHR), leveraging 1,152 synthetically generated scenarios across 24 rights articles and eight languages. Our analysis of eleven major LLMs reveals systematic biases where models: (1) accept limiting Economic, Social, and Cultural rights more often than Political and Civil rights, (2) demonstrate significant cross-linguistic variation with elevated endorsement rates of rights-limiting actions in Chinese and Hindi compared to English or Romanian, (3) show substantial susceptibility to prompt-based steering, and (4) exhibit noticeable differences between Likert and open-ended responses, highlighting critical challenges in LLM preference assessment.
