Search papers, labs, and topics across Lattice.
This study introduces the MoralAltDataset, comprising 307 moral dilemmas that include both original options and alternative compromises, to investigate how large language models (LLMs) and humans respond to these reframed scenarios. The research reveals that when presented with compromise alternatives, LLMs significantly shift their moral judgments, often favoring these alternatives over the original binary choices. Additionally, LLM-generated alternatives are found to be preferred over human-authored ones based on structural and ethical criteria, highlighting the models' potential to enhance moral reasoning beyond traditional binary frameworks.
LLMs often prefer compromise solutions over binary choices, reshaping our understanding of moral decision-making in AI.
As large language models (LLMs) are increasingly deployed as moral advisors and agents, they need to address dilemmas between two competing values. However, existing research on LLMs with moral dilemmas overlooks a central aspect of human moral cognition: the ability to imagine alternatives that move beyond the given options. We introduce MoralAltDataset, a dataset of 307 moral dilemmas spanning narrative Advisor dilemmas and AI-facing Agent dilemmas, each augmented with compromise and reframed alternatives. We first examine whether humans and LLMs shift their judgments when such alternatives are introduced. Across 15 LLMs, we find that compromise alternatives are often preferred over either original option, substantially reshaping moral choice. We then evaluate the quality of LLM-generated alternatives against human-authored ones using pairwise preference and expert-based criteria. Results show that LLM-generated alternatives are often preferred and better satisfy fine-grained structural and ethical criteria, while revealing trade-offs between structural quality and practical feasibility.
AI2