Search papers, labs, and topics across Lattice.
This study evaluates the moral reasoning of large language models (LLMs) in healthcare decision-making, particularly regarding the allocation of scarce medical resources based on patients' responsibility for their health. The researchers found a significant judgment-consequence gap, where LLMs agree with human assessments of responsibility but do not allow these judgments to influence resource allocation, often defaulting to random distribution instead. Notably, LLMs also exhibit a tendency to prioritize access to information, which alters their responsibility assessments, highlighting a fundamental divergence in moral reasoning between LLMs and humans as model capability increases.
LLMs may recognize patient responsibility but shockingly refuse to let it guide resource allocation, often opting for random distribution instead.
As large language models (LLMs) enter high-stakes domains such as healthcare, understanding their moral reasoning becomes essential. Decisions about scarce medical resources often hinge on judgments of responsibility, particularly when patients'own actions contribute to illness. We investigate how LLMs reason about responsibility and its consequences, tracing their judgments across successive levels, from the behavior, to the resulting illness, to the denial of care. We evaluate a wide range of LLMs, spanning different model families and capability levels, on various clinical vignettes adapted from prior studies. Our results identify a judgment-consequence gap: LLMs largely agree with humans that patients bear responsibility for health-harming behaviors, yet overwhelmingly refuse to let that judgment influence how they allocate scarce resources. Specifically, LLMs default to random allocation, whereas humans consistently favor the less-culpable patient. Compared to humans, LLMs also place greater emphasis on access to information, reducing responsibility judgments when health-risk knowledge is unavailable. These findings reveal that LLMs apply a systematically different moral framework than humans when responsibility and resource scarcity intersect, surprisingly often amplifying normative disagreement with humans as reasoning capability increases.
