Search papers, labs, and topics across Lattice.
This paper introduces the Ethical Decision Head (EDH), a reinforcement learning framework that integrates ethical reasoning into the decision-making processes of autonomous vehicles. By employing Proximal Policy Optimization (PPO) and a Bradley-Terry reward model based on human preferences, the EDH is trained to navigate collision-imminent scenarios under two normative ethical frameworks: Utilitarianism and Kantian ethics. The findings reveal a significant divergence between theoretical ethical principles and the moral behaviors that humans actually reward, highlighting that RLHF captures ethics as humans practice it rather than as philosophers define it.
Human preferences in ethical decision-making for autonomous vehicles reveal a troubling preference for self-sacrifice over minimizing casualties, challenging traditional ethical frameworks.
As autonomous vehicles (AVs) approach Level 4 and Level 5 operational capability [SAE International, 2018], their on- board decision systems must handle not only safety-critical locomotion but also their subsequent moral weight. This paper details the Ethical Decision Head (EDH), a deep re- inforcement learning (RL) framework that encodes ethical reasoning as a differentiable reward signal, enabling a pol- icy gradient agent to learn morally-aligned driving behavior in scenarios whose state representation is aligned with the CARLA simulation environment [Dosovitskiy et al., 2017]. Two normative frameworks are instantiated and evaluated: a Utilitarian framework minimizing total casualties and a Kan- tian framework enforcing course maintenance as a categori- cal imperative. The EDH is trained via Proximal Policy Op- timization (PPO) [Schulman et al., 2017] against a Bradley- Terry reward model [Bradley and Terry, 1952] learned from pairwise human preference annotations over 200 collision- imminent scenarios. Results reveal an asymmetry in the learnability of normative ethical frameworks under human su- pervision. The Kantian condition, which reduces to a con- stant prediction task under the codebook, serves as a pipeline control: it confirms training stability and rules out infrastruc- ture failure as an explanation for the utilitarian result. The Utilitarian agent learned something more unsettling: human raters rewarded self-sacrifice over casualty minimization, and the model learned that preference faithfully. This divergence between what humans prescribe in theory and what they re- ward in practice suggests that RLHF does not learn ethics as philosophers define it, but as humans live it.
