Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper critiques the current ethical frameworks for AI systems, arguing they fail to address the "illusion of opting," where individuals appear to have meaningful choices but lack the agency to genuinely choose. It proposes that AI systems should be evaluated on their ability to protect and cultivate "meta-capacity," the agentive capacity to form, contest, and revise means and ends. The authors introduce three normative imperatives—existential honesty, ecological rationality, and counterfactual reparation—to guide the development of AI systems that support genuine choice, particularly for disadvantaged populations.
AI's ethical problem isn't just bias, but the "illusion of opting" – the deceptive appearance of choice that weakens our ability to form our own goals.
Drawing on Ullmann-Margalit's concept of opting (transformative, irrevocable, and shadowed by foreclosed alternatives), we show that current AI systems raise a profound ethical problem that existing AI ethics has not fully captured: the illusion of opting, in which persons and groups encounter the deceptive appearance of meaningful consequential choice while the agency needed to become genuinely capable of choosing is weakened. Against approaches that treat AI primarily as an optimizer of already given ends, we argue that AI systems should be evaluated by whether they protect and cultivate meta-capacity against the illusion of opting: the socially and institutionally scaffolded agentive capacity through which means and ends can be formed, contested, revised, and owned. This reframing is especially urgent for disadvantaged populations, who are least able to absorb the costs of the illusion of opting when AI-mediated pathways misdirect behavior and action. We propose three normative imperatives for AI-mediated consequential decisions: existential honesty, which acknowledges the limits of prediction; ecological rationality, which situates guidance within heterogeneous lived ecologies; and counterfactual reparation, which acknowledges and repairs foreclosed alternatives when AI-mediated decision-making pathways fail.