Search papers, labs, and topics across Lattice.
This paper introduces a novel data preprocessing algorithm that integrates counterfactual fairness into reinforcement learning (RL) by employing a quantile distribution mapping method for estimating counterfactual states and rewards. The approach addresses the critical issue of systematic bias in RL decisions, particularly in high-stakes applications like healthcare, where equitable access to services is paramount. The authors provide theoretical bounds on counterfactual unfairness and suboptimality, and validate their method through empirical tests on both synthetic and real-world datasets.
Counterfactual fairness in reinforcement learning can be achieved with a new preprocessing algorithm that ensures equitable access in high-stakes decision-making.
Reinforcement learning (RL) seeks to optimize sequential decisions to maximize population-level benefits over time. However, when deployed in high-stakes settings such as healthcare, RL decisions might systematically restrict some subpopulation's access to valuable services in a manner contrary to the values and goals of stakeholders. Counterfactual fairness (CF) offers a promising framework to address this problem based on causal reasoning. This paper develops a data preprocessing algorithm that, when used in tandem with policy learning, enables CF in RL. Our algorithm relies on a novel quantile distribution mapping method for sequentially estimating the counterfactual states and rewards in the data preprocessing step, subsuming common additivity assumptions used for counterfactual prediction as a special case. We theoretically prove that the per-step level of counterfactual unfairness and infinite-horizon suboptimality gap can be bounded under mild regularity conditions. We also empirically test our algorithm in numerical experiments as well as in application to a real-world interventional digital health dataset.
