Search papers, labs, and topics across Lattice.
This study critiques the limitations of traditional evaluation metrics in offline reinforcement learning for ICU sepsis management, highlighting the issue of Toxic Mimicry where agents may replicate harmful clinical behaviors. Utilizing the MIMIC-III database, the authors introduce the Counterfactual Clinical Audit (CCA) framework to rigorously assess RL agents against established clinical guidelines. The results show that while the Medical Decision Transformer (MedDT) fails by reducing vasopressor dosage inappropriately, the Historical Causal Transformer (HCT-RL) adheres to safe clinical practices, underscoring the need for more robust evaluation methods in medical RL.
Counterfactual audits reveal that a leading RL model dangerously contradicts clinical guidelines, risking patient safety in ICU settings.
Offline reinforcement learning (RL) offers considerable promise for optimizing ICU treatment decisions, yet standard evaluation metrics Mean Squared Error (MSE) and Fitted Q-Evaluation (FQE) assess only behavioral imitation and cannot detect Toxic Mimicry, a failure mode in which agents replicate harmful patterns such as treatment withdrawal during comfort-care transitions. Using the MIMIC-III database, we propose the Counterfactual Clinical Audit (CCA) framework, which stress-tests RL agents through physiological perturbations anchored in Surviving Sepsis Campaign (SSC) guidelines. We audit a Medical Decision Transformer (MedDT) and a Historical Causal Transformer (HCT-RL), the latter employing Causal Action Shielding, propensity-based importance weighting, and Conservative Q-Learning. CCA reveals that MedDT paradoxically reduces vasopressor dosage as lactate escalates, contradicting resuscitation guidelines, while HCT-RL maintains physiologically consistent responses. These findings expose a systemic misalignment between statistical fit and clinical safety, supporting counterfactual audits as a necessary evaluation standard for medical RL.
