Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This work introduces a hierarchical reinforcement learning framework, FAHOC, to integrate patient preferences into computational treatment planning for chronic multimorbidity by learning factored-action options that decompose joint action spaces and incorporate cooperation-aware masking. The framework enables structured exploration and improved credit assignment, leading to clinically safe and preference-consistent decision-making. Evaluation on a large patient dataset demonstrates significant improvements in quality-adjusted life year expectancy compared to observed clinician practice, with high accuracy in identifying cooperative patients and zero preference violations.
Explicitly modeling patient preferences within a hierarchical RL framework unlocks a 0.67 QALY improvement over clinician practice in chronic multimorbidity management.
Patient preference, defined as a patient's demonstrated willingness and capacity to adhere to clinical recommendations, is a primary determinant of therapeutic effect yet remains structurally absent from existing computational treatment planning models. We address this gap by presenting patient-centered factored-action hierarchical option-critic (FAHOC), a hierarchical reinforcement learning (HRL) framework that jointly learns high-level options corresponding to therapeutic strategies and factored intra-option policies that decompose the joint action space into disease- and intervention-specific subcomponents, while imposing a cooperation-aware action masking mechanism. This enables structured exploration, improved credit assignment across hierarchy levels, and more interpretable decision pathways, while enforcing patients' preferences. Formal guarantees establish that cooperative patients achieve higher optimal expected health outcomes than non-cooperative patients, and that the factored Q-function approximation error is provably bounded. The framework is evaluated using longitudinal data collected from approximately 50,000 comorbid hypertension and type 2 diabetes mellitus patients from five hospitals in the Southeast U.S. FAHOC achieves a quality-adjusted life year expectancy equivalent improvement of 0.669 (vs -0.133 observed clinician practice), correctly identifies cooperative patients in 95.9% of cases and never violates a patient's preference in held-out test, demonstrating that HRL with explicit preference constraints can support preference-consistent, clinically safe decision-making in multimorbidity management.