Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a novel linear programming (LP) formulation that encodes robust policy iteration for Markov decision processes (RMDPs) with rational polyhedral uncertainty in rewards and transitions. The constructed LP's optimal solutions yield both the robust optimal value and all optimal stationary randomized policies, with polynomial dimension and encoding length achievable in strongly polynomial time at fixed discount. Furthermore, a generalized complexity analysis of robust policy iteration leads to improved and new strongly polynomial bounds for various RMDP uncertainty models and turn-based stochastic games.
Robust optimal policies and values for RMDPs are now computable in strongly polynomial time via a single LP, unifying and extending prior results across diverse uncertainty models.
We study linear programming (LP) representations and strongly polynomial algorithms for robust Markov decision processes (RMDPs) with rational polyhedral state-action rectangular uncertainty in rewards and transitions. By encoding a finite sequence of robust policy-iteration steps, we construct a single LP whose optimal solutions recover the robust optimal value and all optimal stationary randomized policies. At fixed discount, the LP has polynomial dimension and encoding length and can be constructed in strongly polynomial time. We also develop a general complexity analysis of robust policy iteration that combines the cost of minimizing over uncertainty sets with the number of iterations needed to evaluate a policy. For a fixed discount factor, we use this analysis to improve the known complexity bounds for $\ell_1$ and $\ell_\infty$ RMDPs and establish new strongly polynomial bounds for general interval, weighted $\ell_1$, and Wasserstein RMDPs, as well as turn-based stochastic games with these uncertainty sets.
Stanford HAI