Search papers, labs, and topics across Lattice.
This paper investigates the geometry of optimal policies in structured Markov decision processes (MDPs) and introduces boundary-based policy approximations that directly learn policy regions. By linking policy loss to action margins, the authors reveal that performance degradation is concentrated near indifference boundaries, leading to insights on error behavior. Experimental results in inventory control and queue admission demonstrate that their approach achieves lower policy error, reduced value gaps, and enhanced stability compared to traditional reinforcement learning methods.
Boundary-based policy approximations reveal that optimal policies can simplify decision-making, significantly reducing errors in structured MDPs.
We study optimal-policy geometry in structured Markov decision processes. While approximate dynamic programming and reinforcement learning typically approximate high-dimensional value functions, we show that optimal policies induce simpler decision tessellations. We propose boundary-based policy approximations that learn policy regions directly. A policy-loss decomposition links performance degradation to action margins and explains why errors concentrate near indifference boundaries. Inventory control and queue admission experiments show lower policy error, smaller value gaps, faster error decay, and stability than reinforcement learning baselines.
