Track the frontier, from condensed matter to cosmology.
Search papers, labs, and topics across Lattice.
We establish an exponential iteration lower bound for Howard's policy iteration on deterministic discounted Markov decision processes with two actions per state, proving it is not strongly polynomial when the discount factor is part of the input. This result demonstrates an exponential separation from the simplex method with Dantzig's pivoting rule, which is strongly polynomial on the same class. The findings highlight a "price" of algorithmic anarchy, stemming from the decentralized nature of policy iteration's improvements compared to Dantzig's coordinated approach.
Howard's policy iteration for deterministic MDPs is not strongly polynomial, exhibiting an exponential iteration complexity that separates it from the simplex method.
We establish an exponential iteration lower bound in the number of states for Howard's policy iteration on deterministic discounted Markov decision processes, with at most two actions per state. This rules out strong polynomiality of Howard's policy iteration when the discount factor is part of the input and yields an exponential separation from the simplex method with Dantzig's pivoting rule, which is proved to be strongly polynomial on this class. Even when each reward is restricted to logarithmic bit length, we obtain a stretched-exponential iteration lower bound. The gap between Howard's decentralized and simultaneous selfish improvements and Dantzig's coordinated selection of a single action with the largest gain across all states reveals a ``price'' of algorithmic anarchy.
Stanford HAI