Search papers, labs, and topics across Lattice.
This paper addresses the challenge of risk-averse policy evaluation in POMDPs by introducing a theoretical framework for accelerating CVaR value function evaluation with performance guarantees. The core idea involves deriving novel bounds on the CVaR of a random variable using an auxiliary random variable, leading to interpretable concentration inequalities. These bounds are then used to establish upper and lower bounds on the CVaR value function computable from a simplified belief-MDP, enabling action elimination and achieving significant computational speedups.
Safely discard suboptimal actions in POMDPs and achieve substantial computational speedups with new CVaR bounds and a particle-belief MDP framework.
Risk-averse decision-making under uncertainty in partially observable domains is a central challenge in artificial intelligence and is essential for developing reliable autonomous agents. The formal framework for such problems is the partially observable Markov decision process (POMDP), where risk sensitivity is introduced through a risk measure applied to the value function, with Conditional Value-at-Risk (CVaR) being a particularly significant criterion. However, solving POMDPs is computationally intractable in general, and approximate methods rely on computationally expensive simulations of future agent trajectories. This work introduces a theoretical framework for accelerating CVaR value function evaluation in POMDPs with formal performance guarantees. We derive new bounds on the CVaR of a random variable X using an auxiliary random variable Y, under assumptions relating their cumulative distribution and density functions; these bounds yield interpretable concentration inequalities and converge as the distributional discrepancy vanishes. Building on this, we establish upper and lower bounds on the CVaR value function computable from a simplified belief-MDP, accommodating general simplifications of the transition dynamics. We develop estimators for these bounds within a particle-belief MDP framework with probabilistic guarantees, and employ them for acceleration via action elimination: actions whose bounds indicate suboptimality under the simplified model are safely discarded while ensuring consistency with the original POMDP. Empirical evaluation across multiple POMDP domains confirms that the bounds reliably separate safe from dangerous policies while achieving substantial computational speedups under the simplified model.
