Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
This paper addresses the challenge of synthesizing policies for finite populations of MDP agents under aggregate reach-avoid chance constraints, moving beyond expectation-based mean-field approximations. By propagating the second-order moment of the empirical density and employing the Cantelli inequality, the authors convert stochastic chance constraints into tractable moment-based deterministic conditions. This approach enables gradient-based policy synthesis with rigorous finite-$N$ certificates, outperforming standard deterministic baselines in evaluated scenarios.
Finite populations of MDP agents can now be controlled under stochastic reach-avoid chance constraints by directly managing the variance of their empirical density.
Consider a finite population of agents with decoupled Markov transition dynamics and empirical-density feedback, subject to the following constraints: with probability at least $1-δ_r$, at least a fraction $α_r$ of agents must reach a target region at some time $t^*$, while, at each time up to $t^*$, the unsafe population fraction must remain below $β_u$ with probability at least $1-δ_u$. However, standard mean-field methods enforce these constraints only in expectation, which fails to account for stochastic fluctuations at finite fleet size $N$. To address this control problem, we propagate the second-order moment (variance) of the empirical density alongside the mean-field trajectory via a discrete-time Lyapunov recursion, and apply the Cantelli inequality to convert chance constraints into tractable deterministic conditions on the moments of the empirical density. We then incorporate these moment-based surrogate constraints into a gradient-based sequential convex approximation procedure for density-feedback policy synthesis. We further introduce additional moment-error bounds to construct a rigorous finite-$N$ certificate. The method is evaluated on a gridworld environment and a power-system EV-charging aggregation problem and compared with a standard deterministic population-level LP baseline.