Search papers, labs, and topics across Lattice.
This paper introduces a robust $Q$-learning algorithm tailored for discrete-time mean-field control problems characterized by Wasserstein uncertainty in common noise. By integrating a quantization-and-projection scheme with a dual reformulation, the authors establish convergence guarantees and finite-time iteration bounds for both synchronous and asynchronous learning. Numerical experiments demonstrate the algorithm's performance in systemic risk and epidemic models, revealing a tradeoff between robustness and performance when faced with common-noise misspecification.
Robust $Q$-learning can outperform idealized Bellman iterations in mean-field control scenarios plagued by common-noise uncertainties.
In this article, we present a robust $Q$-learning algorithm for discrete-time mean-field control problems under Wasserstein uncertainty in the common noise law. The algorithm combines a quantization-and-projection scheme with a Wasserstein dual reformulation on the common-noise space. We establish its convergence together with finite-time iteration bounds for both synchronous and asynchronous learning schemes. Numerical experiments on systemic risk and epidemic models compare the asynchronous implementation with an idealized Bellman iteration, illustrate the robustness-performance tradeoff under common-noise misspecification, and report the observed convergence behavior of the asynchronous $Q$-learning algorithm.
