Search papers, labs, and topics across Lattice.
The paper introduces Recurrent Structural Policy Gradient (RSPG), a novel history-aware Hybrid Structural Method (HSM) for solving Partially Observable Mean Field Games (MFGs) with common noise. RSPG leverages known transition dynamics to reduce variance and enable learning in settings where agents have access to public information but not the full state. The method achieves state-of-the-art performance, demonstrating an order-of-magnitude faster convergence, and successfully solves a macroeconomics MFG with heterogeneous agents, common noise, and history-aware policies.
RSPG solves partially observable mean field games an order of magnitude faster than existing methods, unlocking the ability to model complex macroeconomic systems with heterogeneous agents and common noise.
Mean Field Games (MFGs) provide a principled framework for modeling interactions in large population models: at scale, population dynamics become deterministic, with uncertainty entering only through aggregate shocks, or common noise. However, algorithmic progress has been limited since model-free methods are too high variance and exact methods scale poorly. Recent Hybrid Structural Methods (HSMs) use Monte Carlo rollouts for the common noise in combination with exact estimation of the expected return, conditioned on those samples. However, HSMs have not been scaled to Partially Observable settings. We propose Recurrent Structural Policy Gradient (RSPG), the first history-aware HSM for settings involving public information. We also introduce MFAX, our JAX-based framework for MFGs. By leveraging known transition dynamics, RSPG achieves state-of-the-art performance as well as an order-of-magnitude faster convergence and solves, for the first time, a macroeconomics MFG with heterogeneous agents, common noise and history-aware policies. MFAX is publicly available at: https://github.com/CWibault/mfax.
