Search papers, labs, and topics across Lattice.
This paper addresses the challenge of scaling cooperative multi-agent reinforcement learning (MARL) with heterogeneous agent interactions by introducing Graphon Mean-Field Subsampling (GMFS). GMFS approximates the graphon-weighted mean-field by subsampling κ agents based on interaction strength, thereby reducing computational complexity. The authors demonstrate a sample complexity of poly(κ) and an optimality gap of O(1/√κ), validated through robotic coordination simulations exhibiting near-optimal performance.
Achieve near-optimal multi-agent coordination with heterogeneous agents while drastically reducing computational cost by intelligently subsampling interactions.
Coordinating large populations of interacting agents is a central challenge in multi-agent reinforcement learning (MARL), where the size of the joint state-action space scales exponentially with the number of agents. Mean-field methods alleviate this burden by aggregating agent interactions, but these approaches assume homogeneous interactions. Recent graphon-based frameworks capture heterogeneity, but are computationally expensive as the number of agents grows. Therefore, we introduce $\texttt{GMFS}$, a $\textbf{G}$raphon $\textbf{M}$ean-$\textbf{F}$ield $\textbf{S}$ubsampling framework for scalable cooperative MARL with heterogeneous agent interactions. By subsampling $κ$ agents according to interaction strength, we approximate the graphon-weighted mean-field and learn a policy with sample complexity $\mathrm{poly}(κ)$ and optimality gap $O(1/\sqrtκ)$. We verify our theory with numerical simulations in robotic coordination, showing that $\texttt{GMFS}$ achieves near-optimal performance.
