Search papers, labs, and topics across Lattice.
MARL-GPT trains a single GPT-based model on offline expert trajectories from diverse MARL environments (SMACv2, GRF, POGEMA) using a shared transformer-based observation encoder. This approach eliminates the need for task-specific model tuning and enables generalization across environments. MARL-GPT achieves performance competitive with specialized MARL baselines, demonstrating the feasibility of a foundation model for multi-agent reinforcement learning.
A single transformer can master StarCraft, Football, and POGEMA, suggesting we can unify MARL under one foundation model.
Recent advances in multi-agent reinforcement learning (MARL) have demonstrated success in numerous challenging domains and environments, but typically require specialized models for each task. In this work, we propose a coherent methodology that makes it possible for a single GPT-based model to learn and perform well across diverse MARL environments and tasks, including StarCraft Multi-Agent Challenge, Google Research Football and POGEMA. Our method, MARL-GPT, applies offline reinforcement learning to train at scale on the expert trajectories (400M for SMACv2, 100M for GRF, and 1B for POGEMA) combined with a single transformer-based observation encoder that requires no task-specific tuning. Experiments show that MARL-GPT achieves competitive performance compared to specialized baselines in all tested environments. Thus, our findings suggest that it is, indeed, possible to build a multi-task transformer-based model for a wide variety of (significantly different) multi-agent problems paving the way to the fundamental MARL model (akin to ChatGPT, Llama, Mistral etc. in natural language modeling).
