Search papers, labs, and topics across Lattice.
This paper introduces the Consensus Multi-Agent Transformer (CMAT), a novel centralized MARL framework that bridges cooperative MARL to a hierarchical SARL formulation by treating all agents as a unified entity. CMAT uses a Transformer encoder to process joint observations and a hierarchical Transformer decoder to autoregressively generate a high-level consensus vector in latent space, enabling order-independent joint decision making. Experiments across StarCraft II, Multi-Agent MuJoCo, and Google Research Football demonstrate that CMAT outperforms existing centralized and decentralized MARL methods.
Achieve state-of-the-art MARL performance by having agents reach a consensus in latent space before acting, effectively transforming the multi-agent problem into a hierarchical single-agent one.
Cooperative multi-agent reinforcement learning (MARL) is widely used to address large joint observation and action spaces by decomposing a centralized control problem into multiple interacting agents. However, such decomposition often introduces additional challenges, including non-stationarity, unstable training, weak coordination, and limited theoretical guarantees. In this paper, we propose the Consensus Multi-Agent Transformer (CMAT), a centralized framework that bridges cooperative MARL to a hierarchical single-agent reinforcement learning (SARL) formulation. CMAT treats all agents as a unified entity and employs a Transformer encoder to process the large joint observation space. To handle the extensive joint action space, we introduce a hierarchical decision-making mechanism in which a Transformer decoder autoregressively generates a high-level consensus vector, simulating the process by which agents reach agreement on their strategies in latent space. Conditioned on this consensus, all agents generate their actions simultaneously, enabling order-independent joint decision making and avoiding the sensitivity to action-generation order in conventional Multi-Agent Transformers (MAT). This factorization allows the joint policy to be optimized using single-agent PPO while preserving expressive coordination through the latent consensus. To evaluate the proposed method, we conduct experiments on benchmark tasks from StarCraft II, Multi-Agent MuJoCo, and Google Research Football. The results show that CMAT achieves superior performance over recent centralized solutions, sequential MARL methods, and conventional MARL baselines. The code for this paper is available at:https://github.com/RS2002/CMAT .
