Search papers, labs, and topics across Lattice.
This paper introduces the Consensus-based Communication and Knowledge Sharing (CCKS) framework to enhance cooperative Multi-Agent Reinforcement Learning (MARL) by addressing the limitations of existing action-advising methods. By employing contrastive learning to construct consensus models from local observations, CCKS enables agents to make informed decisions based on consensus-derived constraints while effectively balancing exploration and teacher guidance. Experimental results in environments like Google Research Football and StarCraft II show that CCKS significantly boosts cooperation efficiency, learning speed, and overall performance compared to traditional decentralized training and execution methods.
Agents using CCKS can achieve superior cooperation and learning efficiency by leveraging consensus-driven decision-making rather than rigid adherence to teacher guidance.
In Decentralized Training and Decentralized Execution (DTDE) for cooperative Multi-Agent Reinforcement Learning (MARL), action-advising-based knowledge sharing promotes interpretable and scalable cooperation among agents. However, current action advising approaches often adhere too much to the teacher's guidance without evaluating teacher-student compatibility, which causes excessive advising, suboptimal stability, and degraded performance. To overcome these challenges, this paper presents a Consensus-based Communication and Knowledge Sharing (CCKS) framework, which allows agents to adopt recommendations based on consensus-derived constraints and to follow the teacher's instructions more smartly. This mechanism enables agents to balance exploration and learning from experienced teachers, improving overall performance. The key is the consensus model construction, for which we propose to employ contrastive learning to construct consensus models based on local observations in the agents' training phase. In action selection, agents score and choose actions based on consensus and shared knowledge. Designed as a plug-and-play solution, CCKS integrates seamlessly with existing DTDE algorithms. Experiments conducted in the Google Research Football environment and the complex StarCraft II Multi-Agent Challenge demonstrate that the integration with CCKS significantly improves cooperation efficiency, learning speed, and overall performance compared with current DTDE baselines. The code is available at https://github.com/yuanxpy/CCKS.
