Search papers, labs, and topics across Lattice.
The paper addresses the zero-shot coordination (ZSC) problem in multi-agent reinforcement learning, specifically focusing on scenarios where agents must cooperate with partners trained with diverse reward shaping functions for the same sparse objective. They propose training an ensemble of policies using randomized reward shaping, selected via four different algorithms, to improve generalization. Experiments in the Overcooked environment demonstrate significant performance gains (62.2%-119.2%) compared to baseline ZSC methods when coordinating with agents trained with different reward shapings.
MARL agents can learn to cooperate with unseen partners trained with diverse reward shapings, boosting performance by up to 119% in sparse reward environments.
Many Multi-Agent Reinforcement Learning (MARL) agents fail to adapt properly to cooperating with agents trained with the same objectives but different seeds, algorithms, or other training differences. This is the problem of Zero-Shot Coordination (ZSC), which focuses on training agents to cooperate well with unknown agents. ZSC has been studied for a variety of tabular cases and simple games such as Hanabi, achieving excellent results. However, existing solutions to ZSC only consider identical rewards for your trained agents and all future partners. This is not realistic for the trained agents, as they do not consider the problem of cooperating with agents that have identical sparse objectives but shape the rewards for those objectives in different manner. To address this issue, we show how to train an ensemble of methods using randomized reward shapings chosen using 4 selection algorithms. Experiments done on the Overcooked environment demonstrate consistent improvements of 62.2%-119.2% in sparse reward over baseline ZSC algorithms when playing with agents that have identical sparse rewards but different reward shapings.
