Search papers, labs, and topics across Lattice.
The paper introduces Action Graph Policies (AGP), a novel MARL approach that explicitly models dependencies between agents' actions to improve coordination. AGP constructs "coordination contexts" that allow agents to condition their action selection on global action dependencies, leading to more synchronized and conflict-free behavior. Empirically, AGP significantly outperforms existing MARL methods on coordination tasks, achieving 80-95% success rates compared to 10-25% for other methods, and demonstrates consistent improvements across diverse multi-agent environments.
Forget independent learners: Action Graph Policies unlock dramatically better multi-agent coordination by explicitly modeling action dependencies.
Coordinating actions is the most fundamental form of cooperation in multi-agent reinforcement learning (MARL). Successful decentralized decision-making often depends not only on good individual actions, but on selecting compatible actions across agents to synchronize behavior, avoid conflicts, and satisfy global constraints. In this paper, we propose Action Graph Policies (AGP), that model dependencies among agents' available action choices. It constructs, what we call, \textit{coordination contexts}, that enable agents to condition their decisions on global action dependencies. Theoretically, we show that AGPs induce a strictly more expressive joint policy compared to fully independent policies and can realize coordinated joint actions that are provably more optimal than greedy execution even from centralized value-decomposition methods. Empirically, we show that AGP achieves 80-95\% success on canonical coordination tasks with partial observability and anti-coordination penalties, where other MARL methods reach only 10-25\%. We further demonstrate that AGP consistently outperforms these baselines in diverse multi-agent environments.
