Search papers, labs, and topics across Lattice.
This paper introduces Hierarchical Lead Critic (HLC), a novel MARL architecture and sequential training scheme that learns from multiple hierarchical perspectives to improve coordination in cooperative multi-agent settings. HLC combines high-level objectives with low-level execution, drawing inspiration from natural team structures. Experiments on cooperative MARL benchmarks demonstrate that HLC outperforms single-hierarchy baselines, exhibiting improved sample efficiency, robustness, and scalability with increasing agent counts and task difficulty.
Unleashing hierarchical team structures in MARL agents boosts performance and robustness, demonstrating that multiple perspectives trump single, monolithic approaches.
Cooperative Multi-Agent Reinforcement Learning (MARL) solves complex tasks that require coordination from multiple agents, but is often limited to either local (independent learning) or global (centralized learning) perspectives. In this paper, we introduce a novel sequential training scheme and MARL architecture, which learns from multiple perspectives on different hierarchy levels. We propose the Hierarchical Lead Critic (HLC) - inspired by natural emerging distributions in team structures, where following high-level objectives combines with low-level execution. HLC demonstrates that introducing multiple hierarchies, leveraging local and global perspectives, can lead to improved performance with high sample efficiency and robust policies. Experimental results conducted on cooperative, non-communicative, and partially observable MARL benchmarks demonstrate that HLC outperforms single hierarchy baselines and scales robustly with increasing amounts of agents and difficulty.
