Search papers, labs, and topics across Lattice.
This paper introduces COMAD, a framework for Continual Offline Multi-agent Skill Discovery that enhances learning efficiency by extracting and reusing coordination skills from multi-agent offline datasets. By employing an auto-encoder to transform coordination knowledge into reusable skills and a skill-augmented policy learning objective, COMAD effectively addresses issues of distributional shift and catastrophic forgetting in multi-agent settings. Empirical results demonstrate that COMAD significantly improves both forward and backward transfer across various multi-agent reinforcement learning benchmarks, outperforming existing methods.
COMAD expands the skill library of agents in real-time, dramatically improving their ability to adapt and reuse coordination skills without suffering from interference or forgetting.
Extracting skills from multi-agent offline dataset improves learning efficiency via sharing task-invariant coordination skills among tasks. In settings where tasks occur sequentially and the space of skills grows exponentially, existing approaches that rely on heuristically designed and fixed-sized skill libraries struggle to resolve the problem of distributional shift and interference, facing catastrophic forgetting and plasticity loss. To address this problem and endow agents with the ability to continually discover and reuse coordination skills in open-environment, we propose COMAD, a principled framework for Continual Offline Multi-agent Skill Discovery via Skill Partition and Reuse. We first discover skills from mixed multi-agent behavior data with an auto-encoder to transform coordination knowledge into reusable coordination skills. Then we construct a skill-augmented policy learning objective with multi-head architectures, explicitly guiding the advantage function with reusable skills identified via a density-based reusability estimator. Theoretical analysis shows our method approximates the optimum of a continual skill discovery problem. Empirical results across diverse MARL benchmarks show that COMAD continually expands its skill library to mitigate interference, achieving superior forward and backward transfer for task streams compared to multiple baselines.
