Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ASALT, a novel method for adaptive state alignment in multi-agent reinforcement learning (MARL) that allows for effective transfer learning between domains with mismatched state-space dimensionalities. By employing observation-level and state-level adapters to create a shared embedding space, ASALT enhances the transfer of knowledge across heterogeneous agents, leading to improved sample efficiency and global returns in cooperative environments. Experimental results confirm that ASALT not only outperforms existing baselines but also reduces the risk of negative transfer, a common challenge in policy transfer scenarios.
ASALT transforms multi-agent reinforcement learning by enabling effective knowledge transfer even when source and target domains have mismatched state spaces.
Multi-agent reinforcement learning (MARL) addresses the problem of training multiple agents that pursue collaborative, competitive, or mixed objectives. Prior work has investigated transfer learning between source and target domains in MARL; however, the majority of existing approaches impose the constraint that the dimensionalities of the observation space and the global state space must be identical across domains. In this paper, we introduce a method that explicitly accommodates mismatched state-space dimensionalities between source and target domains. The proposed approach, ASALT, incorporates both observation-level and state-level adapters that map the target-domain observations and global states into a shared embedding space, thereby enabling more effective transfer of knowledge across both actors and critics. These adapters can generate embeddings that support efficient strategy transfer across heterogeneous domains. Experimental results on multiple configurations in standard benchmark environments demonstrate that ASALT surpasses existing baselines in terms of sample efficiency and global return in cooperative settings, but its effectiveness depends on the degree of mismatch between source and target domains. Furthermore, our findings indicate that ASALT mitigates negative transfer, which frequently constitutes a major obstacle when transferring policies between domains with differing observation and action spaces.