Search papers, labs, and topics across Lattice.
This paper introduces M$^{2}$GRPO, a novel multi-agent reinforcement learning framework for cooperative underwater pursuit using biomimetic robots. It combines a selective state-space Mamba policy for long-horizon temporal dependencies and inter-agent interaction encoding with group-relative policy optimization under the CTDE paradigm. Experiments show M$^{2}$GRPO outperforms MAPPO and recurrent baselines in pursuit success rate and capture efficiency in both simulation and real-world pool experiments.
Mamba's ability to model long-range dependencies unlocks significantly improved coordination and efficiency in multi-agent robotic pursuit compared to traditional recurrent and attention-based methods.
Traditional policy learning methods in cooperative pursuit face fundamental challenges in biomimetic underwater robots, where long-horizon decision making, partial observability, and inter-robot coordination require both expressiveness and stability. To address these issues, a novel framework called Mamba-based multi-agent group relative policy optimization (M$^{2}$GRPO) is proposed, which integrates a selective state-space Mamba policy with group-relative policy optimization under the centralized-training and decentralized-execution (CTDE) paradigm. Specifically, the Mamba-based policy leverages observation history to capture long-horizon temporal dependencies and exploits attention-based relational features to encode inter-agent interactions, producing bounded continuous actions through normalized Gaussian sampling. To further improve credit assignment without sacrificing stability, the group-relative advantages are obtained by normalizing rewards across agents within each episode and optimized through a multi-agent extension of GRPO, significantly reducing the demand for training resources while enabling stable and scalable policy updates. Extensive simulations and real-world pool experiments across team scales and evader strategies demonstrate that M$^{2}$GRPO consistently outperforms MAPPO and recurrent baselines in both pursuit success rate and capture efficiency. Overall, the proposed framework provides a practical and scalable solution for cooperative underwater pursuit with biomimetic robot systems.
