M$^{2}$GRPO: Mamba-based Multi-Agent Group Relative Policy Optimization for Biomimetic Underwater Robots Pursuit

2026-04-21

AI Summary

This work addresses the challenges of long-horizon decision-making, partial observability, and multi-agent coordination in cooperative pursuit tasks for bio-inspired underwater robots by proposing the M²GRPO framework. Operating within the centralized training with decentralized execution (CTDE) paradigm, M²GRPO integrates the Mamba state space model with temporal observation modeling and employs an attention mechanism to encode inter-agent relationships. It further introduces Group Relative Policy Optimization (GRPO) to enhance credit assignment among agents. Bounded continuous actions are generated via normalized Gaussian sampling, substantially reducing computational overhead and improving training stability. Experimental results demonstrate that M²GRPO consistently outperforms MAPPO and RNN-based baselines across varying team sizes and evader strategies, achieving significantly higher success rates and efficiency in both simulated and real-world aquatic environments.

Technology Category

• Multiagent Systems: Teamwork
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Planning and Decision Support for Human-Machine Teams

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Traditional policy learning methods in cooperative pursuit face fundamental challenges in biomimetic underwater robots, where long-horizon decision making, partial observability, and inter-robot coordination require both expressiveness and stability. To address these issues, a novel framework called Mamba-based multi-agent group relative policy optimization (M$^{2}$GRPO) is proposed, which integrates a selective state-space Mamba policy with group-relative policy optimization under the centralized-training and decentralized-execution (CTDE) paradigm. Specifically, the Mamba-based policy leverages observation history to capture long-horizon temporal dependencies and exploits attention-based relational features to encode inter-agent interactions, producing bounded continuous actions through normalized Gaussian sampling. To further improve credit assignment without sacrificing stability, the group-relative advantages are obtained by normalizing rewards across agents within each episode and optimized through a multi-agent extension of GRPO, significantly reducing the demand for training resources while enabling stable and scalable policy updates. Extensive simulations and real-world pool experiments across team scales and evader strategies demonstrate that M$^{2}$GRPO consistently outperforms MAPPO and recurrent baselines in both pursuit success rate and capture efficiency. Overall, the proposed framework provides a practical and scalable solution for cooperative underwater pursuit with biomimetic robot systems.

Problem

• cooperative pursuit
• biomimetic underwater robots
• long-horizon decision making
• partial observability
• multi-agent coordination

Innovation

• Mamba
• multi-agent reinforcement learning
• group-relative policy optimization
• biomimetic underwater robots
• credit assignment