Towards Bio-inspired Heuristically Accelerated Reinforcement Learning for Adaptive Underwater Multi-Agents Behaviour
To address the challenges of high dynamic uncertainty, limited communication, and slow training convergence in underwater multi-AUV cooperative area coverage tasks, this paper proposes a PSO-guided multi-agent reinforcement learning (MARL) framework. The method integrates particle swarm optimization (PSO)—a bio-inspired metaheuristic—into the Multi-Agent Soft Actor-Critic (MSAC) algorithm, enabling heuristic-guided exploration of high-value state-action regions during early training stages and thereby improving the exploration-exploitation trade-off. Leveraging deep neural networks for function approximation and continuous-control MARL techniques, the approach significantly reduces training interaction steps and accelerates convergence to optimal collaborative policies in 2D underwater coverage simulations. Experimental results demonstrate that the proposed method achieves comparable task performance while substantially enhancing training efficiency, offering a practical pathway for deploying MARL in real-world underwater robotic systems.