🤖 AI Summary
This work addresses the challenge of cooperative pursuit by heterogeneous unmanned surface vehicles (USVs) in constrained port waterways, where navigation, traffic, and role-based constraints must be respected. To this end, the authors propose the OGR-MARL framework, which integrates shared goal beliefs, a role-conditioned options mechanism, adaptive rule-based penalties, and residual policy learning. This enables multi-agent systems to refine actions while adhering to prior domain rules, rather than exploring from scratch. Notably, the framework decouples the options mechanism and residual learning from specific multi-agent reinforcement learning algorithms—such as MADDPG, MATD3, MAPPO, and MASAC—yielding a general architecture capable of zero-shot transfer to real-world port maps. Evaluated in the simulated Shin-no-Mon port environment, OGR-MASAC achieves a 75.0% capture rate, substantially outperforming baselines, and successfully generalizes without retraining to realistic scenarios derived from QGIS/AIS data, demonstrating both effective coordination and strict rule compliance.
📝 Abstract
Heterogeneous USV cooperative pursuit in constrained port waterways requires evader interception under navigation, traffic, and role constraints. This paper proposes OGR-MARL, an option-guided residual multi-agent reinforcement learning framework that is decoupled from a specific MARL algorithm. OGR-MARL integrates shared evader belief, role-conditioned option targets, adaptive rule penalties, and residual policy learning, allowing different MARL algorithms to learn corrective actions on top of rule-guided behaviors rather than exploring constrained port environments from scratch. We instantiate OGR-MARL with representative continuous-control MARL backbones, including MADDPG, MATD3, MAPPO, and MASAC, yielding OGR-MADDPG, OGR-MATD3, OGR-MAPPO, and OGR-MASAC. Experiments in an abstract Xiazhimen port-waterway scenario show that the OGR-MASAC instantiation achieves a 75.0% capture rate, promising mission-effective rule compliance, and the best heterogeneous coordination among the tested methods. Without retraining, zero-shot transfer to a QGIS/AIS-informed Xiazhimen map achieves promising results, demonstrating the generalization potential of OGR-MARL in more complex port scenarios.