Learning Decentralized Routing Policies via Graph Attention-based Multi-Agent Reinforcement Learning in Lunar Delay-Tolerant Networks

2025-10-23

AI Summary

To address intermittent connectivity, unpredictable mobility, and absence of global topology information in multi-rover collaborative exploration missions within Lunar Delay-Tolerant Networks (LDTN), this paper proposes a decentralized routing framework. Methodologically, it formulates the problem as a Partially Observable Markov Decision Process (POMDP) and integrates Graph Attention Networks with Multi-Agent Reinforcement Learning (GAT-MARL), enabling centralized training and fully decentralized execution—without data replication or global state awareness. The key contribution lies in adaptive routing decisions derived solely from local observations and short-term mobility predictions. Monte Carlo simulations demonstrate that the approach significantly improves end-to-end delivery ratio under stochastic exploration scenarios, achieves zero redundant transmissions, reduces packet loss, and exhibits strong scalability and cross-environment generalization capability.

Technology Category

• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Multiagent Systems: Multiagent Learning
• Intelligent Robots: Multi-Robot Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Decentralized Web and Fediverse systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present a fully decentralized routing framework for multi-robot exploration missions operating under the constraints of a Lunar Delay-Tolerant Network (LDTN). In this setting, autonomous rovers must relay collected data to a lander under intermittent connectivity and unknown mobility patterns. We formulate the problem as a Partially Observable Markov Decision Problem (POMDP) and propose a Graph Attention-based Multi-Agent Reinforcement Learning (GAT-MARL) policy that performs Centralized Training, Decentralized Execution (CTDE). Our method relies only on local observations and does not require global topology updates or packet replication, unlike classical approaches such as shortest path and controlled flooding-based algorithms. Through Monte Carlo simulations in randomized exploration environments, GAT-MARL provides higher delivery rates, no duplications, and fewer packet losses, and is able to leverage short-term mobility forecasts; offering a scalable solution for future space robotic systems for planetary exploration, as demonstrated by successful generalization to larger rover teams.

Problem

• Solving decentralized routing for lunar rovers with intermittent connectivity
• Developing multi-agent reinforcement learning for delay-tolerant networks
• Improving data delivery rates while eliminating packet duplications

Innovation

• Graph Attention-based Multi-Agent Reinforcement Learning for routing
• Decentralized execution using only local observations
• Centralized training with short-term mobility forecasts