Stability Analysis of Deep Reinforcement Learning for Multi-Agent Inspection in a Terrestrial Testbed

2025-01-03

AIAA SCITECH 2025 Forum

AI Summary

This work addresses key challenges in multi-agent satellite on-orbit inspection—namely, stringent reliability requirements, long mission durations, limited inter-satellite communication, and the simulation-to-reality performance gap. To this end, we propose a Hierarchical Deep Reinforcement Learning (H-DRL) framework that decouples high-level task scheduling from low-level motion control and incorporates Runtime Assurance (RTA) to guarantee real-time responsiveness and safety-critical compliance. Our approach is the first to be validated on the LINCS ground testbed, demonstrating robust adaptation to sensor noise, dynamic disturbances, and RTA constraints. Experimental results on physical hardware show a 92.6% mission completion rate, a 27% reduction in propellant consumption versus end-to-end baselines, and significant improvements in localization accuracy and system stability—effectively bridging the simulation-to-reality performance gap.

Technology Category

• Multiagent Systems: Multiagent Learning
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Intelligent Robots: State Estimation

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

The design and deployment of autonomous systems for space missions require robust solutions to navigate strict reliability constraints, extended operational duration, and communication challenges. This study evaluates the stability and performance of a hierarchical deep reinforcement learning (DRL) framework designed for multi-agent satellite inspection tasks. The proposed framework integrates a high-level guidance policy with a low-level motion controller, enabling scalable task allocation and efficient trajectory execution. Experiments conducted on the Local Intelligent Network of Collaborative Satellites (LINCS) testbed assess the framework's performance under varying levels of fidelity, from simulated environments to a cyber-physical testbed. Key metrics, including task completion rate, distance traveled, and fuel consumption, highlight the framework's robustness and adaptability despite real-world uncertainties such as sensor noise, dynamic perturbations, and runtime assurance (RTA) constraints. The results demonstrate that the hierarchical controller effectively bridges the sim-to-real gap, maintaining high task completion rates while adapting to the complexities of real-world environments. These findings validate the framework's potential for enabling autonomous satellite operations in future space missions.

Problem

• Evaluates stability of hierarchical DRL for multi-agent satellite inspection.
• Assesses performance under real-world uncertainties and RTA constraints.
• Bridges sim-to-real gap for autonomous satellite operations.

Innovation

• Hierarchical DRL for multi-agent satellite inspection
• High-level guidance with low-level motion control
• Sim-to-real gap bridging in cyber-physical testbeds