Deep Reinforcement Learning for Spacecraft Attitude Control During Atmospheric Re-Entry

2026-06-30

AI Summary

This study addresses the insufficient robustness of conventional spacecraft attitude control methods during atmospheric re-entry, which arises from strong nonlinearities, parametric uncertainties, and actuator faults. To overcome these challenges, the authors propose a hybrid control architecture that integrates off-policy continuous-action deep reinforcement learning with gain-scheduled PID control. The reinforcement learning policy is trained using dynamics randomization to enhance generalization across variations in mass properties, inertia tensors, and control surface bandwidths. This work also presents the first systematic evaluation of the synergistic performance between deep reinforcement learning and traditional control techniques in the re-entry context. Experimental results demonstrate that the proposed approach achieves superior angle-of-attack tracking accuracy within the prescribed flight envelope compared to conventional controllers and exhibits significantly improved robustness against multiple sources of uncertainty.

Abstract

Deep reinforcement learning has the potential to solve attitude control problems more adaptively, precisely, and robustly by handling nonlinear dynamics, uncertainties, and failure cases more effectively than traditional attitude control approaches. We explore reinforcement learning (RL) for attitude control in spacecraft re-entry. An industry-standard proportional-integral-derivative controller with gain scheduling serves as a strong baseline for model-free RL and hybrid controllers that combine these two approaches. We formalize the application in the RL framework to apply continuous, off-policy RL. State-of-the-art RL achieves comparable performance to traditional control approaches in this domain. However, its out-of-distribution generalization is not sufficient. Hence, we use dynamics randomization to introduce challenging task variations during training and enforce generalization in a predefined operational envelope. Finally, we assess the best obtained RL-based controllers with application-specific metrics to show superior performance in comparison to traditional controllers in the operational envelope, that is, hybrid controllers are able to track the angle of attack better and are more robust under variations of mass, inertia tensor, and flap actuator bandwidth.

Problem

• spacecraft attitude control
• atmospheric re-entry
• deep reinforcement learning
• robustness
• nonlinear dynamics

Innovation

• deep reinforcement learning
• attitude control
• dynamics randomization
• hybrid control
• atmospheric re-entry