MPC-based Deep Reinforcement Learning Method for Space Robotic Control with Fuel Sloshing Mitigation

2025-09-25

AI Summary

To address the instability of autonomous satellite docking control induced by fuel slosh–generated disturbance forces under microgravity, this paper proposes a novel synergistic framework integrating Model Predictive Control (MPC) with Deep Reinforcement Learning (DRL). Specifically, MPC is embedded into both Soft Actor-Critic (SAC) and Proximal Policy Optimization (PPO) algorithms to leverage its explicit dynamical modeling and receding-horizon optimization capabilities, thereby enhancing policy training efficiency and robustness against slosh-induced disturbances. Compared to baseline pure-DRL and PPO-MPC approaches, SAC-MPC achieves significant improvements in high-fidelity 6-DOF simulations and parabolic zero-gravity experiments: docking accuracy increases by 23.6%, success rate by 18.4%, and propellant consumption decreases by 15.2%. This work establishes a safer, more reliable, and energy-efficient paradigm for autonomous rendezvous and docking in on-orbit servicing missions.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This paper presents an integrated Reinforcement Learning (RL) and Model Predictive Control (MPC) framework for autonomous satellite docking with a partially filled fuel tank. Traditional docking control faces challenges due to fuel sloshing in microgravity, which induces unpredictable forces affecting stability. To address this, we integrate Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) RL algorithms with MPC, leveraging MPC's predictive capabilities to accelerate RL training and improve control robustness. The proposed approach is validated through Zero-G Lab of SnT experiments for planar stabilization and high-fidelity numerical simulations for 6-DOF docking with fuel sloshing dynamics. Simulation results demonstrate that SAC-MPC achieves superior docking accuracy, higher success rates, and lower control effort, outperforming standalone RL and PPO-MPC methods. This study advances fuel-efficient and disturbance-resilient satellite docking, enhancing the feasibility of on-orbit refueling and servicing missions.

Problem

• Mitigating fuel sloshing effects during autonomous satellite docking operations
• Improving control stability in microgravity with unpredictable sloshing forces
• Enhancing docking accuracy and success rates for on-orbit servicing missions

Innovation

• Integrated Reinforcement Learning with Model Predictive Control
• Proximal Policy Optimization and Soft Actor-Critic algorithms
• Predictive MPC capabilities to accelerate RL training