Reinforcement Learning-based Fault-Tolerant Control for Quadrotor with Online Transformer Adaptation

2025-05-13

AI Summary

Multirotor UAVs exhibit high sensitivity to actuator faults, often leading to rapid instability and mission failure; existing reinforcement learning (RL)-based fault-tolerant control approaches typically rely on prior dynamical models or lack generalizability across novel vehicle configurations. This paper proposes a hybrid RL fault-tolerant control framework integrating Proximal Policy Optimization (PPO) with an online Transformer-based adaptive module—the first application of Transformers in RL-based fault tolerance—enabling real-time latent-space modeling without prior dynamics knowledge and zero-shot cross-configuration adaptation. Evaluated in PyBullet simulations, the method achieves a 95% fault-handling success rate (+9% absolute improvement), reduces position root-mean-square error to 0.129 m (−15.7%), and maintains high robustness on untrained configurations. It eliminates the need for offline retraining or precise system identification, overcoming key limitations of conventional model-dependent or configuration-specific methods.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Natural Language Processing: Safety and Robustness

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Multirotors play a significant role in diverse field robotics applications but remain highly susceptible to actuator failures, leading to rapid instability and compromised mission reliability. While various fault-tolerant control (FTC) strategies using reinforcement learning (RL) have been widely explored, most previous approaches require prior knowledge of the multirotor model or struggle to adapt to new configurations. To address these limitations, we propose a novel hybrid RL-based FTC framework integrated with a transformer-based online adaptation module. Our framework leverages a transformer architecture to infer latent representations in real time, enabling adaptation to previously unseen system models without retraining. We evaluate our method in a PyBullet simulation under loss-of-effectiveness actuator faults, achieving a 95% success rate and a positional root mean square error (RMSE) of 0.129 m, outperforming existing adaptation methods with 86% success and an RMSE of 0.153 m. Further evaluations on quadrotors with varying configurations confirm the robustness of our framework across untrained dynamics. These results demonstrate the potential of our framework to enhance the adaptability and reliability of multirotors, enabling efficient fault management in dynamic and uncertain environments. Website is available at http://00dhkim.me/paper/rl-ftc

Problem

• Addressing actuator failures in quadrotors for stability and reliability
• Overcoming prior model dependency in fault-tolerant control strategies
• Enabling real-time adaptation to unseen system configurations without retraining

Innovation

• Hybrid RL-based FTC with transformer adaptation
• Real-time latent representation inference via transformer
• Robust performance across untrained quadrotor dynamics