Reinforcement Learning Based Prediction of PID Controller Gains for Quadrotor UAVs

2025-02-06

AI Summary

To address insufficient attitude control accuracy of quadrotor UAVs and the limitations of conventional PID controllers—namely, manual gain tuning and poor adaptability to dynamic environments—this paper proposes an online adaptive PID gain optimization method based on Deep Deterministic Policy Gradient (DDPG). For the first time, DDPG is employed to perform real-time, closed-loop adjustment of PID gains (proportional, integral, and derivative) during flight, using state feedback for continuous parameter updates. High-fidelity simulation is conducted in MATLAB/Simulink integrated with the PX4 UAV Toolbox. Experimental results demonstrate significant reductions in attitude tracking error, markedly improved trajectory tracking accuracy and robustness, and clear superiority over manually tuned PID baselines. This work establishes a deployable, reinforcement learning–driven paradigm for intelligent, adaptive PID tuning in autonomous UAV control.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

A reinforcement learning (RL) based methodology is proposed and implemented for online fine-tuning of PID controller gains, thus, improving quadrotor effective and accurate trajectory tracking. The RL agent is first trained offline on a quadrotor PID attitude controller and then validated through simulations and experimental flights. RL exploits a Deep Deterministic Policy Gradient (DDPG) algorithm, which is an off-policy actor-critic method. Training and simulation studies are performed using Matlab/Simulink and the UAV Toolbox Support Package for PX4 Autopilots. Performance evaluation and comparison studies are performed between the hand-tuned and RL-based tuned approaches. The results show that the controller parameters based on RL are adjusted during flights, achieving the smallest attitude errors, thus significantly improving attitude tracking performance compared to the hand-tuned approach.

Problem

• Online fine-tuning of PID controller gains
• Improving quadrotor trajectory tracking
• Comparison between hand-tuned and RL-based methods

Innovation

• Reinforcement Learning PID tuning
• Deep Deterministic Policy Gradient
• Online quadrotor trajectory optimization