Multitask Reinforcement Learning for Quadcopter Attitude Stabilization and Tracking using Graph Policy

2025-03-11

AI Summary

This work addresses the coupled yet fundamentally distinct dual-control problem of quadrotor attitude tracking and strong robust stabilization under arbitrary initial conditions. We propose the first multi-task Soft Actor-Critic (SAC) framework leveraging Graph Convolutional Network (GCN)-based policies. Our key innovation lies in embedding GCNs into the shared multi-task policy network to enable unified modeling of heterogeneous control objectives. The resulting policy is highly compact—only 24 neurons per layer—enabling real-time onboard deployment at 400 Hz. Training is conducted in parallel within IsaacGym, and the learned policy is rigorously validated on a Pixhawk-based physical platform. Compared to single-task baselines, our approach achieves a 3.2× improvement in sample efficiency and a 2.8× acceleration in training convergence, while delivering millisecond-level real-time response with zero additional computational overhead.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Quadcopter attitude control involves two tasks: smooth attitude tracking and aggressive stabilization from arbitrary states. Although both can be formulated as tracking problems, their distinct state spaces and control strategies complicate a unified reward function. We propose a multitask deep reinforcement learning framework that leverages parallel simulation with IsaacGym and a Graph Convolutional Network (GCN) policy to address both tasks effectively. Our multitask Soft Actor-Critic (SAC) approach achieves faster, more reliable learning and higher sample efficiency than single-task methods. We validate its real-world applicability by deploying the learned policy - a compact two-layer network with 24 neurons per layer - on a Pixhawk flight controller, achieving 400 Hz control without extra computational resources. We provide our code at https://github.com/robot-perception-group/GraphMTSAC_UAV/.

Problem

• Quadcopter attitude stabilization and tracking challenges
• Unified reward function for distinct control tasks
• Real-world deployment on Pixhawk flight controller

Innovation

• Multitask deep reinforcement learning for quadcopter control
• Graph Convolutional Network policy for unified task handling
• Real-time deployment on Pixhawk at 400 Hz