A Neural Network Mode for PX4 on Embedded Flight Controllers

2025-05-01

AI Summary

This work addresses the complexity and lack of end-to-end optimizability inherent in the conventional cascaded control architecture of the PX4 autopilot. We propose an embedded neural network controller framework that directly maps position setpoints to normalized motor RPM commands. Methodologically, we train a policy network in the Aerial Gym simulator and deploy it on ARM Cortex-M microcontrollers via TensorFlow Lite model quantization and optimization. We develop a custom PX4 module and integrate ROS/Gazebo simulation interfaces, establishing an open-source, full-stack pipeline—from simulation training to real-world deployment. Our key contributions are: (1) the first implementation of end-to-end learned closed-loop control on real embedded flight control hardware (PX4 + TensorFlow Lite); (2) real-drone trajectory tracking performance matching simulation results, demonstrating robustness and feasibility; and (3) a substantial reduction in the deployment barrier for learning-based controllers on resource-constrained UAV platforms.

Technology Category

• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Learning to Search
• Machine Learning: Learning on the Edge & Model Compression

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

This paper contributes an open-sourced implementation of a neural-network based controller framework within the PX4 stack. We develop a custom module for inference on the microcontroller while retaining all of the functionality of the PX4 autopilot. Policies trained in the Aerial Gym Simulator are converted to the TensorFlow Lite format and then built together with PX4 and flashed to the flight controller. The policies substitute the control-cascade within PX4 to offer an end-to-end position-setpoint tracking controller directly providing normalized motor RPM setpoints. Experiments conducted in simulation and the real-world show similar tracking performance. We thus provide a flight-ready pipeline for testing neural control policies in the real world. The pipeline simplifies the deployment of neural networks on embedded flight controller hardware thereby accelerating research on learning-based control. Both the Aerial Gym Simulator and the PX4 module are open-sourced at https://github.com/ntnu-arl/aerial_gym_simulator and https://github.com/SindreMHegre/PX4-Autopilot-public/tree/for_paper. Video: https://youtu.be/lY1OKz_UOqM?si=VtzL243BAY3lblTJ.

Problem

• Implement neural-network controller in PX4 for embedded flight
• Convert trained policies to TensorFlow Lite for microcontroller inference
• Enable real-world testing of neural control policies efficiently

Innovation

• Neural network controller in PX4 stack
• TensorFlow Lite for microcontroller inference
• End-to-end position-setpoint tracking controller