Global End-Effector Pose Control of an Underactuated Aerial Manipulator via Reinforcement Learning

2025-12-24

AI Summary

Lightweight underactuated aerial manipulators struggle to achieve globally stable 6-DoF end-effector pose control under strong environmental disturbances. Method: This paper proposes a hierarchical control framework integrating Proximal Policy Optimization (PPO) reinforcement learning with Incremental Nonlinear Dynamic Inversion (INDI), enabling precise full 6-DoF end-effector control using only a 2-DoF onboard manipulator mounted on a quadrotor. Differential mechanism design and sim-to-real transfer training address challenges including limited joint degrees of freedom, model uncertainty, and sensitivity to contact-induced disturbances. Contribution/Results: Experiments demonstrate centimeter-level positional accuracy and degree-level orientation accuracy. The system successfully performs heavy-object transportation and stable contact tasks under significant thrust disturbances, markedly enhancing the robustness and practical applicability of underactuated aerial manipulators in real-world scenarios.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Aerial manipulators, which combine robotic arms with multi-rotor drones, face strict constraints on arm weight and mechanical complexity. In this work, we study a lightweight 2-degree-of-freedom (DoF) arm mounted on a quadrotor via a differential mechanism, capable of full six-DoF end-effector pose control. While the minimal design enables simplicity and reduced payload, it also introduces challenges such as underactuation and sensitivity to external disturbances, including manipulation of heavy loads and pushing tasks. To address these, we employ reinforcement learning, training a Proximal Policy Optimization (PPO) agent in simulation to generate feedforward commands for quadrotor acceleration and body rates, along with joint angle targets. These commands are tracked by an incremental nonlinear dynamic inversion (INDI) attitude controller and a PID joint controller, respectively. Flight experiments demonstrate centimeter-level position accuracy and degree-level orientation precision, with robust performance under external force disturbances. The results highlight the potential of learning-based control strategies for enabling contact-rich aerial manipulation using simple, lightweight platforms.

Problem

• Control underactuated aerial manipulator's end-effector pose
• Handle external disturbances and heavy load manipulation
• Achieve precise control with lightweight, simple mechanical design

Innovation

• Reinforcement learning trains PPO agent for feedforward commands
• INDI attitude controller and PID joint controller track commands
• Achieves centimeter-level position and degree-level orientation precision