Decentralized Aerial Manipulation of a Cable-Suspended Load using Multi-Agent Reinforcement Learning

2025-08-02

AI Summary

This work addresses the challenge of decentralized cooperative aerial manipulation of a cable-suspended payload by multiple micro air vehicles (MAVs) without global state information or inter-vehicle communication, enabling full 6-degree-of-freedom (6-DoF) control. We propose a decentralized multi-agent reinforcement learning (MARL) framework wherein each MAV observes only the payload’s attitude and coordinates implicitly via shared policy learning. A composite action space—comprising translational acceleration and angular velocity—is designed to match the underactuated, dynamic cable-suspension dynamics. A hierarchical architecture integrates an outer-loop MARL policy with a robust inner-loop controller to ensure high-fidelity sim-to-real transfer. To our knowledge, this is the first real-world demonstration of decentralized 6-DoF suspended payload control. Our approach matches centralized methods in pose tracking accuracy, robustness to model uncertainty, compatibility with heterogeneous policies, and graceful degradation under single-agent failure—while enabling high scalability and onboard real-time execution.

Technology Category

• Multiagent Systems: Multiagent Learning
• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization

Abstract

This paper presents the first decentralized method to enable real-world 6-DoF manipulation of a cable-suspended load using a team of Micro-Aerial Vehicles (MAVs). Our method leverages multi-agent reinforcement learning (MARL) to train an outer-loop control policy for each MAV. Unlike state-of-the-art controllers that utilize a centralized scheme, our policy does not require global states, inter-MAV communications, nor neighboring MAV information. Instead, agents communicate implicitly through load pose observations alone, which enables high scalability and flexibility. It also significantly reduces computing costs during inference time, enabling onboard deployment of the policy. In addition, we introduce a new action space design for the MAVs using linear acceleration and body rates. This choice, combined with a robust low-level controller, enables reliable sim-to-real transfer despite significant uncertainties caused by cable tension during dynamic 3D motion. We validate our method in various real-world experiments, including full-pose control under load model uncertainties, showing setpoint tracking performance comparable to the state-of-the-art centralized method. We also demonstrate cooperation amongst agents with heterogeneous control policies, and robustness to the complete in-flight loss of one MAV. Videos of experiments: https://autonomousrobots.nl/paper_websites/aerial-manipulation-marl

Problem

• Decentralized 6-DoF cable-suspended load manipulation using MAVs
• MARL-based control without global states or inter-MAV communication
• Robust sim-to-real transfer under dynamic cable tension uncertainties

Innovation

• Decentralized MARL for MAV team control
• Implicit communication via load pose observations
• Linear acceleration action space for sim-to-real