Aerial Manipulation with Contact-Aware Onboard Perception and Hybrid Control

2026-02-09

AI Summary

This work addresses the limitations of existing aerial manipulation systems, which rely on external motion capture and struggle to precisely regulate contact forces, thereby hindering real-world deployment. The authors propose a fully onboard, integrated perception-control framework that combines contact-aware visual-inertial odometry with a decoupled image-based visual servoing strategy, enabling, for the first time, closed-loop coordinated control of both contact force and motion without external positioning. By incorporating an enhanced visual-inertial odometry module, a contact-consistency factor, and a hybrid force-motion controller, the method reduces velocity estimation error by 66.01% during contact, significantly improving target approach stability and contact force regulation accuracy, thus laying the groundwork for field applications of aerial manipulation systems.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Motion & Tracking
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Aerial manipulation (AM) promises to move Unmanned Aerial Vehicles (UAVs) beyond passive inspection to contact-rich tasks such as grasping, assembly, and in-situ maintenance. Most prior AM demonstrations rely on external motion capture (MoCap) and emphasize position control for coarse interactions, limiting deployability. We present a fully onboard perception-control pipeline for contact-rich AM that achieves accurate motion tracking and regulated contact wrenches without MoCap. The main components are (1) an augmented visual-inertial odometry (VIO) estimator with contact-consistency factors that activate only during interaction, tightening uncertainty around the contact frame and reducing drift, and (2) image-based visual servoing (IBVS) to mitigate perception-control coupling, together with a hybrid force-motion controller that regulates contact wrenches and lateral motion for stable contact. Experiments show that our approach closes the perception-to-wrench loop using only onboard sensing, yielding an velocity estimation improvement of 66.01% at contact, reliable target approach, and stable force holding-pointing toward deployable, in-the-wild aerial manipulation.

Problem

• Aerial Manipulation
• Contact-rich Tasks
• Onboard Perception
• Force Control
• Deployability

Innovation

• contact-aware perception
• visual-inertial odometry
• hybrid force-motion control
• image-based visual servoing
• aerial manipulation