AI-Enabled Image-Based Hybrid Vision/Force Control of Tendon-Driven Aerial Continuum Manipulators

2026-04-20

AI Summary

This work addresses the challenges of visual servoing and contact force control for tendon-driven aerial continuum manipulators during autonomous interaction in static environments. The authors propose a cascaded hybrid visual/force control framework based on an SE(3) constant-strain model. Innovatively, a graph neural network is introduced to extract line features, replacing conventional geometric methods, and is combined with fixed-time sliding mode control and radial basis function neural networks to enable rapid online learning and compensation of uncertainties arising from monocular eye-in-hand vision and force sensing—without requiring offline training. Simulation and experimental results demonstrate that the proposed method consistently drives image feature errors to convergence and accurately tracks desired contact forces under various initial conditions, significantly outperforming traditional rigid-arm approaches in terms of robustness and task execution capability.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Low Level & Physics-based Vision
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This paper presents an AI-enabled cascaded hybrid vision/force control framework for tendon-driven aerial continuum manipulators based on constant-strain modeling in $SE(3)$ as a coupled system. The proposed controller is designed to enable autonomous, physical interaction with a static environment while stabilizing the image feature error. The developed strategy combines the cascaded fast fixed-time sliding mode control and a radial basis function neural network to cope with the uncertainties in the image acquired by the eye-in-hand monocular camera and the measurements from the force sensing apparatus. This ensures rapid, online learning of the vision- and force-related uncertainties without requiring offline training. Furthermore, the features are extracted via a state-of-the-art graph neural network architecture employed by a visual servoing framework using line features, rather than relying on heuristic geometric line extractors, to concurrently contribute to tracking the desired normal interaction force during contact and regulating the image feature error. A comparative study benchmarks the proposed controller against established rigid-arm aerial manipulation methods, evaluating robustness across diverse scenarios and feature extraction strategies. The simulation and experimental results showcase the effectiveness of the proposed methodology under various initial conditions and demonstrate robust performance in executing manipulation tasks.

Problem

• aerial continuum manipulator
• hybrid vision/force control
• tendon-driven
• image feature error
• physical interaction

Innovation

• hybrid vision/force control
• aerial continuum manipulator
• graph neural network
• fixed-time sliding mode control
• online uncertainty learning