Zero-shot Sim-to-Real Transfer for Reinforcement Learning-based Visual Servoing of Soft Continuum Arms

2025-04-23

AI Summary

Soft continuum arms (SCAs) pose significant challenges in modeling and control due to their infinite degrees of freedom and strong nonlinearities, especially for vision-based servoing tasks where zero-shot sim-to-real transfer remains elusive. This paper proposes a decoupled vision-servoing framework grounded in proximal policy optimization (PPO). It introduces a novel dual-controller architecture that explicitly separates kinematic and dynamic control, relying solely on monocular vision and minimal proprioceptive sensing—without requiring real-world data fine-tuning. Training is conducted exclusively in simulation (PyBullet/Gazebo), achieving a 99.8% task success rate. The policy transfers zero-shot to a pneumatic single-segment soft arm, attaining a 67% success rate—substantially outperforming existing end-to-end approaches. Our key contribution is a robust, low-perception-demand framework enabling cross-domain zero-shot transfer, establishing a new paradigm toward practical deployment of SCAs.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Soft continuum arms (SCAs) soft and deformable nature presents challenges in modeling and control due to their infinite degrees of freedom and non-linear behavior. This work introduces a reinforcement learning (RL)-based framework for visual servoing tasks on SCAs with zero-shot sim-to-real transfer capabilities, demonstrated on a single section pneumatic manipulator capable of bending and twisting. The framework decouples kinematics from mechanical properties using an RL kinematic controller for motion planning and a local controller for actuation refinement, leveraging minimal sensing with visual feedback. Trained entirely in simulation, the RL controller achieved a 99.8% success rate. When deployed on hardware, it achieved a 67% success rate in zero-shot sim-to-real transfer, demonstrating robustness and adaptability. This approach offers a scalable solution for SCAs in 3D visual servoing, with potential for further refinement and expanded applications.

Problem

• Modeling and controlling soft continuum arms with infinite degrees of freedom
• Achieving zero-shot sim-to-real transfer for visual servoing tasks
• Decoupling kinematics from mechanical properties for improved control

Innovation

• RL-based visual servoing for soft continuum arms
• Zero-shot sim-to-real transfer capability
• Decoupled kinematics and local control