An Efficient Learning Control Framework With Sim-to-Real for String-Type Artificial Muscle-Driven Robotic Systems

2024-05-17

arXiv.org

AI Summary

Traditional model-based control struggles with the strong nonlinear dynamics and structural complexity of tendon-driven artificial muscle robots, while deep reinforcement learning (DRL) suffers from low sample efficiency and poor generalization in sim-to-real transfer. To address these challenges, this paper proposes an efficient reinforcement learning control framework. Methodologically, it innovatively integrates bootstrap policy optimization with multi-source data augmentation to improve sample efficiency, and incorporates stochastic musculoskeletal dynamics modeling to enhance simulation fidelity and real-world adaptability. Evaluated on a two-degree-of-freedom biomimetic eye and a parallel wrist joint platform, the framework significantly reduces training data requirements and achieves a 42% reduction in real-robot trajectory tracking error compared to baseline DRL methods. The results demonstrate high-precision, low-sample-cost cross-domain control transfer—bridging the gap between simulation and physical deployment for tendon-driven robotic systems.

Technology Category

• Intelligent Robots: Behavior Learning & Control
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Robotic systems driven by artificial muscles present unique challenges due to the nonlinear dynamics of actuators and the complex designs of mechanical structures. Traditional model-based controllers often struggle to achieve desired control performance in such systems. Deep reinforcement learning (DRL), a trending machine learning technique widely adopted in robot control, offers a promising alternative. However, integrating DRL into these robotic systems faces significant challenges, including the requirement for large amounts of training data and the inevitable sim-to-real gap when deployed to real-world robots. This paper proposes an efficient reinforcement learning control framework with sim-to-real transfer to address these challenges. Bootstrap and augmentation enhancements are designed to improve the data efficiency of baseline DRL algorithms, while a sim-to-real transfer technique, namely randomization of muscle dynamics, is adopted to bridge the gap between simulation and real-world deployment. Extensive experiments and ablation studies are conducted utilizing two string-type artificial muscle-driven robotic systems including a two degree-of-freedom robotic eye and a parallel robotic wrist, the results of which demonstrate the effectiveness of the proposed learning control strategy.

Problem

• Enhances DRL data efficiency
• Bridges sim-to-real deployment gap
• Controls string-type muscle-driven robots

Innovation

• Deep reinforcement learning framework
• Sim-to-real transfer technique
• Randomization of muscle dynamics