Multi-Goal Dexterous Hand Manipulation using Probabilistic Model-based Reinforcement Learning

2025-04-30

AI Summary

Addressing the challenges of high-dimensional dynamic modeling and millisecond-level latency sensitivity in dexterous multi-objective manipulation, this paper proposes a Goal-Conditioned Probabilistic Model Predictive Control (GC-PMPC) framework. GC-PMPC integrates an asynchronous MPC architecture with an ensemble of probabilistic neural networks to explicitly model system uncertainty, while enabling joint optimization of tactile feedback and goal-conditioned policies. Evaluated on Shadow Hand simulation and the real-world DexHand 021 platform (12-DoF + 5 tactile channels), GC-PMPC achieves high-precision, real-time, adaptive manipulation of a cube toward arbitrary target poses—requiring only 80 minutes of training. It significantly outperforms state-of-the-art RL-based and deterministic MPC methods. The core contribution lies in the first deep integration of goal conditioning, probabilistic modeling, and asynchronous MPC—thereby simultaneously ensuring robustness, generalization across unseen goals, and sub-10-ms closed-loop response times.

Technology Category

• Intelligent Robots: Manipulation
• Reasoning under Uncertainty: Relational Probabilistic Models
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

This paper tackles the challenge of learning multi-goal dexterous hand manipulation tasks using model-based Reinforcement Learning. We propose Goal-Conditioned Probabilistic Model Predictive Control (GC-PMPC) by designing probabilistic neural network ensembles to describe the high-dimensional dexterous hand dynamics and introducing an asynchronous MPC policy to meet the control frequency requirements in real-world dexterous hand systems. Extensive evaluations on four simulated Shadow Hand manipulation scenarios with randomly generated goals demonstrate GC-PMPC's superior performance over state-of-the-art baselines. It successfully drives a cable-driven Dexterous hand, DexHand 021 with 12 Active DOFs and 5 tactile sensors, to learn manipulating a cubic die to three goal poses within approximately 80 minutes of interactions, demonstrating exceptional learning efficiency and control performance on a cost-effective dexterous hand platform.

Problem

• Learning multi-goal dexterous hand manipulation tasks
• High-dimensional dynamics modeling with probabilistic neural networks
• Real-time control for cost-effective dexterous hand systems

Innovation

• Goal-Conditioned Probabilistic Model Predictive Control
• Probabilistic neural network ensembles for dynamics
• Asynchronous MPC policy for real-time control