Learning to Design Soft Hands using Reward Models

2025-10-19

AI Summary

Soft robotic hands face significant challenges in compliant, multifunctional design and morphology-control co-design due to high-dimensional search spaces and prohibitive computational costs. To address this, we propose CEM-RM—a framework integrating a teleoperation-data-driven reward model with cross-entropy optimization for joint, parallelized simulation-based optimization of both morphology and controller within a compliant-joint soft-finger design space. Our method drastically reduces the number of design evaluations required, enables generation of diverse high-performance configurations, and enhances cross-task generalization. Simulation and physical experiments demonstrate that the optimized soft hands achieve substantially higher success rates in complex object grasping compared to baseline designs, validating their strong adaptability and practical utility.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

Soft robotic hands promise to provide compliant and safe interaction with objects and environments. However, designing soft hands to be both compliant and functional across diverse use cases remains challenging. Although co-design of hardware and control better couples morphology to behavior, the resulting search space is high-dimensional, and even simulation-based evaluation is computationally expensive. In this paper, we propose a Cross-Entropy Method with Reward Model (CEM-RM) framework that efficiently optimizes tendon-driven soft robotic hands based on teleoperation control policy, reducing design evaluations by more than half compared to pure optimization while learning a distribution of optimized hand designs from pre-collected teleoperation data. We derive a design space for a soft robotic hand composed of flexural soft fingers and implement parallelized training in simulation. The optimized hands are then 3D-printed and deployed in the real world using both teleoperation data and real-time teleoperation. Experiments in both simulation and hardware demonstrate that our optimized design significantly outperforms baseline hands in grasping success rates across a diverse set of challenging objects.

Problem

• Optimizing tendon-driven soft robotic hand designs efficiently
• Reducing computational cost of hardware-control co-design evaluations
• Improving grasping success rates across diverse challenging objects

Innovation

• Uses Cross-Entropy Method with Reward Model
• Optimizes tendon-driven soft hand designs
• Reduces evaluations via teleoperation data