Can Real-to-Sim Approaches Capture Dynamic Fabric Behavior for Robotic Fabric Manipulation?

2025-03-20

AI Summary

Real-to-Sim parameter estimation for dynamic cloth modeling in robotic manipulation remains underexplored, particularly regarding robustness across diverse cloth types, real-world dynamics (e.g., lifting, wind, stretching), and generalization to unseen tasks (e.g., folding, flinging, shaking). Method: We propose the first physics-informed neural network (PINN) framework for cloth physical parameter estimation, integrating differential modeling, data-driven regression, and multi-engine simulation (PyBullet/Isaac Gym), optimized jointly on motion trajectories and force feedback. Contribution/Results: We introduce the first cross-simulator, multi-cloth, multi-task Real-to-Sim benchmark for evaluating robustness and generalization. Experiments across five cloth types and six dynamic tasks show that simulator choice and estimation method significantly impact manipulation performance; PINN achieves highest accuracy in quasi-static regimes but exhibits phase lag under high-frequency dynamics. The best-performing method reduces average error by 27.4% on unseen tasks compared to baselines.

Technology Category

• Intelligent Robots: Manipulation
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

This paper presents a rigorous evaluation of Real-to-Sim parameter estimation approaches for fabric manipulation in robotics. The study systematically assesses three state-of-the-art approaches, namely two differential pipelines and a data-driven approach. We also devise a novel physics-informed neural network approach for physics parameter estimation. These approaches are interfaced with two simulations across multiple Real-to-Sim scenarios (lifting, wind blowing, and stretching) for five different fabric types and evaluated on three unseen scenarios (folding, fling, and shaking). We found that the simulation engines and the choice of Real-to-Sim approaches significantly impact fabric manipulation performance in our evaluation scenarios. Moreover, PINN observes superior performance in quasi-static tasks but shows limitations in dynamic scenarios.

Problem

• Evaluates Real-to-Sim parameter estimation for fabric manipulation.
• Compares three approaches: two differential pipelines, one data-driven.
• Tests performance across multiple fabric types and dynamic scenarios.

Innovation

• Evaluates Real-to-Sim parameter estimation methods
• Introduces physics-informed neural network approach
• Tests across multiple fabric types and scenarios