Physics-Driven Data Generation for Contact-Rich Manipulation via Trajectory Optimization

2025-02-27

AI Summary

Constructing high-quality, cross-hardware datasets for contact-rich robotic manipulation remains challenging due to high data acquisition costs and hardware-specific constraints. Method: This paper proposes a low-cost, high-fidelity multi-source data generation framework. It introduces the first “embodiment-flexible” VR-based human demonstration capture system, integrated with physics simulation, kinematic retargeting, and constraint-aware, multi-body parameter-adaptive trajectory optimization—enabling cross-configuration data reuse and zero-shot transfer. Contribution/Results: Using this pipeline, we build a large-scale contact manipulation dataset and train a diffusion-based policy with strong generalization. Evaluated on Allegro Hand and dual-arm iiwa platforms, the policy achieves zero-shot deployment on real dual-arm iiwa hardware, significantly improving success rates on contact-intensive tasks with minimal human intervention. Key contributions are the embodiment-flexible data collection paradigm and the physically consistent, parameter-adaptive optimization mechanism.

Technology Category

• Intelligent Robots: Manipulation
• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Low Level & Physics-based Vision

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We present a low-cost data generation pipeline that integrates physics-based simulation, human demonstrations, and model-based planning to efficiently generate large-scale, high-quality datasets for contact-rich robotic manipulation tasks. Starting with a small number of embodiment-flexible human demonstrations collected in a virtual reality simulation environment, the pipeline refines these demonstrations using optimization-based kinematic retargeting and trajectory optimization to adapt them across various robot embodiments and physical parameters. This process yields a diverse, physically consistent dataset that enables cross-embodiment data transfer, and offers the potential to reuse legacy datasets collected under different hardware configurations or physical parameters. We validate the pipeline's effectiveness by training diffusion policies from the generated datasets for challenging contact-rich manipulation tasks across multiple robot embodiments, including a floating Allegro hand and bimanual robot arms. The trained policies are deployed zero-shot on hardware for bimanual iiwa arms, achieving high success rates with minimal human input. Project website: https://lujieyang.github.io/physicsgen/.

Problem

• Generates datasets for robotic manipulation
• Enables cross-embodiment data transfer
• Trains policies for contact-rich tasks

Innovation

• Physics-based simulation integration
• Optimization-based kinematic retargeting
• Cross-embodiment data transfer