DemoGen: Synthetic Demonstration Generation for Data-Efficient Visuomotor Policy Learning

2025-02-24

AI Summary

Visual motor policies often require extensive human demonstrations and exhibit poor spatial generalization. To address this, we propose a novel method that generates high-fidelity synthetic demonstrations from only a single real-world demonstration. Our approach introduces geometry-consistent data synthesis grounded in 3D point cloud editing, integrated with action trajectory space adaptation, scene-level 3D editing, and geometry-aware action transfer—enabling robust manipulation of deformable objects, dexterous hands, and bimanual platforms. The method significantly improves policy performance across diverse real-world tasks while enhancing out-of-distribution robustness, including interference resistance and obstacle avoidance. Experiments demonstrate that, using just one real demonstration per task, our method achieves generalization performance comparable to conventional approaches relying on dozens to hundreds of manually collected demonstrations.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Computational Photography, Image & Video Synthesis
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Visuomotor policies have shown great promise in robotic manipulation but often require substantial amounts of human-collected data for effective performance. A key reason underlying the data demands is their limited spatial generalization capability, which necessitates extensive data collection across different object configurations. In this work, we present DemoGen, a low-cost, fully synthetic approach for automatic demonstration generation. Using only one human-collected demonstration per task, DemoGen generates spatially augmented demonstrations by adapting the demonstrated action trajectory to novel object configurations. Visual observations are synthesized by leveraging 3D point clouds as the modality and rearranging the subjects in the scene via 3D editing. Empirically, DemoGen significantly enhances policy performance across a diverse range of real-world manipulation tasks, showing its applicability even in challenging scenarios involving deformable objects, dexterous hand end-effectors, and bimanual platforms. Furthermore, DemoGen can be extended to enable additional out-of-distribution capabilities, including disturbance resistance and obstacle avoidance.

Problem

• Generates synthetic demonstrations for visuomotor policies
• Reduces need for extensive human-collected data
• Enhances policy performance in real-world tasks

Innovation

• Synthetic demonstration generation
• 3D point cloud synthesis
• Spatial trajectory adaptation