DemoDiffusion: One-Shot Human Imitation using pre-trained Diffusion Policy

2025-06-25

AI Summary

This work addresses one-shot imitation learning: enabling robots to robustly execute diverse manipulation tasks in natural environments from a single human demonstration. We propose a synergistic framework combining kinematic motion retargeting with a context-aware pre-trained diffusion policy. First, an initial trajectory is generated via kinematic retargeting; then, the diffusion policy refines it through denoising optimization, modeling high-dimensional action distributions and producing open-loop control sequences compatible with robot dynamics. Crucially, our approach requires no online reinforcement learning, paired data, or task-specific fine-tuning. It significantly improves cross-scenario generalization. In both simulation and real-robot experiments, our method outperforms direct retargeting and existing baselines, successfully completing tasks on which the pre-trained diffusion policy originally failed. These results demonstrate its effectiveness, robustness, and practical applicability.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Computer Vision: Diffusion Models for Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

We propose DemoDiffusion, a simple and scalable method for enabling robots to perform manipulation tasks in natural environments by imitating a single human demonstration. Our approach is based on two key insights. First, the hand motion in a human demonstration provides a useful prior for the robot's end-effector trajectory, which we can convert into a rough open-loop robot motion trajectory via kinematic retargeting. Second, while this retargeted motion captures the overall structure of the task, it may not align well with plausible robot actions in-context. To address this, we leverage a pre-trained generalist diffusion policy to modify the trajectory, ensuring it both follows the human motion and remains within the distribution of plausible robot actions. Our approach avoids the need for online reinforcement learning or paired human-robot data, enabling robust adaptation to new tasks and scenes with minimal manual effort. Experiments in both simulation and real-world settings show that DemoDiffusion outperforms both the base policy and the retargeted trajectory, enabling the robot to succeed even on tasks where the pre-trained generalist policy fails entirely. Project page: https://demodiffusion.github.io/

Problem

• Enable robots to imitate human tasks with one demonstration
• Convert human hand motion to robot trajectory via retargeting
• Modify trajectory using diffusion policy for plausible robot actions

Innovation

• Kinematic retargeting converts human motion to robot trajectory
• Pre-trained diffusion policy refines trajectory for robot actions
• No online reinforcement learning or paired data required