Restoring Noisy Demonstration for Imitation Learning With Diffusion Models

2025-10-16

AI Summary

Existing imitation learning (IL) methods assume expert demonstrations are perfect; however, real-world demonstrations often contain noise from human imperfections or sensor/actuation errors, leading to substantial performance degradation. To address this, we propose a “filter-and-recover” framework: first, a filtering strategy isolates high-quality demonstration segments; second, we introduce conditional diffusion models—novel in IL—to jointly model and reconstruct the kinematic and dynamic structure of the remaining noisy trajectories. This two-stage mechanism exhibits strong robustness against diverse noise types and intensities. We evaluate the framework on end-to-end learning for robotic manipulation and motion control tasks. Experiments demonstrate significant improvements over state-of-the-art methods under high-noise conditions, validating the framework’s effectiveness, generalizability, and practical utility.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: State Estimation
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Imitation learning (IL) aims to learn a policy from expert demonstrations and has been applied to various applications. By learning from the expert policy, IL methods do not require environmental interactions or reward signals. However, most existing imitation learning algorithms assume perfect expert demonstrations, but expert demonstrations often contain imperfections caused by errors from human experts or sensor/control system inaccuracies. To address the above problems, this work proposes a filter-and-restore framework to best leverage expert demonstrations with inherent noise. Our proposed method first filters clean samples from the demonstrations and then learns conditional diffusion models to recover the noisy ones. We evaluate our proposed framework and existing methods in various domains, including robot arm manipulation, dexterous manipulation, and locomotion. The experiment results show that our proposed framework consistently outperforms existing methods across all the tasks. Ablation studies further validate the effectiveness of each component and demonstrate the framework's robustness to different noise types and levels. These results confirm the practical applicability of our framework to noisy offline demonstration data.

Problem

• Restoring noisy expert demonstrations for imitation learning
• Addressing imperfections in expert demonstrations from humans
• Filtering and recovering noisy samples using diffusion models

Innovation

• Filter-and-restore framework for noisy demonstrations
• Conditional diffusion models recover noisy samples
• Outperforms existing methods across multiple tasks