How to utilize failure demo data?: Effective data selection for imitation learning using distribution differences in attention mechanism

2026-05-08

AI Summary

This work addresses the challenge of effectively leveraging inevitably collected failed demonstrations in imitation learning. The authors propose a novel approach that learns a latent representation capturing the discrepancy between successful and failed demonstrations and integrates this representation into an attention mechanism. During inference, the model adaptively selects a policy mode based on the initial observation, thereby enhancing action stability. Additionally, they introduce a post-training metric that requires no additional environment interaction; it automatically identifies beneficial failure examples by measuring divergence in attention distributions. This method is the first to incorporate the distributional difference between success and failure into an attention-based mode selection framework, significantly improving both task success rates in simulation and the utilization efficiency of demonstration data.

Abstract

Imitation learning for robotic tasks has relied primarily on policies trained only on successful demonstrations, although failures are unavoidable during human data collection. Many existing approaches for exploiting failure data require additional data processing or iterative policy updates through autonomous rollouts, making it difficult to directly and stably utilize failure data accumulated during data collection. In this work, we propose a method that learns latent representations of success-failure discrepancies and incorporates them into the attention mechanism. During inference, an appropriate latent mode is selected from the initial observation to improve action stability. Furthermore, we introduce a post-training metric that quantifies the attention discrepancy between each failure sample and successful demonstrations to select failure data. Simulation results show that the proposed method improves task success rates when trained with failure data and that the proposed metric identifies failure samples that are beneficial for learning when combined with successful demonstrations. These results suggest that the proposed method can support more efficient use of collected demonstrations in robotic data collection pipelines.

Problem

• imitation learning
• failure demonstrations
• data selection
• robotic tasks
• attention mechanism

Innovation

• imitation learning
• failure demonstrations
• attention mechanism
• distribution discrepancy
• data selection