Bootstrapping Imitation Learning for Long-horizon Manipulation via Hierarchical Data Collection Space

2025-05-23

AI Summary

To address the challenges of imitation learning in long-horizon robotic manipulation—namely, its heavy reliance on large-scale human demonstrations and poor generalization—this paper proposes the Hierarchical Data Collection Space (HD-Space) framework. HD-Space decomposes complex tasks into atomic subtasks and constructs a structured state-action space to enable high-quality, low-overhead demonstration generation. Crucially, it enhances demonstration robustness at the data collection source, enabling end-to-end policy training from only a small number of high-information demonstrations—thereby substantially reducing dependence on data volume and human intervention. Evaluated on two simulated and five real-world long-horizon manipulation tasks, HD-Space consistently outperforms mainstream baselines in success rate, demonstrating simultaneous improvements in data efficiency and policy performance.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Imitation learning (IL) with human demonstrations is a promising method for robotic manipulation tasks. While minimal demonstrations enable robotic action execution, achieving high success rates and generalization requires high cost, e.g., continuously adding data or incrementally conducting human-in-loop processes with complex hardware/software systems. In this paper, we rethink the state/action space of the data collection pipeline as well as the underlying factors responsible for the prediction of non-robust actions. To this end, we introduce a Hierarchical Data Collection Space (HD-Space) for robotic imitation learning, a simple data collection scheme, endowing the model to train with proactive and high-quality data. Specifically, We segment the fine manipulation task into multiple key atomic tasks from a high-level perspective and design atomic state/action spaces for human demonstrations, aiming to generate robust IL data. We conduct empirical evaluations across two simulated and five real-world long-horizon manipulation tasks and demonstrate that IL policy training with HD-Space-based data can achieve significantly enhanced policy performance. HD-Space allows the use of a small amount of demonstration data to train a more powerful policy, particularly for long-horizon manipulation tasks. We aim for HD-Space to offer insights into optimizing data quality and guiding data scaling. project page: https://hd-space-robotics.github.io.

Problem

• Improves imitation learning for long-horizon robotic manipulation tasks
• Reduces high data collection costs in human-in-loop processes
• Enhances policy performance with hierarchical data collection space

Innovation

• Hierarchical Data Collection Space for IL
• Segment tasks into atomic state/action spaces
• Enhance policy with minimal high-quality data