Scalable Behavior Cloning with Open Data, Training, and Evaluation

2026-06-25

AI Summary

Existing behavior cloning research lacks open and reproducible infrastructure, resulting in high barriers to policy development and hindering fair comparisons. This work proposes ABC—an end-to-end open-source behavior cloning framework that includes the large-scale teleoperated dataset ABC-130K (comprising 400 hours of simulated and real-world data), open-source hardware designs, a training framework, and a simulation pipeline. The framework innovatively integrates sim-to-real co-training and joint evaluation mechanisms. Leveraging the Diffusion Transformer (DiT) and vision-language-action (VLA) architectures, policies trained within this ecosystem demonstrate strong performance and generalization on dexterous manipulation tasks such as folding cardboard boxes and retrieving cards from wallets, thereby validating the effectiveness of the proposed infrastructure.

Abstract

We introduce ABC, a fully open-source stack for manipulation with behavior cloning. At its core is ABC-130K: the largest open-source teleoperation dataset to date, featuring 3,500 hours of data spanning over 130K episodes across 195 diverse tasks. Furthermore, we open-source our accessible hardware setup, training infrastructure, and simulation pipeline. We also release 400 hours of sim-teleop data and provide a co-training recipe that produces correlated simulation and real-world evaluation, offering a reliable proxy for ablating model-design and training decisions before costly real-world evaluation. We explore various training recipes and compare common architectural choices for Diffusion Transformers (DiT) and Vision-Language-Action (VLA) models, grounding our findings in real-world evaluations. The resulting policies successfully execute dexterous tasks such as box folding and extracting credit cards from wallets. By providing a reproducible toolkit, we aim to place researchers on an equal footing, establishing the necessary foundation to learn the ABCs of Behavior Cloning together as a community.

Problem

• Behavior Cloning
• Scalable Robotics
• Open-source Dataset
• Manipulation Tasks
• Reproducible Research

Innovation

• Behavior Cloning
• Open-source Robotics
• Teleoperation Dataset
• Co-training
• Diffusion Transformer