One Demonstration Is Enough for Real-World Robotic Reinforcement Learning

2026-07-01

AI Summary

Real-world robotic reinforcement learning faces significant challenges, including high data collection costs, difficulty in reward design, and heavy reliance on human intervention. This work proposes AutoSERL, a framework that, for the first time, enables fully automated reinforcement learning on physical robots using only a single demonstration. AutoSERL employs a sliding-window guided exploration strategy, a safety recovery mechanism based on predefined trajectories, and an automatic termination criterion to efficiently complete tasks and autonomously exit the guidance phase without continuous human oversight. Evaluated on six contact-rich manipulation tasks, AutoSERL achieves a 100% insertion success rate, demonstrates strong robustness to positional perturbations, and matches the performance of HIL-SERL—a method requiring substantial human-in-the-loop intervention.

Abstract

Learning effective robot control policies on physical hardware is challenging due to costly data collection and the difficulty of reward specification. Prior work has incorporated demonstrations into reinforcement learning (RL), yet existing approaches either require large numbers of demonstrations or depend on continuous human intervention during training. To address these limitations, we present AutoSERL, a framework that leverages a single demonstration to fully automate the intervention process in real-world robot RL. The framework includes three complementary mechanisms to accomplish certain tasks: a sliding window intervention mechanism that continuously guides exploration to prevent local optima and unsafe deviations, a safety recovery mechanism that detects and corrects failure states via predefined trajectory recovery points, and an intervention termination criterion that automatically disables guidance once the policy can independently complete the task, preserving its exploration advantage. We evaluate AutoSERL on six contact-intensive manipulation tasks across two robot platforms, spanning insertion, hanging, and hinge-based tasks. AutoSERL consistently outperforms SERL initialized with 20 demonstrations, behavior cloning, and MILES -- a dedicated one-shot imitation learning baseline -- across all tasks while matching HIL-SERL, achieves 100% success rate on insertion tasks, and demonstrates improved robustness to positional variations, all from a single demonstration. Code and videos are available on our project website: https://autoserl.github.io/.

Problem

• robotic reinforcement learning
• single demonstration
• real-world robotics
• reward specification
• human intervention

Innovation

• one-shot demonstration
• robotic reinforcement learning
• automated intervention
• safety recovery
• real-world robotics