Adaptation of Generalist Robot Policies with Minimal Data

2026-08-11

AI Summary

This work addresses the challenge of efficiently adapting pretrained general-purpose robotic policies to novel tasks using only a single human demonstration while enabling reliable autonomous learning. The authors propose MiDAS, a method that first anchors a vision-language-action (VLA) model to the target task via behavioral cloning and then refines the policy through online reinforcement learning with a residual policy parameterization. This approach enables robust policy improvement from extremely sparse demonstrations. MiDAS is the first method to reliably guide policy adaptation from just one demonstration, recovering high performance on the LIBERO and RoboCasa benchmarks and significantly enhancing the success rate and robustness of the initial policy on a real-world YAM bimanual robot platform after approximately six hours of online interaction.

Abstract

A central goal in robot learning is to move beyond task-specific human data collection toward robots that improve through autonomous interaction. Yet fully autonomous learning remains difficult with current policies: sparse rewards and weak zero-shot exploration make it unlikely that a robot will discover successful behavior from scratch. We study minimal-data adaptation, a regime in which a pre-trained robot policy must learn a new task from as little as one demonstration followed by autonomous online interaction. This setting serves as the closest tractable proxy for fully autonomous improvement, allowing us to study whether minimal human guidance can bootstrap autonomous learning and what algorithmic ingredients make it feasible. We build MiDAS, a simple offline-to-online RL recipe that first anchors a pre-trained VLA to the target task with behavior cloning on single/few demonstrations, then improves it through value-based online RL on a residual policy parameterization. Across LIBERO and RoboCasa, MiDAS recovers strong task performance from as little as one demonstration, substantially outperforming baselines and generalizing beyond demonstrated conditions. We further evaluate MiDAS on a bimanual YAM platform. Starting from a fragile low-success policy obtained from a single demonstration, MiDAS improves its robustness and learns new successful behaviors over ~6 hours of online interaction. To the best of our knowledge, this is the first demonstration of reliable robot policy adaptation from a single task demonstration.

Problem

• minimal-data adaptation
• robot learning
• autonomous learning
• policy adaptation
• few-shot demonstration

Innovation

• minimal-data adaptation
• offline-to-online RL
• residual policy learning
• vision-language-action models
• robot policy generalization