RECALL: Recovery Experience Collection for Active Lifelong Learning in Vision-Language-Action Models

2026-06-22

AI Summary

This work addresses the inefficiencies of traditional vision-language-action (VLA) models, which rely on passive imitation learning—collecting data only after task failures—and consequently suffer from poor utilization of demonstration resources and susceptibility to catastrophic forgetting due to the absence of proactive guidance at critical states. To overcome these limitations, the authors propose an uncertainty-guided active continual learning paradigm that, for the first time, integrates active data collection into VLA models. By estimating model uncertainty, the approach selectively acquires corrective demonstrations and combines them with experience replay and elastic weight consolidation to simultaneously enhance fine-tuning efficiency and balance the retention of previously learned tasks with the assimilation of new knowledge. Experimental results demonstrate that this method significantly outperforms passive data collection strategies, achieving a superior trade-off between adaptation efficiency and resistance to forgetting.

Abstract

Vision-Language-Action (VLA) models are commonly fine-tuned through passive imitation learning, where additional demonstrations are collected for tasks where the policy performs poorly. This approach incurs several downsides: it requires the robot to fail before data collection is triggered, provides little guidance about which states require supervision, and wastes demonstrator effort on redundant parts of the task where the policy already performs well. In this paper, we propose an active, continual learning paradigm for VLAs. We demonstrate that active, uncertainty-guided data collection leads to more efficient fine-tuning than when using passively-collected demonstrations. However, we also find that fine-tuning only on actively-collected recovery data leads to catastrophic forgetting. We evaluate techniques for continual learning, including replay-based data mixing and elastic weight consolidation, and identify tradeoffs between plasticity to uncertainty-guided recovery data and retention of previously learned behaviors. Overall, our work contributes an empirical study of active continual learning for autoregressive VLAs, establishing that uncertainty-guided recovery demonstrations can improve adaptation efficiency while also revealing open challenges when targeted new data is incorporated into large robot policies.

Problem

• Vision-Language-Action models
• active learning
• continual learning
• catastrophic forgetting
• uncertainty-guided data collection

Innovation

• active learning
• continual learning
• vision-language-action models
• uncertainty-guided data collection
• catastrophic forgetting