World Action Models Enable Continual Imitation Learning with Recurrent Generative Replays

2026-06-25

AI Summary

This work addresses catastrophic forgetting in continual imitation learning, where human demonstrations cannot be stored for replay. To overcome this limitation, the authors propose a Recursive Generative Replay (REGEN) framework that leverages a World Action Model (WAM) to synthesize pseudo-replay trajectories solely from task instructions and current observations, thereby enabling continual learning without retaining original demonstrations. By modeling conditional trajectory generation and enforcing visual-action consistency, REGEN effectively reconstructs past task experiences while acquiring new skills. Experimental results in both simulated and real-world environments demonstrate that REGEN reduces forgetting by up to 50% compared to sequential fine-tuning and achieves performance approaching that of privileged methods relying on actual replay data.

Abstract

Going beyond predicting robot actions, World Action Models (WAMs) can also generate future visual observations. We build on this generative capability to propose Recurrent Generative Replay (REGEN), a continual imitation learning framework that synthesizes pseudo-replay trajectories, enabling a robot policy to rehearse previously learned tasks without storing their original human demonstrations. During continual adaptation, REGEN recursively queries the WAM to synthesize pseudo-replay trajectories conditioned only on prior task instructions and current-task observations. Experiments in both simulation and real-world manipulation settings show that REGEN reduces catastrophic forgetting by up to $50\%$ relative to sequential fine-tuning, while approaching the performance of privileged experience replay methods that require access to real replay data. Finally, we analyze the factors limiting generated replay, identifying long-horizon visual degradation and action-observation inconsistency as the primary bottlenecks. Our results establish WAMs as a promising foundation for continual robot learning without stored demonstrations.

Problem

• continual imitation learning
• catastrophic forgetting
• replay without stored demonstrations
• robot learning

Innovation

• World Action Models
• Recurrent Generative Replay
• Continual Imitation Learning
• Pseudo-replay Trajectories
• Catastrophic Forgetting