Continual Robot Policy Learning via Variational Neural Dynamics

2026-06-25

AI Summary

In real-world environments, robots are frequently subjected to repetitive yet implicit dynamic disturbances—such as wind gusts, payload variations, and battery degradation—that challenge conventional one-time training strategies due to their inability to adapt online. This work proposes a continual learning framework that integrates analytical physical priors with a neural residual model to learn context-aware dynamics. A recurrent encoder infers the current latent state from recent interactions, enabling online adaptation of both the dynamics model and control policy. Crucially, the approach replaces online refitting with latent state identification, dramatically accelerating recovery under previously encountered disturbances. Evaluated on quadrotor wind-resilient trajectory tracking, the method achieves approximately fivefold faster disturbance recovery and reduces hover and tracking errors by 65.7% and 53.3%, respectively, under significant perturbations.

Abstract

Robots deployed in the real world rarely operate under a single fixed dynamics model: wind changes, payloads vary, batteries drain, contacts shift, and hardware wears. Yet most learning-based controllers are trained once and deployed as if learning were complete. This prevents the robot from using deployment experience to further improve task performance. In this work, we propose a continual learning framework that uses real-world experience to improve robot policies under hidden and recurring dynamics. Our method learns a condition-aware dynamics model from real state-action trajectories by combining an analytical physics prior with a neural residual for unmodeled effects. A recurrent encoder infers the current hidden condition from recent interaction, and this estimate conditions both the residual model and the policy. Policy learning is performed via differentiable simulation using diverse learned dynamics sampled from the latent model. At deployment, these sampled conditions are replaced by conditions inferred online from recent real interaction, allowing the policy to recover recurring dynamics by recognition rather than residual re-fitting. Through extensive simulation studies and real-world experiments, we demonstrate that the framework improves policy performance under diverse unobserved disturbances. On real quadrotor trajectory tracking under changing wind, the policy recovers from recurring disturbances in roughly 1s, about 5x faster than online residual re-fitting. It also reduces large-disturbance hover and tracking errors by 65.7% and 53.3% over the state-of-the-art online adaptation approaches

Problem

• continual learning
• robot policy
• dynamics adaptation
• real-world deployment
• unobserved disturbances

Innovation

• continual learning
• condition-aware dynamics
• differentiable simulation
• neural residual modeling
• online adaptation