🤖 AI Summary
To address the abrupt degradation of control performance caused by sudden system dynamics changes, this paper proposes a reflective world model (RWM) dual-control framework. RWM directly utilizes the deterministic world model’s predictive output as an implicit, dynamically adaptive reference trajectory, thereby decoupling long-horizon reward optimization from millisecond-level latent-space motion execution. It introduces, for the first time, a synergistic dual-control architecture integrating latent-space model predictive control, online latent-action correction, and hierarchical reinforcement learning. Evaluated on high-dimensional continuous control tasks, RWM achieves second-scale dynamics adaptation, reduces computational overhead by 87%, and maintains policy performance above 98% of the optimal—significantly outperforming existing model-based baselines.
📝 Abstract
Deploying learned control policies in real-world environments poses a fundamental challenge. When system dynamics change unexpectedly, performance degrades until models are retrained on new data. We introduce Reflexive World Models (RWM), a dual control framework that uses world model predictions as implicit reference trajectories for rapid adaptation. Our method separates the control problem into long-term reward maximization through reinforcement learning and robust motor execution through rapid latent control. This dual architecture achieves significantly faster adaptation with low online computational cost compared to model-based RL baselines, while maintaining near-optimal performance. The approach combines the benefits of flexible policy learning through reinforcement learning with rapid error correction capabilities, providing a principled approach to maintaining performance in high-dimensional continuous control tasks under varying dynamics.