Latent Energy Action Planning with World Models

2026-09-02

AI Summary

研究解决了高维观测下模型预测控制的问题，通过引入LEAP方法优化动作序列，结合目标匹配和状态能量，提高了控制成功率。

Abstract

Latent world models support efficient model predictive control from high-dimensional observations, yet optimizing a single learned latent objective can favor action sequences whose decoder-predicted terminal descriptor does not match the goal descriptor. We introduce Latent Energy Action Planning (LEAP), which treats the complete action horizon as a differentiable variable and optimizes it through a frozen LeWorldModel (LeWM). LEAP couples terminal latent goal matching with a terminal-window state energy. Low energy requires the predicted terminal latent to agree with the goal latent and the decoder-predicted terminal descriptor to agree with the goal descriptor. A frozen goal-conditioned proposal initializes the search, a quasi-Newton solver refines actions through the autoregressive rollout, and post-optimization projection enforces the admissible action range. Across four control domains using the officially released LeWM checkpoints, the complete LEAP planning system raises mean success from 77.5% for LeWM planned with the cross-entropy method (LeWM+CEM) to 94.8% under a matched protocol, a 17.3-percentage-point improvement, while retaining the frozen LeWM representation and predictor.

Problem

• Latent World Models
• Model Predictive Control
• Action Sequences
• Goal Descriptor Mismatch

Innovation

• Latent Energy Action Planning
• LeWM
• differentiable variable
• terminal latent goal matching
• quasi-Newton solver